1.. _definition_and_criteria:
2
3Sample Definition and Criteria
4##############################
5
6Definition
7==========
8
9A sample is a concise Zephyr application that provides an accessible overview of one or
10more features, subsystems, or modules. This includes kernel services, drivers, protocols, etc.
11Samples should be limited in scope and should focus only on demonstrating non-trivial or
12essential aspects of the subsystem(s) or module(s) being highlighted.
13
14Samples are recommended when submitting new features; however, they are not mandatory.
15
16Sample Criteria
17===============
18
191. Samples must not be used to test features or verify platforms
20++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
21  * The primary purpose of a sample is to provide a reference to the user.
22  * Samples must not use Zephyr's testing framework.
23
24    * Must not use :kconfig:option:`CONFIG_ZTEST`
25    * Must not use zasserts in samples.
26
27  * If a sample can provide output that can be verified, then output should be evaluated against
28    expected value to have some level of testing for the sample itself.
29    Refer to :ref:`twister_script` for more details.
30
31  * Although being able to run a sample successfully does verify the
32    functionality is working as expected, this should not replace dedicated and
33    comprehensive tests with full coverage to be submitted into the
34    :zephyr_file:`tests/` folder.  In a sample, the only thing you test is
35    buildability and, in some cases, if a sample is performing as expected, i.e. you
36    are testing the sample, not the subsystem it builds on top.
37
382. Twister should be able to build every sample.
39++++++++++++++++++++++++++++++++++++++++++++++++
40  * Every sample must have a YAML file. Reference: :ref:`twister_script`.
41
42    **For example:**
43
44    .. code-block:: yaml
45
46      tests:
47        sample.kernel.cond_var:
48          integration_platforms:
49            - native_sim
50          tags:
51            - kernel
52            - condition_variables
53          harness: console
54          harness_config:
55            type: one_line
56            regex:
57              - ".*Waited and joined with 3 threads"
58
59  * Do not mark the test as build only. A sample should be able to build *and*
60    run on the documented platforms. Use the ``harness`` option to skip the
61    execution. Twister only attempts to flash and execute the build binary if
62    the harness is set to ``ztest`` or ``console``.
63  * The default configuration for the sample must be in the :file:`prj.conf`
64    file and should be operational on all supported platforms. Do not rely on the
65    ``extra_args`` or ``extra_configs`` options in the YAML file to build the
66    sample.
67  * The tests should verify the default configuration of the sample and any
68    optional features documented in the sample's README file. Sample should
69    never be used to test functionality of the subsystem or module the sample
70    belongs to.
71  * Any documented configuration options related to the sample and its
72    operation can be verified using ``extra_args`` or
73    ``extra_configs`` options in the YAML file. The :file:`prj.conf` file should have the
74    base configuration options.
75  * Sample output can be validated by leveraging the ``harness_config`` regex option,
76    wherever applicable.
77  * Use ``platform_allow`` to limit test to the platforms the sample was actually
78    verified on. Those platforms should be listed in the sample's README file.
79  * Make use of ``integration_platforms`` to limit the scope when there are timing or
80    resource constraints. Ideally, one platform should be enough to verify the
81    sample when changes are submitted to the Zephyr tree via a pull-request.
82  * Make the sample as generic as possible. Avoid making a sample platform specific unless it is
83    for particular hardware.
84
853. A sample should provide sufficient coverage of a subsystem, feature, or module.
86++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
87  **DO's:**
88    * Cover the most common and important use cases of the functionality.
89    * Keep the code simple and easy to read. Example: :zephyr_file:`samples/philosophers`.
90
91  **DONT's:**
92    * Samples must not test the negative or edge case behaviors.
93    * Must not be unit tests.
94
954. Samples must be documented.
96++++++++++++++++++++++++++++++
97  * Samples must have a ``README.rst`` file in the samples folder.
98    Example: ``samples/subsys/foo/README.rst``. clearly explaining the purpose of the sample, its
99    hardware requirements, and the expected sample output, if applicable.
100  * Ensure that the ``README.rst`` file is accessible in the sample hierarchy starting at
101    ``samples/index.rst``.
102
103  **README Template:**
104    * Overview, if applicable.
105    * Software/Hardware requirements
106    * Building & Running instructions
107    * Sample output, if applicable.
108
109
110As a starting point, this sample is a good example to refer to
111:zephyr_file:`samples/kernel/condition_variables/condvar`.
112