Home
last modified time | relevance | path

Searched +full:twister +full:- +full:test +full:- +full:results (Results 1 – 25 of 40) sorted by relevance

12

/Zephyr-latest/.github/workflows/
Dtwister.yaml1 name: Run tests with twister
6 - main
7 - v*-branch
8 - collab-*
11 - main
12 - v*-branch
13 - collab-*
16 - cron: '0 17 * * 6'
19 group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.head_ref || github.ref }}
20 cancel-in-progress: true
[all …]
Dtwister-publish.yaml1 name: Publish Twister Test Results
5 workflows: ["Run tests with twister"]
7 - main
9 - completed
12 upload-to-elasticsearch:
14 github.repository == 'zephyrproject-rtos/zephyr' &&
19 runs-on: ubuntu-22.04
22 - name: Checkout
25 fetch-depth: 0
26 persist-credentials: false
[all …]
Dbsim-tests.yaml6 - ".github/workflows/bsim-tests.yaml"
7 - ".github/workflows/bsim-tests-publish.yaml"
8 - "west.yml"
9 - "subsys/bluetooth/**"
10 - "tests/bsim/**"
11 - "boards/nordic/nrf5*/*dt*"
12 - "dts/*/nordic/**"
13 - "tests/bluetooth/common/testlib/**"
14 - "samples/bluetooth/**"
15 - "boards/native/**"
[all …]
/Zephyr-latest/scripts/ci/es_upload/
DREADME.md5 script for [Twister](https://docs.zephyrproject.org/latest/develop/test/twister.html)
9 with explicit scheme having all required Twister JSON report objects associated
11 Besides, it allows to track changes in Twister JSON report scheme
17 For other command line options and more details run the upload script with `--help`,
21 in the current Twister report format and in the upload script itself.
24 It is recommended to try the upload script with `--dry-run` option first
36 python3 ./scripts/ci/upload_test_results_es.py --create-index \
37 --index YOUR_INDEX_NAME \
38 --map-file YOUR_INDEX_MAP.json
44 The upload script has several command line options to change `twister.json` data:
[all …]
/Zephyr-latest/doc/develop/test/
Dtwister_statuses.rst3 Twister Status
6 What is a Twister Status?
9 Twister Status formulates the current state of
11 - ``Harness``
12 - ``TestCase``
13 - ``TestSuite``
14 - ``TestInstance``
16 in a comprehensive and easy-to understand way.
18 of Instances and Cases after the conclusion of their Twister runs.
28 ``Harness`` is a Python class inside Twister that allows us
[all …]
Dpytest.rst3 Integration with pytest test framework
6 *Please mind that integration of twister with pytest is still work in progress. Not every platform
17 A pytest plugin ``pytest-twister-harness`` was introduced to provide an integration between pytest
18 and twister, allowing Zephyr’s community to utilize pytest functionality with keeping twister as
21 Integration with twister
24 By default, there is nothing to be done to enable pytest support in twister. The plugin is
25 developed as a part of Zephyr’s tree. To enable install-less operation, twister first extends
27 ``-p twister_harness.plugin`` argument. If one prefers to use the installed version of the plugin,
28 they must add ``--allow-installed-plugin`` flag to twister’s call.
30 Pytest-based test suites are discovered the same way as other twister tests, i.e., by a presence
[all …]
Dtwister.rst3 Test Runner (Twister)
6 Twister scans for the set of test applications in the git repository
7 and attempts to execute them. By default, it tries to build each test
10 The default options will build the majority of the test applications on a
14 Because of the limited test execution coverage, twister
20 When using (at least) one ``-v`` option, twister's console output
21 shows for every test application how the test is run (qemu, native_sim, etc.) or
24 of a test is likewise reported in the ``twister.json`` and other report files.
25 There are a few reasons why twister only builds a test and doesn't run it:
27 - The test is marked as ``build_only: true`` in its ``.yaml``
[all …]
Dbsim.rst9 In the Zephyr project we use the `Babblesim`_ simulator to test some of the Zephyr radio protocols,
32 Tests without radio activity: bsim tests with twister
37 be used just like :ref:`native_sim<native_sim>` with :ref:`twister <twister_script>`,
38 to run all standard Zephyr twister tests, but with models of a real SOC HW, and their drivers.
45 with twister, but with a dedicated set of tests scripts.
57 built with some checks which decide if the test is passing or failing. These embedded
61 * Test using the EDTT_ tool, in which a EDTT (python) test controls the embedded applications over
62 an RPC mechanism, and decides if the test passes or not.
63 Today these tests include a very significant subset of the BT qualification test suite.
68 Test coverage and BabbleSim
[all …]
Dztest.rst1 .. _test-framework:
3 Test Framework
6 The Zephyr Test Framework (Ztest) provides a simple testing framework intended
8 test structure.
13 Creating a test suite
16 Using Ztest to create a test suite is as easy as calling the :c:macro:`ZTEST_SUITE`. The macro
19 * ``suite_name`` - The name of the suite. This name must be unique within a single binary.
20 * :c:type:`ztest_suite_predicate_t` - An optional predicate function to allow choosing when the
21 test will run. The predicate will get a pointer to the global state passed in through
23 * :c:type:`ztest_suite_setup_t` - An optional setup function which returns a test fixture. This
[all …]
/Zephyr-latest/tests/benchmarks/sched_queues/
DKconfig2 # SPDX-License-Identifier: Apache-2.0
12 This option specifies the number of times each test will be executed
19 This option specifies the maximum number of threads that the test
28 Log summary statistics as records to pass results
29 to the Twister JSON report and recording.csv file(s).
32 bool "Display detailed results"
/Zephyr-latest/tests/benchmarks/wait_queues/
DKconfig2 # SPDX-License-Identifier: Apache-2.0
12 This option specifies the number of times each test will be executed
19 This option specifies the maximum number of threads that the test
28 Log summary statistics as records to pass results
29 to the Twister JSON report and recording.csv file(s).
32 bool "Display detailed results"
/Zephyr-latest/doc/develop/test/twister/
Dtwister_blackbox.rst3 Twister blackbox tests
6 This guide aims to explain the structure of a test file so the reader will be able
9 important for any Twister developer.
14 Twister blackbox tests are written in python, using the ``pytest`` library.
16 Auxiliary test data follows whichever format it was in originally.
20 Blackbox tests should not be aware of the internal twister code. Instead, they should
21 call twister as user would and check the results.
23 Sample test file
33 Test above runs the command
35 .. code-block:: console
[all …]
/Zephyr-latest/scripts/pylib/twister/twisterlib/
Dreports.py5 # SPDX-License-Identifier: Apache-2.0
19 logger = logging.getLogger('twister')
42 'twister.json': {
51 def __init__(self, plan, env) -> None:
69 log = f.read().decode("utf-8")
133 message="No results captured, testsuite misconfiguration?"
323 f"Skip test suite '{instance.testsuite.name}'"
330 f"Skip test suite '{instance.testsuite.name}'"
498 # name, datatype, lower results better
506 results = []
[all …]
Denvironment.py4 # Copyright (c) 2018-2025 Intel Corporation
8 # SPDX-License-Identifier: Apache-2.0
29 logger = logging.getLogger('twister')
45 def _get_installed_packages() -> Generator[str, None, None]:
57 print(f"Currently, Twister requires at least Python {min_ver_str}.")
63 PYTEST_PLUGIN_INSTALLED = 'pytest-twister-harness' in installed_packages
71 def add_parse_arguments(parser = None) -> argparse.ArgumentParser:
79 case_select = parser.add_argument_group("Test case selection",
82 $ ./scripts/twister -v \\
83 --testsuite-root tests/ztest/base \\
[all …]
Drunner.py3 # Copyright (c) 2018-2025 Intel Corporation
5 # SPDX-License-Identifier: Apache-2.0
63 logger = logging.getLogger('twister')
70 Most of the stats are at test instance level
71 Except that case statistics are for cases of ALL test instances
73 total = yaml test scenarios * applicable platforms
76 completed = done - filtered_static
79 pass rate = passed / (total - filtered_configs)
80 case pass rate = passed_cases / (cases - filtered_cases - skipped_cases)
114 # initialized to number of test instances
[all …]
Dtestplan.py4 # Copyright (c) 2018-2024 Intel Corporation
7 # SPDX-License-Identifier: Apache-2.0
28 print("Install the anytree module to use the --test-tree option")
40 logger = logging.getLogger('twister')
49 "python-devicetree", "src"))
50 from devicetree import edtlib # pylint: disable=unused-import
65 # in case a test suite was quarantined.
67 # in case a test suite is skipped intentionally .
85 config_re = re.compile('(CONFIG_[A-Za-z0-9_]+)[=]\"?([^\"]*)\"?$')
86 dt_re = re.compile('([A-Za-z0-9_]+)[=]\"?([^\"]*)\"?$')
[all …]
Dtestinstance.py3 # Copyright (c) 2018-2025 Intel Corporation
7 # SPDX-License-Identifier: Apache-2.0
38 logger = logging.getLogger('twister')
44 @param test The TestSuite object we want to build/execute
46 @param base_outdir Base directory for all test results. The actual
47 out directory used is <outdir>/<platform>/<test case name>
80 source_dir_rel = testsuite.source_dir_rel.rsplit(os.pardir+os.path.sep, 1)[-1]
122 def status(self) -> TwisterStatus:
126 def status(self, value : TwisterStatus) -> None:
185 def compose_case_name(self, tc_name) -> str:
[all …]
/Zephyr-latest/scripts/footprint/
Dcompare_footprint3 # SPDX-License-Identifier: Apache-2.0
5 This script help you to compare footprint results with previous commits in git.
7 against the last release results.
8 To run it you need to set up the same environment as twister.
17 BASE_COMMIT is the commit used as base to compare results.
44 proc = subprocess.Popen('git rev-parse --is-inside-work-tree',
62 format = logging.Formatter('%(levelname)-8s: %(message)s')
73 "To run it you need to set up the same environment as twister.",
75 parser.add_argument('-b', '--base-commit', default=None,
79 parser.add_argument('-c', '--commit', default=None,
[all …]
/Zephyr-latest/scripts/tests/twister/
Dtest_environment.py5 # SPDX-License-Identifier: Apache-2.0
25 ['--short-build-path', '-k'],
26 '--short-build-path requires Ninja to be enabled'
32 ['--device-serial-pty', 'dummy'],
33 '--device-serial-pty is not supported on Windows OS'
39 ['--west-runner=dummy'],
40 'west-runner requires west-flash to be enabled'
46 ['--west-flash=\"--board-id=dummy\"'],
47 'west-flash requires device-testing to be enabled'
56 ['--enable-valgrind'],
[all …]
Dtest_runner.py4 # SPDX-License-Identifier: Apache-2.0
25 sys.path.insert(0, os.path.join(ZEPHYR_BASE, "scripts/pylib/twister"))
71 def project_builder(mocked_instance, mocked_env, mocked_jobserver) -> ProjectBuilder:
77 def runners(project_builder: ProjectBuilder) -> dict:
111 ["basearg1", "CONFIG_t=\"test\"", "SNIPPET_t=\"test\""],
119 "-DCONFIG_t=\"test\"",
120 "-Dcmake1=foo", "-Dcmake2=bar",
121 "-Dbasearg1", "-DSNIPPET_t=test",
122 "-Dhandler_arg1", "-Dhandler_arg2",
123 "-DCONF_FILE=a.conf;b.conf;c.conf",
[all …]
Dtest_testsuite.py4 # SPDX-License-Identifier: Apache-2.0
18 sys.path.insert(0, os.path.join(ZEPHYR_BASE, 'scripts', 'pylib', 'twister'))
38 'Found a test that does not start with test_',
46 'Found a test that does not start with test_',
62 'Found a test that does not start with test_',
74 # 'Found a test that does not start with test_',
124 warnings='Found a test that does not start with test_',
199 'test not starting with test_',
209 ztest files for warnings and results
300 Test to check if tests reference the category and subsystem
[all …]
/Zephyr-latest/doc/contribute/
Dguidelines.rst6 As an open-source project, we welcome and encourage the community to submit
38 your Pull Request (PR). You can see any failure results in the workflow
52 https://github.com/zephyrproject-rtos/zephyr/blob/main/LICENSE
54 .. _GitHub repo: https://github.com/zephyrproject-rtos/zephyr
66 https://www.zephyrproject.org/faqs/#1571346989065-9216c551-f523
69 https://www.whitesourcesoftware.com/whitesource-blog/top-10-apache-license-questions-answered/
92 See :ref:`external-contributions` for more information about
112 .. code-block:: C
117 https://www.linuxfoundation.org/blog/copyright-notices-in-open-source-software-projects/
130 later in this document), the developer simply adds a ``Signed-off-by``
[all …]
/Zephyr-latest/doc/develop/test/figures/
Dtwister_and_pytest.svg1 <?xml version="1.0" encoding="UTF-8"?>
2 <!-- Do not edit this file with editors other than draw.io -->
3 <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
4 …-0.5 -0.5 761 769" content="&lt;mxfile host=&quot;ac.draw.io&quot; modified=&quot;2023-05-24T14:31…
/Zephyr-latest/boards/native/doc/
Dbsim_boards_design.rst50 The main purpose of these bsim boards is to be test-benches for
52 Integration testing in the sense that the code under test will, at the very
80 - Unit tests:
85 - Integration tests on real HW: Allows testing with the real SW
90 test in general not being reproducible, and in many cases failures
93 - Integration tests on workstation (what the POSIX arch and these boards enable)
95 - Using bsim boards: Allow testing the embedded SW (or a subset), including
97 it is possible to test the components interactions and their integration.
98 - Using bsim boards with the BabbleSim Physical layer simulation allows
102 - Using bsim boards, and the `EDTT`_ framework: With the EDTT framework we can
[all …]
/Zephyr-latest/doc/releases/
Drelease-notes-3.0.rst22 * CVE-2021-3835: `Zephyr project bug tracker GHSA-fm6v-8625-99jf
23 <https://github.com/zephyrproject-rtos/zephyr/security/advisories/GHSA-fm6v-8625-99jf>`_
25 * CVE-2021-3861: `Zephyr project bug tracker GHSA-hvfp-w4h8-gxvj
26 <https://github.com/zephyrproject-rtos/zephyr/security/advisories/GHSA-hvfp-w4h8-gxvj>`_
28 * CVE-2021-3966: `Zephyr project bug tracker GHSA-hfxq-3w6x-fv2m
29 <https://github.com/zephyrproject-rtos/zephyr/security/advisories/GHSA-hfxq-3w6x-fv2m>`_
36 <https://github.com/zephyrproject-rtos/zephyr/issues?q=is%3Aissue+is%3Aopen+label%3Abug>`_.
53 * Added ``ranges`` and ``dma-ranges`` as invalid property to be used with DT_PROP_LEN()
58 CRC-16-ANSI checksum. A new function, :c:func:`crc16_reflect`, has been
69 * The following Kconfig options related to radio front-end modules (FEMs) were
[all …]

12