Lines Matching +full:- +full:- +full:timeout +full:- +full:multiplier
20 When using (at least) one ``-v`` option, twister's console output
27 - The test is marked as ``build_only: true`` in its ``.yaml``
29 - The test configuration has defined a ``harness`` but you don't have
31 - The target device is not connected and not available for flashing
32 - You or some higher level automation invoked twister with
33 ``--build-only``.
39 .. group-tab:: Linux
41 .. code-block:: bash
43 $ source zephyr-env.sh
46 .. group-tab:: Windows
48 .. code-block:: bat
50 zephyr-env.cmd
59 .. group-tab:: Linux
61 .. code-block:: bash
63 $ ./scripts/twister --all --enable-slow
65 .. group-tab:: Windows
67 .. code-block:: bat
69 python .\scripts\twister --all --enable-slow
75 the ``--platform`` option, it is a platform filter for testing, with this
78 you can use ``--platform board@revision`` to test on a specific revision.
84 .. group-tab:: Linux
86 .. code-block:: bash
88 $ ./scripts/twister --help
90 .. group-tab:: Windows
92 .. code-block:: bat
94 python .\scripts\twister --help
110 .. code-block:: yaml
113 name: NXP FRDM-K64F
117 - zephyr
118 - gnuarmemb
120 - arduino_gpio
121 - arduino_i2c
122 - netif:eth
123 - adc
124 - i2c
125 - nvs
126 - spi
127 - gpio
128 - usb_device
129 - watchdog
130 - can
131 - pwm
141 .. code-block:: console
144 west build -b reel_board
146 cmake -DBOARD=reel_board ..
155 .. code-block:: yaml
158 - name: qemu
159 - name: armfvp
161 - name: custom
165 simulation can be selected with ``--simulation <simulation_name>``.
185 .. code-block:: yaml
188 - pci
193 .. code-block:: yaml
196 - netif:eth
197 - sensor:bmi16
223 Multiply each test scenario timeout by specified ratio. This option allows to tune timeouts only
225 power-efficient but slow CPU or simulation platform which can perform instruction accurate
248 :figclass: align-center
297 The ``--no-detailed-test-id`` command line option modifies the above rules in this way:
313 .. code-block:: yaml
332 .. code-block:: yaml
367 Don't run this test scenario unless ``--enable-slow`` or ``--enable-slow-only`` was
368 passed in on the command line. Intended for time-consuming test scenarios that
379 .. code-block:: yaml
388 - arch:x86:CONFIG_ADC_ASYNC=y
389 - platform:qemu_x86:CONFIG_DEBUG=y
390 - platform:mimxrt1060_evk:SHIELD=rk043fn66hs_ctg
391 - simulation:qemu:CONFIG_MPU=y
397 .. code-block:: yaml
406 - CONFIG_ADC_ASYNC=y
411 .. code-block:: yaml
420 - arch:x86:CONFIG_ADC_ASYNC=y
421 - platform:qemu_x86:CONFIG_DEBUG=y
447 test will be selectable using the command line option ``--level <level name>``
459 timeout: <number of seconds>
488 invoked with the ``--integration`` option. Use this instead of
497 test scenario -> platforms1 -> toolchain1
498 test scenario -> platforms2 -> toolchain1
507 .. code-block:: yaml
510 - toolchain1
511 - toolchain2
515 test scenario -> platforms1 -> toolchain1
516 test scenario -> platforms1 -> toolchain2
517 test scenario -> platforms2 -> toolchain1
518 test scenario -> platforms2 -> toolchain2
524 ``--integration`` option.
538 the images onto the target. The ``--erase`` option of west flash is
556 - ztest
557 - test
558 - console
559 - pytest
560 - gtest
561 - robot
562 - ctest
563 - shell
577 .. code-block:: yaml
580 - arch
581 - simulation
609 .. code-block:: yaml
619 .. code-block:: yaml
626 - [robot file path 1]
627 - [robot file path 2]
628 - [robot file path n]
632 .. code-block:: yaml
640 - --exclude tag
641 - --stop-on-error
647 .. code-block:: none
663 .. code-block:: antlr
687 if the symbol is defined to a non-empty string.
694 * all comparison operators (non-associative)
699 .. code-block:: none
705 .. code-block:: none
713 .. code-block:: none
729 .. code-block:: yaml
734 - cdc-acm-console
735 - user-snippet-example
739 running with ``-v`` or examining the discard report
745 To update this, pass the ``--all --release`` options.
764 - keyboard
765 - net
766 - bluetooth
770 .. code-block:: yaml
781 - "Temperature:(.*)C"
782 - "Relative Humidity:(.*)%"
794 ``ctest_args: [‘--repeat until-pass:5’]``. Note that
795 ``--ctest-args`` can be passed multiple times to pass several arguments
820 .. code-block:: yaml
824 - "pytest/test_shell_help.py"
825 - "../shell/pytest/test_shell.py"
826 - "/tmp/test_shell.py"
827 - "~/tmp/test_shell.py"
828 - "$ZEPHYR_BASE/samples/subsys/testsuite/pytest/shell/pytest/test_shell.py"
829 - "pytest/test_shell_help.py::test_shell2_sample" # select pytest subtest
830 … - "pytest/test_shell_help.py::test_shell2_sample[param_a]" # select pytest parametrized subtest
836 ``pytest_args: [‘-k=test_method’, ‘--log-level=DEBUG’]``. Note that
837 ``--pytest-args`` can be passed multiple times to pass several arguments
852 .. code-block:: yaml
860 - pytest_dir1
861 - $ENV_VAR/samples/test/pytest_dir2
865 - pytest/test_file_1.py
866 - test_file_2.py::test_A
867 - test_file_2.py::test_B[param_a]
906 .. code-block:: yaml
910 - "(?P<metric>.*):(?P<cycles>.*) cycles, (?P<nanoseconds>.*) ns"
924 as-is.
932 .. code-block:: yaml
940 .. code-block:: none
946 .. code-block:: json
973 Harness ``bsim`` is implemented in limited way - it helps only to copy the
1000 .. code-block:: yaml
1004 - command: "kernel cycles"
1006 - command: "kernel version"
1008 - command: "kernel sleep 100"
1018 .. code-block:: yaml
1020 - command: "mpu mtest 1"
1022 - command: "mpu mtest 2"
1043 Using ``--force-platform`` allows to override filtering caused by ``platform_allow``,
1049 * ``-p/--platform <platform_name>`` (can be used multiple times): only platforms
1051 * ``-l/--all``: all available platforms;
1052 * ``-G/--integration``: all platforms from an ``integration_platforms`` list in
1066 * ``timeout`` option in each test scenario. See :ref:`here <twister_test_case_timeout>` for more
1070 * ``--timeout-multiplier`` twister option which can be used to adjust timeouts in exact twister run.
1075 Overall test scenario timeout is a multiplication of these three parameters.
1082 be activated using the ``--integration`` option of twister and narrows down
1092 supports running any out-of-tree custom emulator defined in the board's :file:`board.cmake`.
1096 .. code-block:: yaml
1099 - name: custom
1107 .. code-block:: cmake
1114 .. code-block:: cmake
1141 .. group-tab:: Linux
1143 .. code-block:: bash
1145 scripts/twister --device-testing --device-serial /dev/ttyACM0 \
1146 --device-serial-baud 115200 -p frdm_k64f -T tests/kernel
1148 .. group-tab:: Windows
1150 .. code-block:: bat
1152 python .\scripts\twister --device-testing --device-serial COM1 \
1153 --device-serial-baud 115200 -p frdm_k64f -T tests/kernel
1155 The ``--device-serial`` option denotes the serial device the board is connected to.
1157 only one board at a time, specified using the ``--platform`` option.
1159 The ``--device-serial-baud`` option is only needed if your device does not run at
1162 To support devices without a physical serial port, use the ``--device-serial-pty``
1168 .. group-tab:: Linux
1170 .. code-block:: bash
1172 scripts/twister --device-testing --device-serial-pty "script.py" \
1173 -p intel_adsp/cavs25 -T tests/kernel
1175 .. group-tab:: Windows
1181 The script is user-defined and handles delivering the messages which can be
1184 The ``--device-flash-timeout`` option allows to set explicit timeout on the
1188 The ``--device-flash-with-test`` option indicates that on the platform
1189 the flash operation also executes a test scenario, so the flash timeout is
1190 increased by a test scenario timeout.
1202 .. group-tab:: Linux
1204 .. code-block:: bash
1206 ./scripts/twister --generate-hardware-map map.yml
1208 .. group-tab:: Windows
1210 .. code-block:: bat
1212 python .\scripts\twister --generate-hardware-map map.yml
1219 .. group-tab:: Linux
1221 .. code-block:: yaml
1223 - connected: true
1226 product: DAPLink CMSIS-DAP
1229 - connected: true
1232 product: J-Link
1236 .. group-tab:: Windows
1238 .. code-block:: yaml
1240 - connected: true
1246 - connected: true
1261 .. group-tab:: Linux
1263 .. code-block:: yaml
1265 - connected: true
1268 product: DAPLink CMSIS-DAP
1272 - connected: true
1275 product: J-Link
1280 .. group-tab:: Windows
1282 .. code-block:: yaml
1284 - connected: true
1287 product: DAPLink CMSIS-DAP
1291 - connected: true
1294 product: J-Link
1309 .. group-tab:: Linux
1311 .. code-block:: bash
1313 ./scripts/twister --device-testing --hardware-map map.yml -T samples/hello_world/
1315 .. group-tab:: Windows
1317 .. code-block:: bat
1319 python .\scripts\twister --device-testing --hardware-map map.yml -T samples\hello_world
1331 Hardware map allows to set ``--device-flash-timeout`` and ``--device-flash-with-test``
1332 command line options as ``flash-timeout`` and ``flash-with-test`` fields respectively.
1335 Serial PTY support using ``--device-serial-pty`` can also be used in the
1338 .. code-block:: yaml
1340 - connected: true
1347 - --remote-host=remote_host_ip_addr
1348 - --key=/path/to/key.pem
1357 .. group-tab:: Linux
1359 .. code-block:: bash
1361 west flash --remote-host remote_host_ip_addr --key /path/to/key.pem
1363 twister -p intel_adsp/cavs25 --device-testing --device-serial-pty script.py
1364 --west-flash="--remote-host=remote_host_ip_addr,--key=/path/to/key.pem"
1366 .. group-tab:: Windows
1374 For serial PTY, the "--generate-hardware-map" option cannot scan it out
1390 .. code-block:: yaml
1392 - connected: true
1394 - gpio_loopback
1397 product: DAPLink CMSIS-DAP
1401 When running ``twister`` with ``--device-testing``, the configured fixture
1406 :figclass: align-center
1408 Fixtures can also be provided via twister command option ``--fixture``, this option
1424 .. code-block:: yaml
1426 - connected: false
1428 - gpio_loopback
1436 product: J-Link
1443 When (re-)generated the hardware map file will contain an ``id`` keyword
1444 that serves as the argument to ``--board-id`` when flashing. In some
1446 using an external J-Link probe. The ``probe_id`` keyword overrides the
1449 .. code-block:: yaml
1451 - connected: false
1455 product: DAPLink CMSIS-DAP
1465 re-configuring the hardware map file for each variant. For example:
1467 .. code-block:: yaml
1469 - connected: true
1472 - nrf5340dk/nrf5340/cpuapp
1473 - nrf5340dk/nrf5340/cpuapp/ns
1474 product: J-Link
1488 ``--quarantine-list <PATH_TO_QUARANTINE_YAML>`` to a twister call.
1491 ``--quarantine-verify`` to the above argument. This will make twister skip all tests
1507 .. code-block:: yaml
1509 - scenarios:
1510 - sample.basic.helloworld
1511 comment: "Link to the issue: https://github.com/zephyrproject-rtos/zephyr/pull/33287"
1513 - scenarios:
1514 - kernel.common
1515 - kernel.common.(misra|tls)
1516 - kernel.common.nano64
1518 - .*_cortex_.*
1519 - native_sim
1523 .. code-block:: yaml
1525 - platforms:
1526 - qemu_x86
1556 - Ability to ignore default platforms as defined in board definitions
1559 - Option to specify your own list of default platforms overriding what
1561 - Ability to override ``build_on_all`` options used in some test scenarios.
1564 - Ignore some logic in twister to expand platform coverage in cases where
1573 - ``override_default_platforms``: override default key a platform sets in board
1577 - ``increased_platform_scope``: This option is set to True by default, when
1580 - ``default_platforms``: A list of additional default platforms to add. This list
1586 .. code-block:: yaml
1592 - qemu_x86
1608 .. code-block:: yaml
1611 - name: my-test-level
1615 - kernel.threads.*
1616 - kernel.timer.behavior
1617 - arch.interrupt
1618 - boards.*
1628 .. code-block:: yaml
1633 - frdm_k64f
1635 - name: smoke
1638 - name: unit
1641 - name: integration
1644 - name: acceptance
1647 - name: system
1650 - name: regression
1660 .. group-tab:: Linux
1662 .. code-block:: bash
1664 scripts/twister --test-config=<path to>/test_config.yaml
1665 -T tests --level="smoke"
1674 the seed to the random number generator by providing ``-seed=value`` as an
1682 Robot files allow you to express interactive test scenarios in human-readable text format and execu…
1689 .. group-tab:: Linux
1691 .. code-block:: bash
1693 …$ ./scripts/twister --platform hifive1 --test samples/subsys/shell/shell_module/sample.shell.shell…
1695 .. group-tab:: Windows
1697 .. code-block:: bat
1699 …python .\scripts\twister --platform hifive1 --test samples/subsys/shell/shell_module/sample.shell.…
1709 …t test suite files, as an external Python library or, like Renode does it, dynamically via XML-RPC.
1710 …ramework.org/robotframework/latest/RobotFrameworkUserGuide.html#extending-robot-framework>`_ secti…
1717 .. code-block:: bash
1719 $ twister -p qemu_riscv32 -s tests/kernel/interrupt/arch.shared_interrupt