README.rst
1Bluetooth mesh BabbleSim tests
2##############################
3
4This directory contains a set of high level system tests for the Bluetooth mesh
5subsystem. The tests run on the BabbleSim simulator, using the BabbleSim test
6framework.
7
8Tests are organized into bash scripts under the test_scripts folder. Each
9subfolder of test_scripts contains tests for a specific module in the Bluetooth
10mesh subsystem, and each folder has a corresponding test_<subfolder>.c under the
11src/ directory containing the necessary test harnesses to execute the tests.
12
13There's only a single test application for all the Bluetooth mesh BabbleSim
14tests. The test application is built from this directory, and includes all test
15harnesses in every build. The overlying bsim test framework selects the harness
16to use at runtime, using the test identifiers passed in the test scripts.
17
18Running the tests
19*****************
20
21The Bluetooth mesh tests have no extra requirements, and can be run using the
22procedure described in the parent folder.
23
24To only run the mesh tests, set ``SEARCH_PATH`` to point to this folder before
25calling ``run_parallel.sh``
26
27Debugging the tests
28===================
29
30Individual test harnesses can be run outside of ``RunTest()`` by setting the
31``$SKIP`` array environment variable before calling ``RunTest``. This causes
32``RunTest`` to skip starting processes for each application in the ``$SKIPPED``
33array, so that they can be started independently in gdb, valgrind or some other
34external tool.
35
36When running test applications independently, make sure to assign the original
37device number with -dXXX to the process.
38
39Example: To debug the ``transport_tx_seg_block`` test harness in the transport
40seg_block test, change transport/seg_block.sh to
41
42..code-block::
43
44 SKIP=(transport_tx_seg_block)
45 RunTest mesh_transport_seg_block \
46 transport_tx_seg_block \
47 transport_rx_seg_block
48
49Then separately, call
50
51...code-block::
52
53 gdb bs_nrf52_bsim_tests_bluetooth_bsim_bt_bsim_test_mesh_prj_conf \
54 -s=mesh_transport_seg_block -d=0 -RealEncryption=1 \
55 -testid=transport_tx_seg_block
56
57Framework
58*********
59
60The Bluetooth mesh BabbleSim tests mainly operate on the test framework for the
61BabbleSim platform, but with some mesh specific additions:
62
63mesh_test.sh
64=============
65
66All test scripts in the Bluetooth mesh BabbleSim test suite follow a common
67pattern for running a single test across N devices with different test
68harnesses. ``mesh_test.sh`` is sourced in each test script, and its ``RunTest``
69function is called once in each script with the following parameters:
70
71..code-block::
72
73 RunTest <test name> <harness_name_1> <harness_name_2>...
74
75``RunTest`` starts one mesh application for each selected harness, before
76starting the BabbleSim phy process.
77
78mesh_test.c
79===========
80
81Common target side mesh behavior is collected in mesh_test.c and mesh_test.h.
82This includes ``PASS``/``FAIL`` macros and some common mesh actions, such as
83mesh initialization, as well as message sending and receiving.
84
85The mesh_test module is never called from the framework, so each test harness
86is free to call the parts that are appropriate for its implementation.
87All utility functions can be replaced by custom behavior in each harness if the
88generic implementation does not fit with the harness requirements.
89
90Adding tests
91************
92
93Each test needs a separate test script and one or more test harnesses that
94execute the test on target.
95
96Test scripts should be organized by submodules under the test_scripts
97directory, and each test case needs a separate test script with a single test
98run.
99
100Each test harness is defined by a ``struct bst_test_instance` structure, that
101is presented to the test runner through an ``install`` function called from the
102test application's main function. The harness contains a set of callback
103functions that allows starting the test behavior.
104
105Each mesh test harness should call the ``bt_mesh_test_cfg`` function from its
106``test_init_f`` callback to set up the test timeout and mesh metadata. The test
107will run until the mesh_test module's ``FAIL`` macro is called, or until the
108test times out. At the test timeout, the test will pass if the ``PASS()`` macro
109has been called - otherwise, it will fail.
110
111..note::
112
113 The Bluetooth stack must be initialized in the ``test_main_f`` function of
114 the harness, as the previous callbacks are all executed in hardware threads.
115
116The Bluetooth mesh tests include the entire Bluetooth host and controller
117subsystems, so timing requirements should generally be kept fairly liberal to
118avoid regressions on changes in the underlying layers.
119
