1.. _coverage:
2
3Generating coverage reports
4###########################
5
6With Zephyr, you can generate code coverage reports to analyze which parts of
7the code are covered by a given test or application.
8
9You can do this in two ways:
10
11* In a real embedded target or QEMU, using Zephyr's gcov integration
12* Directly in your host computer, by compiling your application targeting
13  the POSIX architecture
14
15Test coverage reports in embedded devices or QEMU
16*************************************************
17
18Overview
19========
20`GCC GCOV <gcov_>`_ is a test coverage program
21used together with the GCC compiler to analyze and create test coverage reports
22for your programs, helping you create more efficient, faster running code and
23discovering untested code paths
24
25In Zephyr, gcov collects coverage profiling data in RAM (and not to a file
26system) while your application is running. Support for gcov collection and
27reporting is limited by available RAM size and so is currently enabled only
28for QEMU emulation of embedded targets.
29
30Details
31=======
32There are 2 parts to enable this feature. The first is to enable the coverage for the
33device and the second to enable in the test application. As explained earlier the
34code coverage with gcov is a function of RAM available. Therefore ensure that the
35device has enough RAM when enabling the coverage for it. For example a small device
36like frdm_k64f can run a simple test application but the more complex test
37cases which consume more RAM will crash when coverage is enabled.
38
39To enable the device for coverage, select :kconfig:option:`CONFIG_HAS_COVERAGE_SUPPORT`
40in the Kconfig.board file.
41
42To report the coverage for the particular test application set :kconfig:option:`CONFIG_COVERAGE`.
43
44Steps to generate code coverage reports
45=======================================
46
47These steps will produce an HTML coverage report for a single application.
48
491. Build the code with CONFIG_COVERAGE=y.
50
51   .. zephyr-app-commands::
52      :board: mps2/an385
53      :gen-args: -DCONFIG_COVERAGE=y -DCONFIG_COVERAGE_DUMP=y
54      :goals: build
55      :compact:
56
57#. Capture the emulator output into a log file. You may need to terminate
58   the emulator with :kbd:`Ctrl-A X` for this to complete after the coverage dump
59   has been printed:
60
61   .. code-block:: console
62
63     $ ninja -Cbuild run | tee log.log
64
65   or
66
67   .. code-block:: console
68
69     $ ninja -Cbuild run | tee log.log
70
71#. Generate the gcov ``.gcda`` and ``.gcno`` files from the log file that was
72   saved:
73
74   .. code-block:: console
75
76     $ python3 scripts/gen_gcov_files.py -i log.log
77
78#. Find the gcov binary placed in the SDK. You will need to pass the path to
79   the gcov binary for the appropriate architecture when you later invoke
80   ``gcovr``:
81
82   .. code-block:: console
83
84     $ find $ZEPHYR_SDK_INSTALL_DIR -iregex ".*gcov"
85
86#. Create an output directory for the reports:
87
88   .. code-block:: console
89
90     $ mkdir -p gcov_report
91
92#. Run ``gcovr`` to get the reports:
93
94   .. code-block:: console
95
96     $ gcovr -r $ZEPHYR_BASE . --html -o gcov_report/coverage.html --html-details --gcov-executable <gcov_path_in_SDK>
97
98   .. _coverage_posix:
99
100Coverage reports using the POSIX architecture
101*********************************************
102
103When compiling for the POSIX architecture, you utilize your host native tooling
104to build a native executable which contains your application, the Zephyr OS,
105and some basic HW emulation.
106
107That means you can use the same tools you would while developing any
108other desktop application.
109
110To build your application with ``gcc``'s `gcov`_, simply set
111:kconfig:option:`CONFIG_COVERAGE` before compiling it.
112When you run your application, ``gcov`` coverage data will be dumped into the
113respective ``gcda`` and ``gcno`` files.
114You may postprocess these with your preferred tools. For example:
115
116.. zephyr-app-commands::
117   :zephyr-app: samples/hello_world
118   :gen-args: -DCONFIG_COVERAGE=y
119   :host-os: unix
120   :board: native_sim
121   :goals: build
122   :compact:
123
124.. code-block:: console
125
126   $ ./build/zephyr/zephyr.exe
127   # Press Ctrl+C to exit
128   $ lcov --capture --directory ./ --output-file lcov.info -q --rc lcov_branch_coverage=1
129   $ genhtml lcov.info --output-directory lcov_html -q --ignore-errors source --branch-coverage --highlight --legend
130
131.. note::
132
133   You need a recent version of lcov (at least 1.14) with support for
134   intermediate text format. Such packages exist in recent Linux distributions.
135
136   Alternatively, you can use gcovr (at least version 4.2).
137
138Coverage reports using Twister
139******************************
140
141Zephyr's :ref:`twister script <twister_script>` can automatically
142generate a coverage report from the tests which were executed.
143You just need to invoke it with the ``--coverage`` command line option.
144
145For example, you may invoke:
146
147.. code-block:: console
148
149    $ twister --coverage -p qemu_x86 -T tests/kernel
150
151or:
152
153.. code-block:: console
154
155    $ twister --coverage -p native_sim -T tests/bluetooth
156
157which will produce ``twister-out/coverage/index.html`` report as well as
158the coverage data collected by ``gcovr`` tool in ``twister-out/coverage.json``.
159
160Other reports might be chosen with ``--coverage-tool`` and ``--coverage-formats``
161command line options.
162
163To generate code coverage report including Zephyr sources as well as your application
164code outside of Zephyr repository (see :ref:`Application Types <zephyr-app-types>`)
165call Twister from your project directory with ``--coverage-basedir $ZEPHYR_BASE``
166command line option, for example:
167
168.. code-block:: console
169
170   $ $ZEPHYR_BASE/scripts/twister --coverage -p native_sim --coverage_basedir $ZEPHYR_BASE -T your_project_dir
171
172.. note::
173
174   By default, Twister calls ``gcovr`` tool which filters source files assuming real paths
175   are everywhere with `all symlinks resolved <gcovr_symlinks_>`_, so when your development
176   environment has directories with symlinks then, to avoid incomplete ``gcovr`` reports,
177   either your :ref:`ZEPHYR_BASE <important-build-vars>` should contain a real path,
178   or ``lcov`` tool used instead of ``gcovr`` with additional Twister command line
179   option ``--coverage-tool lcov``.
180
181The process differs for unit tests, which are built with the host
182toolchain and require a different board:
183
184.. code-block:: console
185
186   $ twister --coverage -p unit_testing -T tests/unit
187
188which produces a report in the same location as non-unit testing.
189
190.. _gcovr_symlinks:
191   https://github.com/gcovr/gcovr/blob/main/doc/source/guide/filters.rst#filters-for-symlinks
192
193.. _gcov:
194   https://gcc.gnu.org/onlinedocs/gcc/Gcov.html
195
196Using different toolchains
197==========================
198
199Twister looks at the environment variable ``ZEPHYR_TOOLCHAIN_VARIANT``
200to check which gcov tool to use by default. The following are used as the
201default for the Twister ``--gcov-tool`` argument default:
202
203+-----------+-----------------------+
204| Toolchain | ``--gcov-tool`` value |
205+-----------+-----------------------+
206| host      | ``gcov``              |
207+-----------+-----------------------+
208| llvm      | ``llvm-cov gcov``     |
209+-----------+-----------------------+
210| zephyr    | ``gcov``              |
211+-----------+-----------------------+
212