1.. _coverage:
2
3Generating coverage reports
4###########################
5
6With Zephyr, you can generate code coverage reports to analyze which parts of
7the code are covered by a given test or application.
8
9You can do this in two ways:
10
11* In a real embedded target or QEMU, using Zephyr's gcov integration
12* Directly in your host computer, by compiling your application targeting
13  the POSIX architecture
14
15Test coverage reports in embedded devices or QEMU
16*************************************************
17
18Overview
19========
20`GCC GCOV <gcov_>`_ is a test coverage program
21used together with the GCC compiler to analyze and create test coverage reports
22for your programs, helping you create more efficient, faster running code and
23discovering untested code paths
24
25In Zephyr, gcov collects coverage profiling data in RAM (and not to a file
26system) while your application is running. Support for gcov collection and
27reporting is limited by available RAM size and so is currently enabled only
28for QEMU emulation of embedded targets.
29
30Details
31=======
32There are 2 parts to enable this feature. The first is to enable the coverage for the
33device and the second to enable in the test application. As explained earlier the
34code coverage with gcov is a function of RAM available. Therefore ensure that the
35device has enough RAM when enabling the coverage for it. For example a small device
36like frdm_k64f can run a simple test application but the more complex test
37cases which consume more RAM will crash when coverage is enabled.
38
39To enable the device for coverage, select :kconfig:option:`CONFIG_HAS_COVERAGE_SUPPORT`
40in the Kconfig.board file.
41
42To report the coverage for the particular test application set :kconfig:option:`CONFIG_COVERAGE`.
43
44Steps to generate code coverage reports
45=======================================
46
47These steps will produce an HTML coverage report for a single application.
48
491. Build the code with CONFIG_COVERAGE=y.
50
51   .. zephyr-app-commands::
52      :board: mps2/an385
53      :gen-args: -DCONFIG_COVERAGE=y -DCONFIG_COVERAGE_DUMP=y
54      :goals: build
55      :compact:
56
57#. Capture the emulator output into a log file. You may need to terminate
58   the emulator with :kbd:`Ctrl-A X` for this to complete after the coverage dump
59   has been printed:
60
61   .. code-block:: console
62
63      ninja -Cbuild run | tee log.log
64
65   or
66
67   .. code-block:: console
68
69      ninja -Cbuild run | tee log.log
70
71#. Generate the gcov ``.gcda`` and ``.gcno`` files from the log file that was
72   saved::
73
74     $ python3 scripts/gen_gcov_files.py -i log.log
75
76#. Find the gcov binary placed in the SDK. You will need to pass the path to
77   the gcov binary for the appropriate architecture when you later invoke
78   ``gcovr``::
79
80     $ find $ZEPHYR_SDK_INSTALL_DIR -iregex ".*gcov"
81
82#. Create an output directory for the reports::
83
84     $ mkdir -p gcov_report
85
86#. Run ``gcovr`` to get the reports::
87
88     $ gcovr -r $ZEPHYR_BASE . --html -o gcov_report/coverage.html --html-details --gcov-executable <gcov_path_in_SDK>
89
90.. _coverage_posix:
91
92Coverage reports using the POSIX architecture
93*********************************************
94
95When compiling for the POSIX architecture, you utilize your host native tooling
96to build a native executable which contains your application, the Zephyr OS,
97and some basic HW emulation.
98
99That means you can use the same tools you would while developing any
100other desktop application.
101
102To build your application with ``gcc``'s `gcov`_, simply set
103:kconfig:option:`CONFIG_COVERAGE` before compiling it.
104When you run your application, ``gcov`` coverage data will be dumped into the
105respective ``gcda`` and ``gcno`` files.
106You may postprocess these with your preferred tools. For example:
107
108.. zephyr-app-commands::
109   :zephyr-app: samples/hello_world
110   :gen-args: -DCONFIG_COVERAGE=y
111   :host-os: unix
112   :board: native_sim
113   :goals: build
114   :compact:
115
116.. code-block:: console
117
118   $ ./build/zephyr/zephyr.exe
119   # Press Ctrl+C to exit
120   lcov --capture --directory ./ --output-file lcov.info -q --rc lcov_branch_coverage=1
121   genhtml lcov.info --output-directory lcov_html -q --ignore-errors source --branch-coverage --highlight --legend
122
123.. note::
124
125   You need a recent version of lcov (at least 1.14) with support for
126   intermediate text format. Such packages exist in recent Linux distributions.
127
128   Alternatively, you can use gcovr (at least version 4.2).
129
130Coverage reports using Twister
131******************************
132
133Zephyr's :ref:`twister script <twister_script>` can automatically
134generate a coverage report from the tests which were executed.
135You just need to invoke it with the ``--coverage`` command line option.
136
137For example, you may invoke::
138
139    $ twister --coverage -p qemu_x86 -T tests/kernel
140
141or::
142
143    $ twister --coverage -p native_sim -T tests/bluetooth
144
145which will produce ``twister-out/coverage/index.html`` report as well as
146the coverage data collected by ``gcovr`` tool in ``twister-out/coverage.json``.
147
148Other reports might be chosen with ``--coverage-tool`` and ``--coverage-formats``
149command line options.
150
151The process differs for unit tests, which are built with the host
152toolchain and require a different board::
153
154    $ twister --coverage -p unit_testing -T tests/unit
155
156which produces a report in the same location as non-unit testing.
157
158.. _gcov:
159   https://gcc.gnu.org/onlinedocs/gcc/Gcov.html
160
161Using different toolchains
162==========================
163
164Twister looks at the environment variable ``ZEPHYR_TOOLCHAIN_VARIANT``
165to check which gcov tool to use by default. The following are used as the
166default for the Twister ``--gcov-tool`` argument default:
167
168+-----------+-----------------------+
169| Toolchain | ``--gcov-tool`` value |
170+-----------+-----------------------+
171| host      | ``gcov``              |
172+-----------+-----------------------+
173| llvm      | ``llvm-cov gcov``     |
174+-----------+-----------------------+
175| zephyr    | ``gcov``              |
176+-----------+-----------------------+
177