1.. _cmake-details:
2
3Build System (CMake)
4********************
5
6
7CMake is used to build your application together with the Zephyr kernel. A
8CMake build is done in two stages. The first stage is called
9**configuration**. During configuration, the CMakeLists.txt build scripts are
10executed. After configuration is finished, CMake has an internal model of the
11Zephyr build, and can generate build scripts that are native to the host
12platform.
13
14CMake supports generating scripts for several build systems, but only Ninja and
15Make are tested and supported by Zephyr. After configuration, you begin the
16**build** stage by executing the generated build scripts. These build scripts
17can recompile the application without involving CMake following
18most code changes. However, after certain changes, the configuration step must
19be executed again before building. The build scripts can detect some of these
20situations and reconfigure automatically, but there are cases when this must be
21done manually.
22
23Zephyr uses CMake's concept of a 'target' to organize the build. A
24target can be an executable, a library, or a generated file. For
25application developers, the library target is the most important to
26understand. All source code that goes into a Zephyr build does so by
27being included in a library target, even application code.
28
29Library targets have source code, that is added through CMakeLists.txt
30build scripts like this:
31
32.. code-block:: cmake
33
34   target_sources(app PRIVATE src/main.c)
35
36In the above :file:`CMakeLists.txt`, an existing library target named ``app``
37is configured to include the source file :file:`src/main.c`. The ``PRIVATE``
38keyword indicates that we are modifying the internals of how the library is
39being built. Using the keyword ``PUBLIC`` would modify how other
40libraries that link with app are built. In this case, using ``PUBLIC``
41would cause libraries that link with ``app`` to also include the
42source file :file:`src/main.c`, behavior that we surely do not want. The
43``PUBLIC`` keyword could however be useful when modifying the include
44paths of a target library.
45
46
47Build and Configuration Phases
48==============================
49
50The Zephyr build process can be divided into two main phases: a configuration
51phase (driven by CMake) and a build phase (driven by Make or Ninja).
52
53.. _build_configuration_phase:
54
55Configuration Phase
56-------------------
57
58The configuration phase begins when the user invokes *CMake* to generate a
59build system, specifying a source application directory and a board target.
60
61.. figure:: build-config-phase.svg
62    :align: center
63    :alt: Zephyr's build configuration phase
64    :figclass: align-center
65    :width: 80%
66
67CMake begins by processing the :file:`CMakeLists.txt` file in the application
68directory, which refers to the :file:`CMakeLists.txt` file in the Zephyr
69top-level directory, which in turn refers to :file:`CMakeLists.txt` files
70throughout the build tree (directly and indirectly). Its primary output is a
71set of Makefiles or Ninja files to drive the build process, but the CMake
72scripts also do some processing of their own, which is explained here.
73
74Note that paths beginning with :file:`build/` below refer to the build
75directory you create when running CMake.
76
77Devicetree
78   :file:`*.dts` (*devicetree source*) and :file:`*.dtsi` (*devicetree source
79   include*) files are collected from the target's architecture, SoC, board,
80   and application directories.
81
82   :file:`*.dtsi` files are included by :file:`*.dts` files via the C
83   preprocessor (often abbreviated *cpp*, which should not be confused with
84   C++). The C preprocessor is also used to merge in any devicetree
85   :file:`*.overlay` files, and to expand macros in :file:`*.dts`,
86   :file:`*.dtsi`, and :file:`*.overlay` files. The preprocessor output is
87   placed in :file:`build/zephyr/zephyr.dts.pre`.
88
89   The preprocessed devicetree sources are parsed by
90   :zephyr_file:`gen_defines.py <scripts/dts/gen_defines.py>` to generate a
91   :file:`build/zephyr/include/generated/devicetree_generated.h` header with
92   preprocessor macros.
93
94   Source code should access preprocessor macros generated from devicetree by
95   including the :zephyr_file:`devicetree.h <include/zephyr/devicetree.h>` header,
96   which includes :file:`devicetree_generated.h`.
97
98   :file:`gen_defines.py` also writes the final devicetree to
99   :file:`build/zephyr/zephyr.dts` in the build directory. This file's contents
100   may be useful for debugging.
101
102   If the devicetree compiler ``dtc`` is installed, it is run on
103   :file:`build/zephyr/zephyr.dts` to catch any extra warnings and errors
104   generated by this tool. The output from ``dtc`` is unused otherwise, and
105   this step is skipped if ``dtc`` is not installed.
106
107   The above is just a brief overview. For more information on devicetree, see
108   :ref:`dt-guide`.
109
110Kconfig
111   :file:`Kconfig` files define available configuration options for the
112   target architecture, SoC, board, and application, as well as dependencies
113   between options.
114
115   Kconfig configurations are stored in *configuration files*. The initial
116   configuration is generated by merging configuration fragments from the board
117   and application (e.g. :file:`prj.conf`).
118
119   The output from Kconfig is an :file:`autoconf.h` header with preprocessor
120   assignments, and a :file:`.config` file that acts both as a saved
121   configuration and as configuration output (used by CMake). The definitions in
122   :file:`autoconf.h` are automatically exposed at compile time, so there is no
123   need to include this header.
124
125   Information from devicetree is available to Kconfig, through the functions
126   defined in :zephyr_file:`kconfigfunctions.py
127   <scripts/kconfig/kconfigfunctions.py>`.
128
129   See :ref:`the Kconfig section of the manual <kconfig>` for more information.
130
131Build Phase
132-----------
133
134The build phase begins when the user invokes ``make`` or ``ninja``. Its
135ultimate output is a complete Zephyr application in a format suitable for
136loading/flashing on the desired target board (:file:`zephyr.elf`,
137:file:`zephyr.hex`, etc.) The build phase can be broken down, conceptually,
138into four stages: the pre-build, first-pass binary, final binary, and
139post-processing.
140
141Pre-build
142+++++++++
143
144Pre-build occurs before any source files are compiled, because during
145this phase header files used by the source files are generated.
146
147Offset generation
148   Access to high-level data structures and members is sometimes
149   required when the definitions of those structures is not
150   immediately accessible (e.g., assembly language). The generation of
151   *offsets.h* (by *gen_offset_header.py*) facilitates this.
152
153System call boilerplate
154   The *gen_syscall.py* and *parse_syscalls.py*  scripts work
155   together to bind potential system call functions with their
156   implementations.
157
158.. figure:: build-build-phase-1.svg
159    :align: center
160    :alt: Zephyr's build stage I
161    :figclass: align-center
162    :width: 80%
163
164Intermediate binaries
165+++++++++++++++++++++
166
167Compilation proper begins with the first intermediate binary. Source files (C
168and assembly) are collected from various subsystems (which ones is
169decided during the configuration phase), and compiled into archives
170(with reference to header files in the tree, as well as those
171generated during the configuration phase and the pre-build stage(s)).
172
173.. figure:: build-build-phase-2.svg
174    :align: center
175    :alt: Zephyr's build stage II
176    :figclass: align-center
177    :width: 80%
178
179The exact number of intermediate binaries is decided during the configuration
180phase.
181
182If memory protection is enabled, then:
183
184Partition grouping
185   The *gen_app_partitions.py* script scans all the
186   generated archives and outputs linker scripts to ensure that
187   application partitions are properly grouped and aligned for the
188   target’s memory protection hardware.
189
190Then *cpp* is used to combine linker script fragments from the target’s
191architecture/SoC, the kernel tree, optionally the partition output if
192memory protection is enabled, and any other fragments selected during
193the configuration process, into a *linker.cmd* file. The compiled
194archives are then linked with *ld* as specified in the
195*linker.cmd*.
196
197Unfixed size binary
198   The unfixed size intermediate binary is produced when :ref:`usermode_api`
199   is enabled or :ref:`devicetree` is in use.
200   It produces a binary where sizes are not fixed and thus it may be used
201   by post-process steps that will impact the size of the final binary.
202
203.. figure:: build-build-phase-3.svg
204    :align: center
205    :alt: Zephyr's build stage III
206    :figclass: align-center
207    :width: 80%
208
209Fixed size binary
210   The fixed size intermediate binary is produced when :ref:`usermode_api`
211   is enabled or when generated IRQ tables are used,
212   :kconfig:option:`CONFIG_GEN_ISR_TABLES`
213   It produces a binary where sizes are fixed and thus the size must not change
214   between the intermediate binary and the final binary.
215
216.. figure:: build-build-phase-4.svg
217    :align: center
218    :alt: Zephyr's build stage IV
219    :figclass: align-center
220    :width: 80%
221
222Intermediate binaries post-processing
223+++++++++++++++++++++++++++++++++++++
224
225The binaries from the previous stage are incomplete, with empty and/or
226placeholder sections that must be filled in by, essentially, reflection.
227
228To complete the build procedure the following scripts are executed on the
229intermediate binaries to produce the missing pieces needed for the final
230binary.
231
232When :ref:`usermode_api` is enabled:
233
234Partition alignment
235   The *gen_app_partitions.py* script scans the unfixed size binary and
236   generates an app shared memory aligned linker script snippet where the
237   partitions are sorted in descending order.
238
239.. figure:: build-postprocess-1.svg
240    :align: center
241    :alt: Zephyr's intermediate binary post-process I
242    :figclass: align-center
243    :width: 80%
244
245When :ref:`devicetree` is used:
246
247Device dependencies
248   The *gen_device_deps.py* script scans the unfixed size binary to determine
249   relationships between devices that were recorded from devicetree data,
250   and replaces the encoded relationships with values that are optimized to
251   locate the devices actually present in the application.
252
253.. figure:: build-postprocess-2.svg
254    :align: center
255    :alt: Zephyr's intermediate binary post-process II
256    :figclass: align-center
257    :width: 80%
258
259When :kconfig:option:`CONFIG_GEN_ISR_TABLES` is enabled:
260   The *gen_isr_tables.py* script scans the fixed size binary and creates
261   an isr_tables.c source file with a hardware vector table and/or software
262   IRQ table.
263
264.. figure:: build-postprocess-3.svg
265    :align: center
266    :alt: Zephyr's intermediate binary post-process III
267    :figclass: align-center
268    :width: 80%
269
270When :ref:`usermode_api` is enabled:
271
272Kernel object hashing
273   The *gen_kobject_list.py* scans the *ELF DWARF*
274   debug data to find the address of the all kernel objects. This
275   list is passed to *gperf*, which generates a perfect hash function and
276   table of those addresses, then that output is optimized by
277   *process_gperf.py*, using known properties of our special case.
278
279.. figure:: build-postprocess-4.svg
280    :align: center
281    :alt: Zephyr's intermediate binary post-process IV
282    :figclass: align-center
283    :width: 80%
284
285When no intermediate binary post-processing is required then the first
286intermediate binary will be directly used as the final binary.
287
288Final binary
289++++++++++++
290
291The binary from the previous stage is incomplete, with empty and/or
292placeholder sections that must be filled in by, essentially, reflection.
293
294The link from the previous stage is repeated, this time with the missing
295pieces populated.
296
297.. figure:: build-build-phase-5.svg
298    :align: center
299    :alt: Zephyr's build final stage
300    :figclass: align-center
301    :width: 80%
302
303Post processing
304+++++++++++++++
305
306Finally, if necessary, the completed kernel is converted from *ELF* to
307the format expected by the loader and/or flash tool required by the
308target. This is accomplished in a straightforward manner with *objdump*.
309
310.. figure:: build-build-phase-6.svg
311    :align: center
312    :alt: Zephyr's build final stage post-process
313    :figclass: align-center
314    :width: 80%
315
316
317.. _build_system_scripts:
318
319Supporting Scripts and Tools
320============================
321
322The following is a detailed description of the scripts used during the build process.
323
324.. _gen_syscalls.py:
325
326:zephyr_file:`scripts/build/gen_syscalls.py`
327--------------------------------------------
328
329.. include:: ../../../scripts/build/gen_syscalls.py
330   :start-after: """
331   :end-before: """
332
333.. _gen_device_deps.py:
334
335:zephyr_file:`scripts/build/gen_device_deps.py`
336-----------------------------------------------
337
338.. include:: ../../../scripts/build/gen_device_deps.py
339   :start-after: """
340   :end-before: """
341
342.. _gen_kobject_list.py:
343
344:zephyr_file:`scripts/build/gen_kobject_list.py`
345------------------------------------------------
346
347.. include:: ../../../scripts/build/gen_kobject_list.py
348   :start-after: """
349   :end-before: """
350
351.. _gen_offset_header.py:
352
353:zephyr_file:`scripts/build/gen_offset_header.py`
354-------------------------------------------------
355
356.. include:: ../../../scripts/build/gen_offset_header.py
357   :start-after: """
358   :end-before: """
359
360.. _parse_syscalls.py:
361
362:zephyr_file:`scripts/build/parse_syscalls.py`
363----------------------------------------------
364
365
366.. include:: ../../../scripts/build/parse_syscalls.py
367   :start-after: """
368   :end-before: """
369
370.. _gen_idt.py:
371
372:zephyr_file:`arch/x86/gen_idt.py`
373----------------------------------
374
375.. include:: ../../../arch/x86/gen_idt.py
376   :start-after: """
377   :end-before: """
378
379.. _gen_gdt.py:
380
381:zephyr_file:`arch/x86/gen_gdt.py`
382----------------------------------
383
384.. include:: ../../../arch/x86/gen_gdt.py
385   :start-after: """
386   :end-before: """
387
388.. _gen_relocate_app.py:
389
390:zephyr_file:`scripts/build/gen_relocate_app.py`
391------------------------------------------------
392
393.. include:: ../../../scripts/build/gen_relocate_app.py
394   :start-after: """
395   :end-before: """
396
397.. _process_gperf.py:
398
399:zephyr_file:`scripts/build/process_gperf.py`
400---------------------------------------------
401
402.. include:: ../../../scripts/build/process_gperf.py
403   :start-after: """
404   :end-before: """
405
406:zephyr_file:`scripts/build/gen_app_partitions.py`
407--------------------------------------------------
408
409.. include:: ../../../scripts/build/gen_app_partitions.py
410   :start-after: """
411   :end-before: """
412
413.. _check_init_priorities.py:
414
415:zephyr_file:`scripts/build/check_init_priorities.py`
416-----------------------------------------------------
417
418.. include:: ../../../scripts/build/check_init_priorities.py
419   :start-after: """
420   :end-before: """
421