1.. _cmake-details: 2 3Build System (CMake) 4******************** 5 6 7CMake is used to build your application together with the Zephyr kernel. A 8CMake build is done in two stages. The first stage is called 9**configuration**. During configuration, the CMakeLists.txt build scripts are 10executed. After configuration is finished, CMake has an internal model of the 11Zephyr build, and can generate build scripts that are native to the host 12platform. 13 14CMake supports generating scripts for several build systems, but only Ninja and 15Make are tested and supported by Zephyr. After configuration, you begin the 16**build** stage by executing the generated build scripts. These build scripts 17can recompile the application without involving CMake following 18most code changes. However, after certain changes, the configuration step must 19be executed again before building. The build scripts can detect some of these 20situations and reconfigure automatically, but there are cases when this must be 21done manually. 22 23Zephyr uses CMake's concept of a 'target' to organize the build. A 24target can be an executable, a library, or a generated file. For 25application developers, the library target is the most important to 26understand. All source code that goes into a Zephyr build does so by 27being included in a library target, even application code. 28 29Library targets have source code, that is added through CMakeLists.txt 30build scripts like this: 31 32.. code-block:: cmake 33 34 target_sources(app PRIVATE src/main.c) 35 36In the above :file:`CMakeLists.txt`, an existing library target named ``app`` 37is configured to include the source file :file:`src/main.c`. The ``PRIVATE`` 38keyword indicates that we are modifying the internals of how the library is 39being built. Using the keyword ``PUBLIC`` would modify how other 40libraries that link with app are built. In this case, using ``PUBLIC`` 41would cause libraries that link with ``app`` to also include the 42source file :file:`src/main.c`, behavior that we surely do not want. The 43``PUBLIC`` keyword could however be useful when modifying the include 44paths of a target library. 45 46 47Build and Configuration Phases 48============================== 49 50The Zephyr build process can be divided into two main phases: a configuration 51phase (driven by CMake) and a build phase (driven by Make or Ninja). 52 53.. _build_configuration_phase: 54 55Configuration Phase 56------------------- 57 58The configuration phase begins when the user invokes *CMake* to generate a 59build system, specifying a source application directory and a board target. 60 61.. figure:: build-config-phase.svg 62 :align: center 63 :alt: Zephyr's build configuration phase 64 :figclass: align-center 65 :width: 80% 66 67CMake begins by processing the :file:`CMakeLists.txt` file in the application 68directory, which refers to the :file:`CMakeLists.txt` file in the Zephyr 69top-level directory, which in turn refers to :file:`CMakeLists.txt` files 70throughout the build tree (directly and indirectly). Its primary output is a 71set of Makefiles or Ninja files to drive the build process, but the CMake 72scripts also do some processing of their own, which is explained here. 73 74Note that paths beginning with :file:`build/` below refer to the build 75directory you create when running CMake. 76 77Devicetree 78 :file:`*.dts` (*devicetree source*) and :file:`*.dtsi` (*devicetree source 79 include*) files are collected from the target's architecture, SoC, board, 80 and application directories. 81 82 :file:`*.dtsi` files are included by :file:`*.dts` files via the C 83 preprocessor (often abbreviated *cpp*, which should not be confused with 84 C++). The C preprocessor is also used to merge in any devicetree 85 :file:`*.overlay` files, and to expand macros in :file:`*.dts`, 86 :file:`*.dtsi`, and :file:`*.overlay` files. The preprocessor output is 87 placed in :file:`build/zephyr/zephyr.dts.pre`. 88 89 The preprocessed devicetree sources are parsed by 90 :zephyr_file:`gen_defines.py <scripts/dts/gen_defines.py>` to generate a 91 :file:`build/zephyr/include/generated/devicetree_generated.h` header with 92 preprocessor macros. 93 94 Source code should access preprocessor macros generated from devicetree by 95 including the :zephyr_file:`devicetree.h <include/zephyr/devicetree.h>` header, 96 which includes :file:`devicetree_generated.h`. 97 98 :file:`gen_defines.py` also writes the final devicetree to 99 :file:`build/zephyr/zephyr.dts` in the build directory. This file's contents 100 may be useful for debugging. 101 102 If the devicetree compiler ``dtc`` is installed, it is run on 103 :file:`build/zephyr/zephyr.dts` to catch any extra warnings and errors 104 generated by this tool. The output from ``dtc`` is unused otherwise, and 105 this step is skipped if ``dtc`` is not installed. 106 107 The above is just a brief overview. For more information on devicetree, see 108 :ref:`dt-guide`. 109 110Kconfig 111 :file:`Kconfig` files define available configuration options for for the 112 target architecture, SoC, board, and application, as well as dependencies 113 between options. 114 115 Kconfig configurations are stored in *configuration files*. The initial 116 configuration is generated by merging configuration fragments from the board 117 and application (e.g. :file:`prj.conf`). 118 119 The output from Kconfig is an :file:`autoconf.h` header with preprocessor 120 assignments, and a :file:`.config` file that acts both as a saved 121 configuration and as configuration output (used by CMake). The definitions in 122 :file:`autoconf.h` are automatically exposed at compile time, so there is no 123 need to include this header. 124 125 Information from devicetree is available to Kconfig, through the functions 126 defined in :zephyr_file:`kconfigfunctions.py 127 <scripts/kconfig/kconfigfunctions.py>`. 128 129 See :ref:`the Kconfig section of the manual <kconfig>` for more information. 130 131Build Phase 132----------- 133 134The build phase begins when the user invokes ``make`` or ``ninja``. Its 135ultimate output is a complete Zephyr application in a format suitable for 136loading/flashing on the desired target board (:file:`zephyr.elf`, 137:file:`zephyr.hex`, etc.) The build phase can be broken down, conceptually, 138into four stages: the pre-build, first-pass binary, final binary, and 139post-processing. 140 141Pre-build 142+++++++++ 143 144Pre-build occurs before any source files are compiled, because during 145this phase header files used by the source files are generated. 146 147Offset generation 148 Access to high-level data structures and members is sometimes 149 required when the definitions of those structures is not 150 immediately accessible (e.g., assembly language). The generation of 151 *offsets.h* (by *gen_offset_header.py*) facilitates this. 152 153System call boilerplate 154 The *gen_syscall.py* and *parse_syscalls.py* scripts work 155 together to bind potential system call functions with their 156 implementations. 157 158.. figure:: build-build-phase-1.svg 159 :align: center 160 :alt: Zephyr's build stage I 161 :figclass: align-center 162 :width: 80% 163 164Intermediate binaries 165+++++++++++++++++++++ 166 167Compilation proper begins with the first intermediate binary. Source files (C 168and assembly) are collected from various subsystems (which ones is 169decided during the configuration phase), and compiled into archives 170(with reference to header files in the tree, as well as those 171generated during the configuration phase and the pre-build stage(s)). 172 173.. figure:: build-build-phase-2.svg 174 :align: center 175 :alt: Zephyr's build stage II 176 :figclass: align-center 177 :width: 80% 178 179The exact number of intermediate binaries is decided during the configuration 180phase. 181 182If memory protection is enabled, then: 183 184Partition grouping 185 The *gen_app_partitions.py* script scans all the 186 generated archives and outputs linker scripts to ensure that 187 application partitions are properly grouped and aligned for the 188 target’s memory protection hardware. 189 190Then *cpp* is used to combine linker script fragments from the target’s 191architecture/SoC, the kernel tree, optionally the partition output if 192memory protection is enabled, and any other fragments selected during 193the configuration process, into a *linker.cmd* file. The compiled 194archives are then linked with *ld* as specified in the 195*linker.cmd*. 196 197Unfixed size binary 198 The unfixed size intermediate binary is produced when :ref:`usermode_api` 199 is enabled or :ref:`devicetree` is in use. 200 It produces a binary where sizes are not fixed and thus it may be used 201 by post-process steps that will impact the size of the final binary. 202 203.. figure:: build-build-phase-3.svg 204 :align: center 205 :alt: Zephyr's build stage III 206 :figclass: align-center 207 :width: 80% 208 209Fixed size binary 210 The fixed size intermediate binary is produced when :ref:`usermode_api` 211 is enabled or when generated IRQ tables are used, 212 :kconfig:option:`CONFIG_GEN_ISR_TABLES` 213 It produces a binary where sizes are fixed and thus the size must not change 214 between the intermediate binary and the final binary. 215 216.. figure:: build-build-phase-4.svg 217 :align: center 218 :alt: Zephyr's build stage IV 219 :figclass: align-center 220 :width: 80% 221 222Intermediate binaries post-processing 223+++++++++++++++++++++++++++++++++++++ 224 225The binaries from the previous stage are incomplete, with empty and/or 226placeholder sections that must be filled in by, essentially, reflection. 227 228To complete the build procedure the following scripts are executed on the 229intermediate binaries to produce the missing pieces needed for the final 230binary. 231 232When :ref:`usermode_api` is enabled: 233 234Partition alignment 235 The *gen_app_partitions.py* script scans the unfixed size binary and 236 generates an app shared memory aligned linker script snippet where the 237 partitions are sorted in descending order. 238 239.. figure:: build-postprocess-1.svg 240 :align: center 241 :alt: Zephyr's intermediate binary post-process I 242 :figclass: align-center 243 :width: 80% 244 245When :ref:`devicetree` is used: 246 247Device dependencies 248 The *gen_device_deps.py* script scans the unfixed size binary to determine 249 relationships between devices that were recorded from devicetree data, 250 and replaces the encoded relationships with values that are optimized to 251 locate the devices actually present in the application. 252 253.. figure:: build-postprocess-2.svg 254 :align: center 255 :alt: Zephyr's intermediate binary post-process II 256 :figclass: align-center 257 :width: 80% 258 259When :kconfig:option:`CONFIG_GEN_ISR_TABLES` is enabled: 260 The *gen_isr_tables.py* script scans the fixed size binary and creates 261 an isr_tables.c source file with a hardware vector table and/or software 262 IRQ table. 263 264.. figure:: build-postprocess-3.svg 265 :align: center 266 :alt: Zephyr's intermediate binary post-process III 267 :figclass: align-center 268 :width: 80% 269 270When :ref:`usermode_api` is enabled: 271 272Kernel object hashing 273 The *gen_kobject_list.py* scans the *ELF DWARF* 274 debug data to find the address of the all kernel objects. This 275 list is passed to *gperf*, which generates a perfect hash function and 276 table of those addresses, then that output is optimized by 277 *process_gperf.py*, using known properties of our special case. 278 279.. figure:: build-postprocess-4.svg 280 :align: center 281 :alt: Zephyr's intermediate binary post-process IV 282 :figclass: align-center 283 :width: 80% 284 285When no intermediate binary post-processing is required then the first 286intermediate binary will be directly used as the final binary. 287 288Final binary 289++++++++++++ 290 291The binary from the previous stage is incomplete, with empty and/or 292placeholder sections that must be filled in by, essentially, reflection. 293 294The link from the previous stage is repeated, this time with the missing 295pieces populated. 296 297.. figure:: build-build-phase-5.svg 298 :align: center 299 :alt: Zephyr's build final stage 300 :figclass: align-center 301 :width: 80% 302 303Post processing 304+++++++++++++++ 305 306Finally, if necessary, the completed kernel is converted from *ELF* to 307the format expected by the loader and/or flash tool required by the 308target. This is accomplished in a straightforward manner with *objdump*. 309 310.. figure:: build-build-phase-6.svg 311 :align: center 312 :alt: Zephyr's build final stage post-process 313 :figclass: align-center 314 :width: 80% 315 316 317.. _build_system_scripts: 318 319Supporting Scripts and Tools 320============================ 321 322The following is a detailed description of the scripts used during the build process. 323 324.. _gen_syscalls.py: 325 326:zephyr_file:`scripts/build/gen_syscalls.py` 327-------------------------------------------- 328 329.. include:: ../../../scripts/build/gen_syscalls.py 330 :start-after: """ 331 :end-before: """ 332 333.. _gen_device_deps.py: 334 335:zephyr_file:`scripts/build/gen_device_deps.py` 336----------------------------------------------- 337 338.. include:: ../../../scripts/build/gen_device_deps.py 339 :start-after: """ 340 :end-before: """ 341 342.. _gen_kobject_list.py: 343 344:zephyr_file:`scripts/build/gen_kobject_list.py` 345------------------------------------------------ 346 347.. include:: ../../../scripts/build/gen_kobject_list.py 348 :start-after: """ 349 :end-before: """ 350 351.. _gen_offset_header.py: 352 353:zephyr_file:`scripts/build/gen_offset_header.py` 354------------------------------------------------- 355 356.. include:: ../../../scripts/build/gen_offset_header.py 357 :start-after: """ 358 :end-before: """ 359 360.. _parse_syscalls.py: 361 362:zephyr_file:`scripts/build/parse_syscalls.py` 363---------------------------------------------- 364 365 366.. include:: ../../../scripts/build/parse_syscalls.py 367 :start-after: """ 368 :end-before: """ 369 370.. _gen_idt.py: 371 372:zephyr_file:`arch/x86/gen_idt.py` 373---------------------------------- 374 375.. include:: ../../../arch/x86/gen_idt.py 376 :start-after: """ 377 :end-before: """ 378 379.. _gen_gdt.py: 380 381:zephyr_file:`arch/x86/gen_gdt.py` 382---------------------------------- 383 384.. include:: ../../../arch/x86/gen_gdt.py 385 :start-after: """ 386 :end-before: """ 387 388.. _gen_relocate_app.py: 389 390:zephyr_file:`scripts/build/gen_relocate_app.py` 391------------------------------------------------ 392 393.. include:: ../../../scripts/build/gen_relocate_app.py 394 :start-after: """ 395 :end-before: """ 396 397.. _process_gperf.py: 398 399:zephyr_file:`scripts/build/process_gperf.py` 400--------------------------------------------- 401 402.. include:: ../../../scripts/build/process_gperf.py 403 :start-after: """ 404 :end-before: """ 405 406:zephyr_file:`scripts/build/gen_app_partitions.py` 407-------------------------------------------------- 408 409.. include:: ../../../scripts/build/gen_app_partitions.py 410 :start-after: """ 411 :end-before: """ 412 413.. _check_init_priorities.py: 414 415:zephyr_file:`scripts/build/check_init_priorities.py` 416----------------------------------------------------- 417 418.. include:: ../../../scripts/build/check_init_priorities.py 419 :start-after: """ 420 :end-before: """ 421
