1# Nanopb: Overview
2
3Nanopb is an ANSI-C library for encoding and decoding messages in
4Google's [Protocol Buffers](https://developers.google.com/protocol-buffers/docs/reference/overview)
5format with minimal requirements for RAM and code space. It is primarily
6suitable for 32-bit microcontrollers.
7
8Documentation version
9---------------------
10
11This documentation applies for nanopb 0.4.0 and later versions. For
12documentation of older releases,
13[see here](https://github.com/nanopb/nanopb/blob/maintenance_0.3/docs/index.rst).
14
15Overall structure
16-----------------
17
18For the runtime program, you always need `pb.h` for type declarations
19and `pb_common.h/c` for base functions. Depending on whether you want
20to encode, decode, or both, you also need `pb_encode.h/c` or
21`pb_decode.h/c`.
22
23The high-level encoding and decoding functions take a pointer to
24`pb_msgdesc_t` structure, which describes the fields of a message
25structure. Usually you want these autogenerated from a `.proto` file.
26The tool script `nanopb_generator.py` accomplishes this.
27
28![Image: Nanopb generator flow](generator_flow.svg)
29
30So a typical project might include these files:
31
321. Nanopb runtime library:
33    -   pb.h
34    -   pb_common.h and pb_common.c (always needed)
35    -   pb_decode.h and pb_decode.c (needed for decoding messages)
36    -   pb_encode.h and pb_encode.c (needed for encoding messages)
37
382. Protocol description (you can have many):
39    -   person.proto (just an example)
40    -   person.pb.c (autogenerated, contains message descriptors)
41    -   person.pb.h (autogenerated, contains type declarations and macros)
42
43Features and limitations
44------------------------
45
46**Features**
47
481)  Pure C runtime
492)  Small code size (5--20 kB depending on processor and compilation options, plus any message definitions)
503)  Small ram usage (typically \~1 kB stack, plus any message structs)
514)  Allows specifying maximum size for strings and arrays, so that they can be allocated statically.
525)  No malloc needed: everything can be allocated statically or on the stack. Optional malloc support available.
536)  You can use either encoder or decoder alone to cut the code size in half.
547)  Support for most protobuf features, including: all data types,
55    nested submessages, default values, repeated and optional fields,
56    oneofs, packed arrays, extension fields.
578)  Callback mechanism for handling messages larger than can fit in available RAM.
589)  Extensive set of tests.
59
60**Limitations**
61
621)  Some speed has been sacrificed for code size.
632)  Encoding is focused on writing to streams. For memory buffers only it could be made more efficient.
643)  The deprecated Protocol Buffers feature called "groups" is not supported.
654)  Fields in the generated structs are ordered by the tag number, instead of the natural ordering in .proto file. (Since nanopb-0.4.2 this can be configured with `sort_by_tag` setting.)
665)  Unknown fields are not preserved when decoding and re-encoding a message.
676)  Reflection (runtime introspection) is not supported. E.g. you can't request a field by giving its name in a string.
687)  Numeric arrays are always encoded as packed, even if not marked as packed in .proto.
698)  Cyclic references between messages are supported only in callback and malloc mode.
709)  Nanopb doesn't have a stable ABI (application binary interface)
71    between versions, so using it as a shared library (.so / .dll)
72    requires extra care.
73
74Getting started
75---------------
76
77For starters, consider this simple message:
78
79~~~~ protobuf
80message Example {
81    required int32 value = 1;
82}
83~~~~
84
85Save this in `message.proto` and compile it:
86
87    user@host:~$ python nanopb/generator/nanopb_generator.py message.proto
88
89You should now have in `message.pb.h`:
90
91    typedef struct {
92       int32_t value;
93    } Example;
94
95    extern const pb_msgdesc_t Example_msg;
96    #define Example_fields &Example_msg
97
98Then you have to include the nanopb headers and the generated header:
99
100    #include <pb_encode.h>
101    #include "message.pb.h"
102
103Now in your main program do this to encode a message:
104
105    Example mymessage = {42};
106    uint8_t buffer[10];
107    pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
108    pb_encode(&stream, Example_fields, &mymessage);
109
110After that, buffer will contain the encoded message. The number of bytes
111in the message is stored in `stream.bytes_written`. You can feed the
112message to `protoc --decode=Example message.proto` to verify its
113validity.
114
115For a complete example of the simple case, see [examples/simple/simple.c](https://github.com/nanopb/nanopb/blob/master/examples/simple/simple.c).
116For a more complex example with network interface, see the [examples/network_server](https://github.com/nanopb/nanopb/tree/master/examples/network_server) subdirectory.
117
118Compiler requirements
119---------------------
120
121Nanopb should compile with most ansi-C compatible compilers. It however
122requires a few header files to be available:
123
1241)  `string.h`, with these functions: `strlen`, `memcpy`, `memset`
1252)  `stdint.h`, for definitions of `int32_t` etc.
1263)  `stddef.h`, for definition of `size_t`
1274)  `stdbool.h`, for definition of `bool`
1285)  `limits.h`, for definition of `CHAR_BIT`
129
130If these header files do not come with your compiler, you can use the
131file `extra/pb_syshdr.h` instead. It contains an example of how to
132provide the dependencies. You may have to edit it a bit to suit your
133custom platform.
134
135To use the pb_syshdr.h, define `PB_SYSTEM_HEADER` as
136`"pb_syshdr.h"` (including the quotes). Similarly, you can provide a
137custom include file, which should provide all the dependencies listed
138above.
139
140Running the test cases
141----------------------
142
143Extensive unittests and test cases are included under the `tests`
144folder.
145
146To build the tests, you will need the [scons](http://www.scons.org/)
147build system. The tests should be runnable on most platforms. Windows
148and Linux builds are regularly tested. The tests also support embedded
149targets: STM32 (ARM Cortex-M) and AVR builds are regularly tested.
150
151In addition to the build system, you will also need a working Google
152Protocol Buffers `protoc` compiler, and the Python bindings for Protocol
153Buffers.
154
155Easiest way to install dependencies is to use the Python package manager
156[pip](https://pypi.org/project/pip/), which works on all platforms supported by Python:
157
158    pip3 install scons protobuf grpcio-tools
159