Lines Matching +full:sector +full:- +full:count
5 Zephyr Memory Storage is a new key-value storage system that is designed to work with all types
6 of non-volatile storage technologies. It supports classical on-chip NOR flash as well as new
12 ZMS divides the memory space into sectors (minimum 2), and each sector is filled with key-value
15 The key-value pair is divided into two parts:
17 - The key part is written in an ATE (Allocation Table Entry) called "ID-ATE" which is stored
18 starting from the bottom of the sector
19 - The value part is defined as "DATA" and is stored raw starting from the top of the sector
21 Additionally, for each sector we store at the last positions Header-ATEs which are ATEs that
22 are needed for the sector to describe its status (closed, open) and the current version of ZMS.
24 When the current sector is full we verify first that the following sector is empty, we garbage
25 collect the N+2 sector (where N is the current sector number) by moving the valid ATEs to the
26 N+1 empty sector, we erase the garbage collected sector and then we close the current sector by
28 Afterwards we move forward to the next sector and start writing entries again.
31 the first sector after garbage collecting it and erasing its content.
33 Composition of a sector
35 A sector is organized in this form (example with 3 sectors):
37 .. list-table::
39 :header-rows: 1
41 * - Sector 0 (closed)
42 - Sector 1 (open)
43 - Sector 2 (empty)
44 * - Data_a0
45 - Data_b0
46 - Data_c0
47 * - Data_a1
48 - Data_b1
49 - Data_c1
50 * - Data_a2
51 - Data_b2
52 - Data_c2
53 * - GC_done
54 - .
55 - .
56 * - .
57 - .
58 - .
59 * - .
60 - .
61 - .
62 * - .
63 - ATE_b2
64 - ATE_c2
65 * - ATE_a2
66 - ATE_b1
67 - ATE_c1
68 * - ATE_a1
69 - ATE_b0
70 - ATE_c0
71 * - ATE_a0
72 - GC_done
73 - GC_done
74 * - Close (cyc=1)
75 - Close (cyc=1)
76 - Close (cyc=1)
77 * - Empty (cyc=1)
78 - Empty (cyc=2)
79 - Empty (cyc=2)
81 Definition of each element in the sector
84 ``Empty ATE:`` is written when erasing a sector (last position of the sector).
86 ``Close ATE:`` is written when closing a sector (second to last position of the sector).
88 ``GC_done ATE:`` is written to indicate that the next sector has been already garbage
89 collected. This ATE could be in any position of the sector.
91 ``ID-ATE:`` are entries that contain a 32 bits Key and describe where the data is stored, its
94 ``Data:`` is the actual value associated to the ID-ATE
108 .. code-block:: c
114 /** Storage system is split into sectors, each sector size must be multiple of
115 * erase-blocks if the device has erase capabilities
128 As ZMS has a fast-forward write mechanism, we must find the last sector and the last pointer of
130 It must look for a closed sector followed by an open one, then within the open sector, it finds
132 After that, it checks that the sector after this one is empty, or it will erase it.
134 ZMS ID-Data write
139 If we must perform a write, then an ATE and Data (if not a delete) are written in the sector.
140 If the sector is full (cannot hold the current data + ATE) we have to move to the next sector,
141 garbage collect the sector after the newly opened one then erase it.
150 If history count is provided that is different than 0, older data with same ID is retrieved.
164 Each sector has a lead cycle counter which is a uin8_t that is used to validate all the other
168 Each time an ATE is moved from a sector to another it must get the cycle counter of the
169 destination sector.
170 To erase a sector, the cycle counter of the empty ATE is incremented and a single write of the
172 All the ATEs in that sector become invalid.
177 To close a sector a close ATE is added at the end of the sector and it must have the same cycle
179 When closing a sector, all the remaining space that has not been used is filled with garbage data
186 When calling a ZMS write, the current sector could be almost full and we need to trigger the GC
187 to switch to the next sector.
190 ZMS adds an API for the application to get the current remaining free space in a sector.
191 The application could then decide when needed to switch to the next sector if the current one is
192 almost full and of course it will trigger the garbage collection on the next sector.
200 .. code-block:: c
204 uint8_t cycle_cnt; /* cycle counter for non-erasable devices */
205 uint16_t len; /* data len within sector */
210 uint32_t offset; /* data offset within sector */
227 .. _free-space:
229 Available space for user data (key-value pairs)
232 For both scenarios ZMS should always have an empty sector to be able to perform the
235 Key-value pairs and keep one sector empty to be able to launch GC.
236 The empty sector will rotate between the 4 sectors in the partition.
238 .. note:: The maximum single data length that could be written at once in a sector is 64K
245 at the top of the sector.
251 \small\frac{(NUM\_SECTORS - 1) \times (SECTOR\_SIZE - (5 \times ATE\_SIZE))}{2}
257 ``SECTOR_SIZE:`` Size of the sector
268 Large data values ( > 8 bytes) are stored separately at the top of the sector.
271 16 bytes of ATE must be added at the bottom of the sector.
277 Each Key-value pair needs an extra 16 bytes for ATE which makes it possible to store 11 pairs
281 .. _wear-leveling:
287 Using storage systems that rely on an erase-value (NVS as an example) will need to emulate the
290 ZMS uses a cycle count mechanism that avoids emulating erase operation for these devices.
291 It also guarantees that every memory location is written only once for each cycle of sector write.
293 As an example, to erase a 4096 bytes sector on a non-erasable device using NVS, 256 flash writes
294 must be performed (supposing that write-block-size=16 bytes), while using ZMS only 1 write of
298 is moving some blocks from one sector to another.
303 See :ref:`free-space`.
311 memory cells cannot be overwritten) and for non-erasable storage devices memory cells can be
318 As we have 944 bytes available for ATEs for each sector, and because ZMS is a fast-forward
319 storage system, we are going to rewrite the first location of the first sector after
341 …\small\frac{(SECTOR\_EFFECTIVE\_SIZE \times SECTOR\_NUMBER \times MAX\_NUM\_WRITES)}{(TOTAL\_EFFEC…
345 ``SECTOR_EFFECTIVE_SIZE``: is the size sector - header_size(80 bytes)
364 --------
365 - Supports non-erasable devices (only one write operation to erase a sector)
366 - Supports large partition size and sector size (64 bits address space)
367 - Supports 32-bit IDs to store ID/Value pairs
368 - Small sized data ( <= 8 bytes) are stored in the ATE itself
369 - Built-in Data CRC32 (included in the ATE)
370 - Versioning of ZMS (to handle future evolution)
371 - Supports large write-block-size (Only for platforms that need this)
376 - Add multiple format ATE support to be able to use ZMS with different ATE formats that satisfies
378 - Add the possibility to skip garbage collector for some application usage where ID/value pairs
381 - Divide IDs into namespaces and allocate IDs on demand from application to handle collisions
383 - Add the possibility to retrieve the wear out value of the device based on the cycle count value
384 - Add a recovery function that can recover a storage partition if something went wrong
385 - Add a library/application to allow migration from NVS entries to ZMS entries
386 - Add the possibility to force formatting the storage partition to the ZMS format if something
392 but simpler, non-hierarchical ones).
398 - If you are using a non-erasable technology device like RRAM or MRAM, :ref:`ZMS <zms_api>` is defi…
401 - For devices with large write_block_size and/or needs a sector size that is different than the
404 - For classical flash technology devices, :ref:`NVS <nvs_api>` is recommended as it has low footpri…
408 - If your application needs more than 64K IDs for storage, :ref:`ZMS <zms_api>` is recommended here…
409 has a 32-bit ID field.
410 - If your application is working in a FIFO mode (First-in First-out) then :ref:`FCB <fcb_api>` is
416 life expectancy of the device described in this section: :ref:`wear-leveling`.
421 Sector size and count
424 - The total size of the storage partition should be well dimensioned to achieve the best
427 in the documentation. See :ref:`free-space`.
428 We recommend choosing a storage partition that can hold double the size of the key-value pairs
430 - The size of a sector needs to be dimensioned to hold the maximum data length that will be stored.
431 Increasing the size of a sector will slow down the garbage collection operation which will
435 - For some subsystems like :ref:`Settings <settings_api>`, all path-value pairs are split into two …
437 - Using small data to store in the ZMS entries can increase the performance, as this data is
445 - When using ZMS API directly, the recommended cache size should be, at least, equal to
447 - Each additional cache entry will add 8 bytes to your RAM usage. Cache size should be carefully
449 - If you use ZMS through :ref:`Settings <settings_api>`, you have to take into account that each Se…
456 A sample of how ZMS can be used is supplied in :zephyr:code-sample:`zms`.