1NILFS2 2------ 3 4NILFS2 is a log-structured file system (LFS) supporting continuous 5snapshotting. In addition to versioning capability of the entire file 6system, users can even restore files mistakenly overwritten or 7destroyed just a few seconds ago. Since NILFS2 can keep consistency 8like conventional LFS, it achieves quick recovery after system 9crashes. 10 11NILFS2 creates a number of checkpoints every few seconds or per 12synchronous write basis (unless there is no change). Users can select 13significant versions among continuously created checkpoints, and can 14change them into snapshots which will be preserved until they are 15changed back to checkpoints. 16 17There is no limit on the number of snapshots until the volume gets 18full. Each snapshot is mountable as a read-only file system 19concurrently with its writable mount, and this feature is convenient 20for online backup. 21 22The userland tools are included in nilfs-utils package, which is 23available from the following download page. At least "mkfs.nilfs2", 24"mount.nilfs2", "umount.nilfs2", and "nilfs_cleanerd" (so called 25cleaner or garbage collector) are required. Details on the tools are 26described in the man pages included in the package. 27 28Project web page: https://nilfs.sourceforge.io/ 29Download page: https://nilfs.sourceforge.io/en/download.html 30List info: http://vger.kernel.org/vger-lists.html#linux-nilfs 31 32Caveats 33======= 34 35Features which NILFS2 does not support yet: 36 37 - atime 38 - extended attributes 39 - POSIX ACLs 40 - quotas 41 - fsck 42 - defragmentation 43 44Mount options 45============= 46 47NILFS2 supports the following mount options: 48(*) == default 49 50barrier(*) This enables/disables the use of write barriers. This 51nobarrier requires an IO stack which can support barriers, and 52 if nilfs gets an error on a barrier write, it will 53 disable again with a warning. 54errors=continue Keep going on a filesystem error. 55errors=remount-ro(*) Remount the filesystem read-only on an error. 56errors=panic Panic and halt the machine if an error occurs. 57cp=n Specify the checkpoint-number of the snapshot to be 58 mounted. Checkpoints and snapshots are listed by lscp 59 user command. Only the checkpoints marked as snapshot 60 are mountable with this option. Snapshot is read-only, 61 so a read-only mount option must be specified together. 62order=relaxed(*) Apply relaxed order semantics that allows modified data 63 blocks to be written to disk without making a 64 checkpoint if no metadata update is going. This mode 65 is equivalent to the ordered data mode of the ext3 66 filesystem except for the updates on data blocks still 67 conserve atomicity. This will improve synchronous 68 write performance for overwriting. 69order=strict Apply strict in-order semantics that preserves sequence 70 of all file operations including overwriting of data 71 blocks. That means, it is guaranteed that no 72 overtaking of events occurs in the recovered file 73 system after a crash. 74norecovery Disable recovery of the filesystem on mount. 75 This disables every write access on the device for 76 read-only mounts or snapshots. This option will fail 77 for r/w mounts on an unclean volume. 78discard This enables/disables the use of discard/TRIM commands. 79nodiscard(*) The discard/TRIM commands are sent to the underlying 80 block device when blocks are freed. This is useful 81 for SSD devices and sparse/thinly-provisioned LUNs. 82 83Ioctls 84====== 85 86There is some NILFS2 specific functionality which can be accessed by applications 87through the system call interfaces. The list of all NILFS2 specific ioctls are 88shown in the table below. 89 90Table of NILFS2 specific ioctls 91.............................................................................. 92 Ioctl Description 93 NILFS_IOCTL_CHANGE_CPMODE Change mode of given checkpoint between 94 checkpoint and snapshot state. This ioctl is 95 used in chcp and mkcp utilities. 96 97 NILFS_IOCTL_DELETE_CHECKPOINT Remove checkpoint from NILFS2 file system. 98 This ioctl is used in rmcp utility. 99 100 NILFS_IOCTL_GET_CPINFO Return info about requested checkpoints. This 101 ioctl is used in lscp utility and by 102 nilfs_cleanerd daemon. 103 104 NILFS_IOCTL_GET_CPSTAT Return checkpoints statistics. This ioctl is 105 used by lscp, rmcp utilities and by 106 nilfs_cleanerd daemon. 107 108 NILFS_IOCTL_GET_SUINFO Return segment usage info about requested 109 segments. This ioctl is used in lssu, 110 nilfs_resize utilities and by nilfs_cleanerd 111 daemon. 112 113 NILFS_IOCTL_SET_SUINFO Modify segment usage info of requested 114 segments. This ioctl is used by 115 nilfs_cleanerd daemon to skip unnecessary 116 cleaning operation of segments and reduce 117 performance penalty or wear of flash device 118 due to redundant move of in-use blocks. 119 120 NILFS_IOCTL_GET_SUSTAT Return segment usage statistics. This ioctl 121 is used in lssu, nilfs_resize utilities and 122 by nilfs_cleanerd daemon. 123 124 NILFS_IOCTL_GET_VINFO Return information on virtual block addresses. 125 This ioctl is used by nilfs_cleanerd daemon. 126 127 NILFS_IOCTL_GET_BDESCS Return information about descriptors of disk 128 block numbers. This ioctl is used by 129 nilfs_cleanerd daemon. 130 131 NILFS_IOCTL_CLEAN_SEGMENTS Do garbage collection operation in the 132 environment of requested parameters from 133 userspace. This ioctl is used by 134 nilfs_cleanerd daemon. 135 136 NILFS_IOCTL_SYNC Make a checkpoint. This ioctl is used in 137 mkcp utility. 138 139 NILFS_IOCTL_RESIZE Resize NILFS2 volume. This ioctl is used 140 by nilfs_resize utility. 141 142 NILFS_IOCTL_SET_ALLOC_RANGE Define lower limit of segments in bytes and 143 upper limit of segments in bytes. This ioctl 144 is used by nilfs_resize utility. 145 146NILFS2 usage 147============ 148 149To use nilfs2 as a local file system, simply: 150 151 # mkfs -t nilfs2 /dev/block_device 152 # mount -t nilfs2 /dev/block_device /dir 153 154This will also invoke the cleaner through the mount helper program 155(mount.nilfs2). 156 157Checkpoints and snapshots are managed by the following commands. 158Their manpages are included in the nilfs-utils package above. 159 160 lscp list checkpoints or snapshots. 161 mkcp make a checkpoint or a snapshot. 162 chcp change an existing checkpoint to a snapshot or vice versa. 163 rmcp invalidate specified checkpoint(s). 164 165To mount a snapshot, 166 167 # mount -t nilfs2 -r -o cp=<cno> /dev/block_device /snap_dir 168 169where <cno> is the checkpoint number of the snapshot. 170 171To unmount the NILFS2 mount point or snapshot, simply: 172 173 # umount /dir 174 175Then, the cleaner daemon is automatically shut down by the umount 176helper program (umount.nilfs2). 177 178Disk format 179=========== 180 181A nilfs2 volume is equally divided into a number of segments except 182for the super block (SB) and segment #0. A segment is the container 183of logs. Each log is composed of summary information blocks, payload 184blocks, and an optional super root block (SR): 185 186 ______________________________________________________ 187 | |SB| | Segment | Segment | Segment | ... | Segment | | 188 |_|__|_|____0____|____1____|____2____|_____|____N____|_| 189 0 +1K +4K +8M +16M +24M +(8MB x N) 190 . . (Typical offsets for 4KB-block) 191 . . 192 .______________________. 193 | log | log |... | log | 194 |__1__|__2__|____|__m__| 195 . . 196 . . 197 . . 198 .______________________________. 199 | Summary | Payload blocks |SR| 200 |_blocks__|_________________|__| 201 202The payload blocks are organized per file, and each file consists of 203data blocks and B-tree node blocks: 204 205 |<--- File-A --->|<--- File-B --->| 206 _______________________________________________________________ 207 | Data blocks | B-tree blocks | Data blocks | B-tree blocks | ... 208 _|_____________|_______________|_____________|_______________|_ 209 210 211Since only the modified blocks are written in the log, it may have 212files without data blocks or B-tree node blocks. 213 214The organization of the blocks is recorded in the summary information 215blocks, which contains a header structure (nilfs_segment_summary), per 216file structures (nilfs_finfo), and per block structures (nilfs_binfo): 217 218 _________________________________________________________________________ 219 | Summary | finfo | binfo | ... | binfo | finfo | binfo | ... | binfo |... 220 |_blocks__|___A___|_(A,1)_|_____|(A,Na)_|___B___|_(B,1)_|_____|(B,Nb)_|___ 221 222 223The logs include regular files, directory files, symbolic link files 224and several meta data files. The mata data files are the files used 225to maintain file system meta data. The current version of NILFS2 uses 226the following meta data files: 227 228 1) Inode file (ifile) -- Stores on-disk inodes 229 2) Checkpoint file (cpfile) -- Stores checkpoints 230 3) Segment usage file (sufile) -- Stores allocation state of segments 231 4) Data address translation file -- Maps virtual block numbers to usual 232 (DAT) block numbers. This file serves to 233 make on-disk blocks relocatable. 234 235The following figure shows a typical organization of the logs: 236 237 _________________________________________________________________________ 238 | Summary | regular file | file | ... | ifile | cpfile | sufile | DAT |SR| 239 |_blocks__|_or_directory_|_______|_____|_______|________|________|_____|__| 240 241 242To stride over segment boundaries, this sequence of files may be split 243into multiple logs. The sequence of logs that should be treated as 244logically one log, is delimited with flags marked in the segment 245summary. The recovery code of nilfs2 looks this boundary information 246to ensure atomicity of updates. 247 248The super root block is inserted for every checkpoints. It includes 249three special inodes, inodes for the DAT, cpfile, and sufile. Inodes 250of regular files, directories, symlinks and other special files, are 251included in the ifile. The inode of ifile itself is included in the 252corresponding checkpoint entry in the cpfile. Thus, the hierarchy 253among NILFS2 files can be depicted as follows: 254 255 Super block (SB) 256 | 257 v 258 Super root block (the latest cno=xx) 259 |-- DAT 260 |-- sufile 261 `-- cpfile 262 |-- ifile (cno=c1) 263 |-- ifile (cno=c2) ---- file (ino=i1) 264 : : |-- file (ino=i2) 265 `-- ifile (cno=xx) |-- file (ino=i3) 266 : : 267 `-- file (ino=yy) 268 ( regular file, directory, or symlink ) 269 270For detail on the format of each file, please see nilfs2_ondisk.h 271located at include/uapi/linux directory. 272 273There are no patents or other intellectual property that we protect 274with regard to the design of NILFS2. It is allowed to replicate the 275design in hopes that other operating systems could share (mount, read, 276write, etc.) data stored in this format. 277
