linux/Documentation/filesystems/nilfs2.rst
<<
>>
Prefs
   1.. SPDX-License-Identifier: GPL-2.0
   2
   3======
   4NILFS2
   5======
   6
   7NILFS2 is a log-structured file system (LFS) supporting continuous
   8snapshotting.  In addition to versioning capability of the entire file
   9system, users can even restore files mistakenly overwritten or
  10destroyed just a few seconds ago.  Since NILFS2 can keep consistency
  11like conventional LFS, it achieves quick recovery after system
  12crashes.
  13
  14NILFS2 creates a number of checkpoints every few seconds or per
  15synchronous write basis (unless there is no change).  Users can select
  16significant versions among continuously created checkpoints, and can
  17change them into snapshots which will be preserved until they are
  18changed back to checkpoints.
  19
  20There is no limit on the number of snapshots until the volume gets
  21full.  Each snapshot is mountable as a read-only file system
  22concurrently with its writable mount, and this feature is convenient
  23for online backup.
  24
  25The userland tools are included in nilfs-utils package, which is
  26available from the following download page.  At least "mkfs.nilfs2",
  27"mount.nilfs2", "umount.nilfs2", and "nilfs_cleanerd" (so called
  28cleaner or garbage collector) are required.  Details on the tools are
  29described in the man pages included in the package.
  30
  31:Project web page:    https://nilfs.sourceforge.io/
  32:Download page:       https://nilfs.sourceforge.io/en/download.html
  33:List info:           http://vger.kernel.org/vger-lists.html#linux-nilfs
  34
  35Caveats
  36=======
  37
  38Features which NILFS2 does not support yet:
  39
  40        - atime
  41        - extended attributes
  42        - POSIX ACLs
  43        - quotas
  44        - fsck
  45        - defragmentation
  46
  47Mount options
  48=============
  49
  50NILFS2 supports the following mount options:
  51(*) == default
  52
  53======================= =======================================================
  54barrier(*)              This enables/disables the use of write barriers.  This
  55nobarrier               requires an IO stack which can support barriers, and
  56                        if nilfs gets an error on a barrier write, it will
  57                        disable again with a warning.
  58errors=continue         Keep going on a filesystem error.
  59errors=remount-ro(*)    Remount the filesystem read-only on an error.
  60errors=panic            Panic and halt the machine if an error occurs.
  61cp=n                    Specify the checkpoint-number of the snapshot to be
  62                        mounted.  Checkpoints and snapshots are listed by lscp
  63                        user command.  Only the checkpoints marked as snapshot
  64                        are mountable with this option.  Snapshot is read-only,
  65                        so a read-only mount option must be specified together.
  66order=relaxed(*)        Apply relaxed order semantics that allows modified data
  67                        blocks to be written to disk without making a
  68                        checkpoint if no metadata update is going.  This mode
  69                        is equivalent to the ordered data mode of the ext3
  70                        filesystem except for the updates on data blocks still
  71                        conserve atomicity.  This will improve synchronous
  72                        write performance for overwriting.
  73order=strict            Apply strict in-order semantics that preserves sequence
  74                        of all file operations including overwriting of data
  75                        blocks.  That means, it is guaranteed that no
  76                        overtaking of events occurs in the recovered file
  77                        system after a crash.
  78norecovery              Disable recovery of the filesystem on mount.
  79                        This disables every write access on the device for
  80                        read-only mounts or snapshots.  This option will fail
  81                        for r/w mounts on an unclean volume.
  82discard                 This enables/disables the use of discard/TRIM commands.
  83nodiscard(*)            The discard/TRIM commands are sent to the underlying
  84                        block device when blocks are freed.  This is useful
  85                        for SSD devices and sparse/thinly-provisioned LUNs.
  86======================= =======================================================
  87
  88Ioctls
  89======
  90
  91There is some NILFS2 specific functionality which can be accessed by applications
  92through the system call interfaces. The list of all NILFS2 specific ioctls are
  93shown in the table below.
  94
  95Table of NILFS2 specific ioctls:
  96
  97 ============================== ===============================================
  98 Ioctl                          Description
  99 ============================== ===============================================
 100 NILFS_IOCTL_CHANGE_CPMODE      Change mode of given checkpoint between
 101                                checkpoint and snapshot state. This ioctl is
 102                                used in chcp and mkcp utilities.
 103
 104 NILFS_IOCTL_DELETE_CHECKPOINT  Remove checkpoint from NILFS2 file system.
 105                                This ioctl is used in rmcp utility.
 106
 107 NILFS_IOCTL_GET_CPINFO         Return info about requested checkpoints. This
 108                                ioctl is used in lscp utility and by
 109                                nilfs_cleanerd daemon.
 110
 111 NILFS_IOCTL_GET_CPSTAT         Return checkpoints statistics. This ioctl is
 112                                used by lscp, rmcp utilities and by
 113                                nilfs_cleanerd daemon.
 114
 115 NILFS_IOCTL_GET_SUINFO         Return segment usage info about requested
 116                                segments. This ioctl is used in lssu,
 117                                nilfs_resize utilities and by nilfs_cleanerd
 118                                daemon.
 119
 120 NILFS_IOCTL_SET_SUINFO         Modify segment usage info of requested
 121                                segments. This ioctl is used by
 122                                nilfs_cleanerd daemon to skip unnecessary
 123                                cleaning operation of segments and reduce
 124                                performance penalty or wear of flash device
 125                                due to redundant move of in-use blocks.
 126
 127 NILFS_IOCTL_GET_SUSTAT         Return segment usage statistics. This ioctl
 128                                is used in lssu, nilfs_resize utilities and
 129                                by nilfs_cleanerd daemon.
 130
 131 NILFS_IOCTL_GET_VINFO          Return information on virtual block addresses.
 132                                This ioctl is used by nilfs_cleanerd daemon.
 133
 134 NILFS_IOCTL_GET_BDESCS         Return information about descriptors of disk
 135                                block numbers. This ioctl is used by
 136                                nilfs_cleanerd daemon.
 137
 138 NILFS_IOCTL_CLEAN_SEGMENTS     Do garbage collection operation in the
 139                                environment of requested parameters from
 140                                userspace. This ioctl is used by
 141                                nilfs_cleanerd daemon.
 142
 143 NILFS_IOCTL_SYNC               Make a checkpoint. This ioctl is used in
 144                                mkcp utility.
 145
 146 NILFS_IOCTL_RESIZE             Resize NILFS2 volume. This ioctl is used
 147                                by nilfs_resize utility.
 148
 149 NILFS_IOCTL_SET_ALLOC_RANGE    Define lower limit of segments in bytes and
 150                                upper limit of segments in bytes. This ioctl
 151                                is used by nilfs_resize utility.
 152 ============================== ===============================================
 153
 154NILFS2 usage
 155============
 156
 157To use nilfs2 as a local file system, simply::
 158
 159 # mkfs -t nilfs2 /dev/block_device
 160 # mount -t nilfs2 /dev/block_device /dir
 161
 162This will also invoke the cleaner through the mount helper program
 163(mount.nilfs2).
 164
 165Checkpoints and snapshots are managed by the following commands.
 166Their manpages are included in the nilfs-utils package above.
 167
 168  ====     ===========================================================
 169  lscp     list checkpoints or snapshots.
 170  mkcp     make a checkpoint or a snapshot.
 171  chcp     change an existing checkpoint to a snapshot or vice versa.
 172  rmcp     invalidate specified checkpoint(s).
 173  ====     ===========================================================
 174
 175To mount a snapshot::
 176
 177 # mount -t nilfs2 -r -o cp=<cno> /dev/block_device /snap_dir
 178
 179where <cno> is the checkpoint number of the snapshot.
 180
 181To unmount the NILFS2 mount point or snapshot, simply::
 182
 183 # umount /dir
 184
 185Then, the cleaner daemon is automatically shut down by the umount
 186helper program (umount.nilfs2).
 187
 188Disk format
 189===========
 190
 191A nilfs2 volume is equally divided into a number of segments except
 192for the super block (SB) and segment #0.  A segment is the container
 193of logs.  Each log is composed of summary information blocks, payload
 194blocks, and an optional super root block (SR)::
 195
 196   ______________________________________________________
 197  | |SB| | Segment | Segment | Segment | ... | Segment | |
 198  |_|__|_|____0____|____1____|____2____|_____|____N____|_|
 199  0 +1K +4K       +8M       +16M      +24M  +(8MB x N)
 200       .             .            (Typical offsets for 4KB-block)
 201    .                  .
 202  .______________________.
 203  | log | log |... | log |
 204  |__1__|__2__|____|__m__|
 205        .       .
 206      .               .
 207    .                       .
 208  .______________________________.
 209  | Summary | Payload blocks  |SR|
 210  |_blocks__|_________________|__|
 211
 212The payload blocks are organized per file, and each file consists of
 213data blocks and B-tree node blocks::
 214
 215    |<---       File-A        --->|<---       File-B        --->|
 216   _______________________________________________________________
 217    | Data blocks | B-tree blocks | Data blocks | B-tree blocks | ...
 218   _|_____________|_______________|_____________|_______________|_
 219
 220
 221Since only the modified blocks are written in the log, it may have
 222files without data blocks or B-tree node blocks.
 223
 224The organization of the blocks is recorded in the summary information
 225blocks, which contains a header structure (nilfs_segment_summary), per
 226file structures (nilfs_finfo), and per block structures (nilfs_binfo)::
 227
 228  _________________________________________________________________________
 229 | Summary | finfo | binfo | ... | binfo | finfo | binfo | ... | binfo |...
 230 |_blocks__|___A___|_(A,1)_|_____|(A,Na)_|___B___|_(B,1)_|_____|(B,Nb)_|___
 231
 232
 233The logs include regular files, directory files, symbolic link files
 234and several meta data files.  The mata data files are the files used
 235to maintain file system meta data.  The current version of NILFS2 uses
 236the following meta data files::
 237
 238 1) Inode file (ifile)             -- Stores on-disk inodes
 239 2) Checkpoint file (cpfile)       -- Stores checkpoints
 240 3) Segment usage file (sufile)    -- Stores allocation state of segments
 241 4) Data address translation file  -- Maps virtual block numbers to usual
 242    (DAT)                             block numbers.  This file serves to
 243                                      make on-disk blocks relocatable.
 244
 245The following figure shows a typical organization of the logs::
 246
 247  _________________________________________________________________________
 248 | Summary | regular file | file  | ... | ifile | cpfile | sufile | DAT |SR|
 249 |_blocks__|_or_directory_|_______|_____|_______|________|________|_____|__|
 250
 251
 252To stride over segment boundaries, this sequence of files may be split
 253into multiple logs.  The sequence of logs that should be treated as
 254logically one log, is delimited with flags marked in the segment
 255summary.  The recovery code of nilfs2 looks this boundary information
 256to ensure atomicity of updates.
 257
 258The super root block is inserted for every checkpoints.  It includes
 259three special inodes, inodes for the DAT, cpfile, and sufile.  Inodes
 260of regular files, directories, symlinks and other special files, are
 261included in the ifile.  The inode of ifile itself is included in the
 262corresponding checkpoint entry in the cpfile.  Thus, the hierarchy
 263among NILFS2 files can be depicted as follows::
 264
 265  Super block (SB)
 266       |
 267       v
 268  Super root block (the latest cno=xx)
 269       |-- DAT
 270       |-- sufile
 271       `-- cpfile
 272              |-- ifile (cno=c1)
 273              |-- ifile (cno=c2) ---- file (ino=i1)
 274              :        :          |-- file (ino=i2)
 275              `-- ifile (cno=xx)  |-- file (ino=i3)
 276                                  :        :
 277                                  `-- file (ino=yy)
 278                                    ( regular file, directory, or symlink )
 279
 280For detail on the format of each file, please see nilfs2_ondisk.h
 281located at include/uapi/linux directory.
 282
 283There are no patents or other intellectual property that we protect
 284with regard to the design of NILFS2.  It is allowed to replicate the
 285design in hopes that other operating systems could share (mount, read,
 286write, etc.) data stored in this format.
 287