linux/Documentation/filesystems/fsverity.rst
<<
>>
Prefs
   1.. SPDX-License-Identifier: GPL-2.0
   2
   3.. _fsverity:
   4
   5=======================================================
   6fs-verity: read-only file-based authenticity protection
   7=======================================================
   8
   9Introduction
  10============
  11
  12fs-verity (``fs/verity/``) is a support layer that filesystems can
  13hook into to support transparent integrity and authenticity protection
  14of read-only files.  Currently, it is supported by the ext4 and f2fs
  15filesystems.  Like fscrypt, not too much filesystem-specific code is
  16needed to support fs-verity.
  17
  18fs-verity is similar to `dm-verity
  19<https://www.kernel.org/doc/Documentation/device-mapper/verity.txt>`_
  20but works on files rather than block devices.  On regular files on
  21filesystems supporting fs-verity, userspace can execute an ioctl that
  22causes the filesystem to build a Merkle tree for the file and persist
  23it to a filesystem-specific location associated with the file.
  24
  25After this, the file is made readonly, and all reads from the file are
  26automatically verified against the file's Merkle tree.  Reads of any
  27corrupted data, including mmap reads, will fail.
  28
  29Userspace can use another ioctl to retrieve the root hash (actually
  30the "fs-verity file digest", which is a hash that includes the Merkle
  31tree root hash) that fs-verity is enforcing for the file.  This ioctl
  32executes in constant time, regardless of the file size.
  33
  34fs-verity is essentially a way to hash a file in constant time,
  35subject to the caveat that reads which would violate the hash will
  36fail at runtime.
  37
  38Use cases
  39=========
  40
  41By itself, the base fs-verity feature only provides integrity
  42protection, i.e. detection of accidental (non-malicious) corruption.
  43
  44However, because fs-verity makes retrieving the file hash extremely
  45efficient, it's primarily meant to be used as a tool to support
  46authentication (detection of malicious modifications) or auditing
  47(logging file hashes before use).
  48
  49Trusted userspace code (e.g. operating system code running on a
  50read-only partition that is itself authenticated by dm-verity) can
  51authenticate the contents of an fs-verity file by using the
  52`FS_IOC_MEASURE_VERITY`_ ioctl to retrieve its hash, then verifying a
  53digital signature of it.
  54
  55A standard file hash could be used instead of fs-verity.  However,
  56this is inefficient if the file is large and only a small portion may
  57be accessed.  This is often the case for Android application package
  58(APK) files, for example.  These typically contain many translations,
  59classes, and other resources that are infrequently or even never
  60accessed on a particular device.  It would be slow and wasteful to
  61read and hash the entire file before starting the application.
  62
  63Unlike an ahead-of-time hash, fs-verity also re-verifies data each
  64time it's paged in.  This ensures that malicious disk firmware can't
  65undetectably change the contents of the file at runtime.
  66
  67fs-verity does not replace or obsolete dm-verity.  dm-verity should
  68still be used on read-only filesystems.  fs-verity is for files that
  69must live on a read-write filesystem because they are independently
  70updated and potentially user-installed, so dm-verity cannot be used.
  71
  72The base fs-verity feature is a hashing mechanism only; actually
  73authenticating the files is up to userspace.  However, to meet some
  74users' needs, fs-verity optionally supports a simple signature
  75verification mechanism where users can configure the kernel to require
  76that all fs-verity files be signed by a key loaded into a keyring; see
  77`Built-in signature verification`_.  Support for fs-verity file hashes
  78in IMA (Integrity Measurement Architecture) policies is also planned.
  79
  80User API
  81========
  82
  83FS_IOC_ENABLE_VERITY
  84--------------------
  85
  86The FS_IOC_ENABLE_VERITY ioctl enables fs-verity on a file.  It takes
  87in a pointer to a struct fsverity_enable_arg, defined as
  88follows::
  89
  90    struct fsverity_enable_arg {
  91            __u32 version;
  92            __u32 hash_algorithm;
  93            __u32 block_size;
  94            __u32 salt_size;
  95            __u64 salt_ptr;
  96            __u32 sig_size;
  97            __u32 __reserved1;
  98            __u64 sig_ptr;
  99            __u64 __reserved2[11];
 100    };
 101
 102This structure contains the parameters of the Merkle tree to build for
 103the file, and optionally contains a signature.  It must be initialized
 104as follows:
 105
 106- ``version`` must be 1.
 107- ``hash_algorithm`` must be the identifier for the hash algorithm to
 108  use for the Merkle tree, such as FS_VERITY_HASH_ALG_SHA256.  See
 109  ``include/uapi/linux/fsverity.h`` for the list of possible values.
 110- ``block_size`` must be the Merkle tree block size.  Currently, this
 111  must be equal to the system page size, which is usually 4096 bytes.
 112  Other sizes may be supported in the future.  This value is not
 113  necessarily the same as the filesystem block size.
 114- ``salt_size`` is the size of the salt in bytes, or 0 if no salt is
 115  provided.  The salt is a value that is prepended to every hashed
 116  block; it can be used to personalize the hashing for a particular
 117  file or device.  Currently the maximum salt size is 32 bytes.
 118- ``salt_ptr`` is the pointer to the salt, or NULL if no salt is
 119  provided.
 120- ``sig_size`` is the size of the signature in bytes, or 0 if no
 121  signature is provided.  Currently the signature is (somewhat
 122  arbitrarily) limited to 16128 bytes.  See `Built-in signature
 123  verification`_ for more information.
 124- ``sig_ptr``  is the pointer to the signature, or NULL if no
 125  signature is provided.
 126- All reserved fields must be zeroed.
 127
 128FS_IOC_ENABLE_VERITY causes the filesystem to build a Merkle tree for
 129the file and persist it to a filesystem-specific location associated
 130with the file, then mark the file as a verity file.  This ioctl may
 131take a long time to execute on large files, and it is interruptible by
 132fatal signals.
 133
 134FS_IOC_ENABLE_VERITY checks for write access to the inode.  However,
 135it must be executed on an O_RDONLY file descriptor and no processes
 136can have the file open for writing.  Attempts to open the file for
 137writing while this ioctl is executing will fail with ETXTBSY.  (This
 138is necessary to guarantee that no writable file descriptors will exist
 139after verity is enabled, and to guarantee that the file's contents are
 140stable while the Merkle tree is being built over it.)
 141
 142On success, FS_IOC_ENABLE_VERITY returns 0, and the file becomes a
 143verity file.  On failure (including the case of interruption by a
 144fatal signal), no changes are made to the file.
 145
 146FS_IOC_ENABLE_VERITY can fail with the following errors:
 147
 148- ``EACCES``: the process does not have write access to the file
 149- ``EBADMSG``: the signature is malformed
 150- ``EBUSY``: this ioctl is already running on the file
 151- ``EEXIST``: the file already has verity enabled
 152- ``EFAULT``: the caller provided inaccessible memory
 153- ``EINTR``: the operation was interrupted by a fatal signal
 154- ``EINVAL``: unsupported version, hash algorithm, or block size; or
 155  reserved bits are set; or the file descriptor refers to neither a
 156  regular file nor a directory.
 157- ``EISDIR``: the file descriptor refers to a directory
 158- ``EKEYREJECTED``: the signature doesn't match the file
 159- ``EMSGSIZE``: the salt or signature is too long
 160- ``ENOKEY``: the fs-verity keyring doesn't contain the certificate
 161  needed to verify the signature
 162- ``ENOPKG``: fs-verity recognizes the hash algorithm, but it's not
 163  available in the kernel's crypto API as currently configured (e.g.
 164  for SHA-512, missing CONFIG_CRYPTO_SHA512).
 165- ``ENOTTY``: this type of filesystem does not implement fs-verity
 166- ``EOPNOTSUPP``: the kernel was not configured with fs-verity
 167  support; or the filesystem superblock has not had the 'verity'
 168  feature enabled on it; or the filesystem does not support fs-verity
 169  on this file.  (See `Filesystem support`_.)
 170- ``EPERM``: the file is append-only; or, a signature is required and
 171  one was not provided.
 172- ``EROFS``: the filesystem is read-only
 173- ``ETXTBSY``: someone has the file open for writing.  This can be the
 174  caller's file descriptor, another open file descriptor, or the file
 175  reference held by a writable memory map.
 176
 177FS_IOC_MEASURE_VERITY
 178---------------------
 179
 180The FS_IOC_MEASURE_VERITY ioctl retrieves the digest of a verity file.
 181The fs-verity file digest is a cryptographic digest that identifies
 182the file contents that are being enforced on reads; it is computed via
 183a Merkle tree and is different from a traditional full-file digest.
 184
 185This ioctl takes in a pointer to a variable-length structure::
 186
 187    struct fsverity_digest {
 188            __u16 digest_algorithm;
 189            __u16 digest_size; /* input/output */
 190            __u8 digest[];
 191    };
 192
 193``digest_size`` is an input/output field.  On input, it must be
 194initialized to the number of bytes allocated for the variable-length
 195``digest`` field.
 196
 197On success, 0 is returned and the kernel fills in the structure as
 198follows:
 199
 200- ``digest_algorithm`` will be the hash algorithm used for the file
 201  digest.  It will match ``fsverity_enable_arg::hash_algorithm``.
 202- ``digest_size`` will be the size of the digest in bytes, e.g. 32
 203  for SHA-256.  (This can be redundant with ``digest_algorithm``.)
 204- ``digest`` will be the actual bytes of the digest.
 205
 206FS_IOC_MEASURE_VERITY is guaranteed to execute in constant time,
 207regardless of the size of the file.
 208
 209FS_IOC_MEASURE_VERITY can fail with the following errors:
 210
 211- ``EFAULT``: the caller provided inaccessible memory
 212- ``ENODATA``: the file is not a verity file
 213- ``ENOTTY``: this type of filesystem does not implement fs-verity
 214- ``EOPNOTSUPP``: the kernel was not configured with fs-verity
 215  support, or the filesystem superblock has not had the 'verity'
 216  feature enabled on it.  (See `Filesystem support`_.)
 217- ``EOVERFLOW``: the digest is longer than the specified
 218  ``digest_size`` bytes.  Try providing a larger buffer.
 219
 220FS_IOC_READ_VERITY_METADATA
 221---------------------------
 222
 223The FS_IOC_READ_VERITY_METADATA ioctl reads verity metadata from a
 224verity file.  This ioctl is available since Linux v5.12.
 225
 226This ioctl allows writing a server program that takes a verity file
 227and serves it to a client program, such that the client can do its own
 228fs-verity compatible verification of the file.  This only makes sense
 229if the client doesn't trust the server and if the server needs to
 230provide the storage for the client.
 231
 232This is a fairly specialized use case, and most fs-verity users won't
 233need this ioctl.
 234
 235This ioctl takes in a pointer to the following structure::
 236
 237   #define FS_VERITY_METADATA_TYPE_MERKLE_TREE     1
 238   #define FS_VERITY_METADATA_TYPE_DESCRIPTOR      2
 239   #define FS_VERITY_METADATA_TYPE_SIGNATURE       3
 240
 241   struct fsverity_read_metadata_arg {
 242           __u64 metadata_type;
 243           __u64 offset;
 244           __u64 length;
 245           __u64 buf_ptr;
 246           __u64 __reserved;
 247   };
 248
 249``metadata_type`` specifies the type of metadata to read:
 250
 251- ``FS_VERITY_METADATA_TYPE_MERKLE_TREE`` reads the blocks of the
 252  Merkle tree.  The blocks are returned in order from the root level
 253  to the leaf level.  Within each level, the blocks are returned in
 254  the same order that their hashes are themselves hashed.
 255  See `Merkle tree`_ for more information.
 256
 257- ``FS_VERITY_METADATA_TYPE_DESCRIPTOR`` reads the fs-verity
 258  descriptor.  See `fs-verity descriptor`_.
 259
 260- ``FS_VERITY_METADATA_TYPE_SIGNATURE`` reads the signature which was
 261  passed to FS_IOC_ENABLE_VERITY, if any.  See `Built-in signature
 262  verification`_.
 263
 264The semantics are similar to those of ``pread()``.  ``offset``
 265specifies the offset in bytes into the metadata item to read from, and
 266``length`` specifies the maximum number of bytes to read from the
 267metadata item.  ``buf_ptr`` is the pointer to the buffer to read into,
 268cast to a 64-bit integer.  ``__reserved`` must be 0.  On success, the
 269number of bytes read is returned.  0 is returned at the end of the
 270metadata item.  The returned length may be less than ``length``, for
 271example if the ioctl is interrupted.
 272
 273The metadata returned by FS_IOC_READ_VERITY_METADATA isn't guaranteed
 274to be authenticated against the file digest that would be returned by
 275`FS_IOC_MEASURE_VERITY`_, as the metadata is expected to be used to
 276implement fs-verity compatible verification anyway (though absent a
 277malicious disk, the metadata will indeed match).  E.g. to implement
 278this ioctl, the filesystem is allowed to just read the Merkle tree
 279blocks from disk without actually verifying the path to the root node.
 280
 281FS_IOC_READ_VERITY_METADATA can fail with the following errors:
 282
 283- ``EFAULT``: the caller provided inaccessible memory
 284- ``EINTR``: the ioctl was interrupted before any data was read
 285- ``EINVAL``: reserved fields were set, or ``offset + length``
 286  overflowed
 287- ``ENODATA``: the file is not a verity file, or
 288  FS_VERITY_METADATA_TYPE_SIGNATURE was requested but the file doesn't
 289  have a built-in signature
 290- ``ENOTTY``: this type of filesystem does not implement fs-verity, or
 291  this ioctl is not yet implemented on it
 292- ``EOPNOTSUPP``: the kernel was not configured with fs-verity
 293  support, or the filesystem superblock has not had the 'verity'
 294  feature enabled on it.  (See `Filesystem support`_.)
 295
 296FS_IOC_GETFLAGS
 297---------------
 298
 299The existing ioctl FS_IOC_GETFLAGS (which isn't specific to fs-verity)
 300can also be used to check whether a file has fs-verity enabled or not.
 301To do so, check for FS_VERITY_FL (0x00100000) in the returned flags.
 302
 303The verity flag is not settable via FS_IOC_SETFLAGS.  You must use
 304FS_IOC_ENABLE_VERITY instead, since parameters must be provided.
 305
 306statx
 307-----
 308
 309Since Linux v5.5, the statx() system call sets STATX_ATTR_VERITY if
 310the file has fs-verity enabled.  This can perform better than
 311FS_IOC_GETFLAGS and FS_IOC_MEASURE_VERITY because it doesn't require
 312opening the file, and opening verity files can be expensive.
 313
 314Accessing verity files
 315======================
 316
 317Applications can transparently access a verity file just like a
 318non-verity one, with the following exceptions:
 319
 320- Verity files are readonly.  They cannot be opened for writing or
 321  truncate()d, even if the file mode bits allow it.  Attempts to do
 322  one of these things will fail with EPERM.  However, changes to
 323  metadata such as owner, mode, timestamps, and xattrs are still
 324  allowed, since these are not measured by fs-verity.  Verity files
 325  can also still be renamed, deleted, and linked to.
 326
 327- Direct I/O is not supported on verity files.  Attempts to use direct
 328  I/O on such files will fall back to buffered I/O.
 329
 330- DAX (Direct Access) is not supported on verity files, because this
 331  would circumvent the data verification.
 332
 333- Reads of data that doesn't match the verity Merkle tree will fail
 334  with EIO (for read()) or SIGBUS (for mmap() reads).
 335
 336- If the sysctl "fs.verity.require_signatures" is set to 1 and the
 337  file is not signed by a key in the fs-verity keyring, then opening
 338  the file will fail.  See `Built-in signature verification`_.
 339
 340Direct access to the Merkle tree is not supported.  Therefore, if a
 341verity file is copied, or is backed up and restored, then it will lose
 342its "verity"-ness.  fs-verity is primarily meant for files like
 343executables that are managed by a package manager.
 344
 345File digest computation
 346=======================
 347
 348This section describes how fs-verity hashes the file contents using a
 349Merkle tree to produce the digest which cryptographically identifies
 350the file contents.  This algorithm is the same for all filesystems
 351that support fs-verity.
 352
 353Userspace only needs to be aware of this algorithm if it needs to
 354compute fs-verity file digests itself, e.g. in order to sign files.
 355
 356.. _fsverity_merkle_tree:
 357
 358Merkle tree
 359-----------
 360
 361The file contents is divided into blocks, where the block size is
 362configurable but is usually 4096 bytes.  The end of the last block is
 363zero-padded if needed.  Each block is then hashed, producing the first
 364level of hashes.  Then, the hashes in this first level are grouped
 365into 'blocksize'-byte blocks (zero-padding the ends as needed) and
 366these blocks are hashed, producing the second level of hashes.  This
 367proceeds up the tree until only a single block remains.  The hash of
 368this block is the "Merkle tree root hash".
 369
 370If the file fits in one block and is nonempty, then the "Merkle tree
 371root hash" is simply the hash of the single data block.  If the file
 372is empty, then the "Merkle tree root hash" is all zeroes.
 373
 374The "blocks" here are not necessarily the same as "filesystem blocks".
 375
 376If a salt was specified, then it's zero-padded to the closest multiple
 377of the input size of the hash algorithm's compression function, e.g.
 37864 bytes for SHA-256 or 128 bytes for SHA-512.  The padded salt is
 379prepended to every data or Merkle tree block that is hashed.
 380
 381The purpose of the block padding is to cause every hash to be taken
 382over the same amount of data, which simplifies the implementation and
 383keeps open more possibilities for hardware acceleration.  The purpose
 384of the salt padding is to make the salting "free" when the salted hash
 385state is precomputed, then imported for each hash.
 386
 387Example: in the recommended configuration of SHA-256 and 4K blocks,
 388128 hash values fit in each block.  Thus, each level of the Merkle
 389tree is approximately 128 times smaller than the previous, and for
 390large files the Merkle tree's size converges to approximately 1/127 of
 391the original file size.  However, for small files, the padding is
 392significant, making the space overhead proportionally more.
 393
 394.. _fsverity_descriptor:
 395
 396fs-verity descriptor
 397--------------------
 398
 399By itself, the Merkle tree root hash is ambiguous.  For example, it
 400can't a distinguish a large file from a small second file whose data
 401is exactly the top-level hash block of the first file.  Ambiguities
 402also arise from the convention of padding to the next block boundary.
 403
 404To solve this problem, the fs-verity file digest is actually computed
 405as a hash of the following structure, which contains the Merkle tree
 406root hash as well as other fields such as the file size::
 407
 408    struct fsverity_descriptor {
 409            __u8 version;           /* must be 1 */
 410            __u8 hash_algorithm;    /* Merkle tree hash algorithm */
 411            __u8 log_blocksize;     /* log2 of size of data and tree blocks */
 412            __u8 salt_size;         /* size of salt in bytes; 0 if none */
 413            __le32 __reserved_0x04; /* must be 0 */
 414            __le64 data_size;       /* size of file the Merkle tree is built over */
 415            __u8 root_hash[64];     /* Merkle tree root hash */
 416            __u8 salt[32];          /* salt prepended to each hashed block */
 417            __u8 __reserved[144];   /* must be 0's */
 418    };
 419
 420Built-in signature verification
 421===============================
 422
 423With CONFIG_FS_VERITY_BUILTIN_SIGNATURES=y, fs-verity supports putting
 424a portion of an authentication policy (see `Use cases`_) in the
 425kernel.  Specifically, it adds support for:
 426
 4271. At fs-verity module initialization time, a keyring ".fs-verity" is
 428   created.  The root user can add trusted X.509 certificates to this
 429   keyring using the add_key() system call, then (when done)
 430   optionally use keyctl_restrict_keyring() to prevent additional
 431   certificates from being added.
 432
 4332. `FS_IOC_ENABLE_VERITY`_ accepts a pointer to a PKCS#7 formatted
 434   detached signature in DER format of the file's fs-verity digest.
 435   On success, this signature is persisted alongside the Merkle tree.
 436   Then, any time the file is opened, the kernel will verify the
 437   file's actual digest against this signature, using the certificates
 438   in the ".fs-verity" keyring.
 439
 4403. A new sysctl "fs.verity.require_signatures" is made available.
 441   When set to 1, the kernel requires that all verity files have a
 442   correctly signed digest as described in (2).
 443
 444fs-verity file digests must be signed in the following format, which
 445is similar to the structure used by `FS_IOC_MEASURE_VERITY`_::
 446
 447    struct fsverity_formatted_digest {
 448            char magic[8];                  /* must be "FSVerity" */
 449            __le16 digest_algorithm;
 450            __le16 digest_size;
 451            __u8 digest[];
 452    };
 453
 454fs-verity's built-in signature verification support is meant as a
 455relatively simple mechanism that can be used to provide some level of
 456authenticity protection for verity files, as an alternative to doing
 457the signature verification in userspace or using IMA-appraisal.
 458However, with this mechanism, userspace programs still need to check
 459that the verity bit is set, and there is no protection against verity
 460files being swapped around.
 461
 462Filesystem support
 463==================
 464
 465fs-verity is currently supported by the ext4 and f2fs filesystems.
 466The CONFIG_FS_VERITY kconfig option must be enabled to use fs-verity
 467on either filesystem.
 468
 469``include/linux/fsverity.h`` declares the interface between the
 470``fs/verity/`` support layer and filesystems.  Briefly, filesystems
 471must provide an ``fsverity_operations`` structure that provides
 472methods to read and write the verity metadata to a filesystem-specific
 473location, including the Merkle tree blocks and
 474``fsverity_descriptor``.  Filesystems must also call functions in
 475``fs/verity/`` at certain times, such as when a file is opened or when
 476pages have been read into the pagecache.  (See `Verifying data`_.)
 477
 478ext4
 479----
 480
 481ext4 supports fs-verity since Linux v5.4 and e2fsprogs v1.45.2.
 482
 483To create verity files on an ext4 filesystem, the filesystem must have
 484been formatted with ``-O verity`` or had ``tune2fs -O verity`` run on
 485it.  "verity" is an RO_COMPAT filesystem feature, so once set, old
 486kernels will only be able to mount the filesystem readonly, and old
 487versions of e2fsck will be unable to check the filesystem.  Moreover,
 488currently ext4 only supports mounting a filesystem with the "verity"
 489feature when its block size is equal to PAGE_SIZE (often 4096 bytes).
 490
 491ext4 sets the EXT4_VERITY_FL on-disk inode flag on verity files.  It
 492can only be set by `FS_IOC_ENABLE_VERITY`_, and it cannot be cleared.
 493
 494ext4 also supports encryption, which can be used simultaneously with
 495fs-verity.  In this case, the plaintext data is verified rather than
 496the ciphertext.  This is necessary in order to make the fs-verity file
 497digest meaningful, since every file is encrypted differently.
 498
 499ext4 stores the verity metadata (Merkle tree and fsverity_descriptor)
 500past the end of the file, starting at the first 64K boundary beyond
 501i_size.  This approach works because (a) verity files are readonly,
 502and (b) pages fully beyond i_size aren't visible to userspace but can
 503be read/written internally by ext4 with only some relatively small
 504changes to ext4.  This approach avoids having to depend on the
 505EA_INODE feature and on rearchitecturing ext4's xattr support to
 506support paging multi-gigabyte xattrs into memory, and to support
 507encrypting xattrs.  Note that the verity metadata *must* be encrypted
 508when the file is, since it contains hashes of the plaintext data.
 509
 510Currently, ext4 verity only supports the case where the Merkle tree
 511block size, filesystem block size, and page size are all the same.  It
 512also only supports extent-based files.
 513
 514f2fs
 515----
 516
 517f2fs supports fs-verity since Linux v5.4 and f2fs-tools v1.11.0.
 518
 519To create verity files on an f2fs filesystem, the filesystem must have
 520been formatted with ``-O verity``.
 521
 522f2fs sets the FADVISE_VERITY_BIT on-disk inode flag on verity files.
 523It can only be set by `FS_IOC_ENABLE_VERITY`_, and it cannot be
 524cleared.
 525
 526Like ext4, f2fs stores the verity metadata (Merkle tree and
 527fsverity_descriptor) past the end of the file, starting at the first
 52864K boundary beyond i_size.  See explanation for ext4 above.
 529Moreover, f2fs supports at most 4096 bytes of xattr entries per inode
 530which wouldn't be enough for even a single Merkle tree block.
 531
 532Currently, f2fs verity only supports a Merkle tree block size of 4096.
 533Also, f2fs doesn't support enabling verity on files that currently
 534have atomic or volatile writes pending.
 535
 536Implementation details
 537======================
 538
 539Verifying data
 540--------------
 541
 542fs-verity ensures that all reads of a verity file's data are verified,
 543regardless of which syscall is used to do the read (e.g. mmap(),
 544read(), pread()) and regardless of whether it's the first read or a
 545later read (unless the later read can return cached data that was
 546already verified).  Below, we describe how filesystems implement this.
 547
 548Pagecache
 549~~~~~~~~~
 550
 551For filesystems using Linux's pagecache, the ``->readpage()`` and
 552``->readpages()`` methods must be modified to verify pages before they
 553are marked Uptodate.  Merely hooking ``->read_iter()`` would be
 554insufficient, since ``->read_iter()`` is not used for memory maps.
 555
 556Therefore, fs/verity/ provides a function fsverity_verify_page() which
 557verifies a page that has been read into the pagecache of a verity
 558inode, but is still locked and not Uptodate, so it's not yet readable
 559by userspace.  As needed to do the verification,
 560fsverity_verify_page() will call back into the filesystem to read
 561Merkle tree pages via fsverity_operations::read_merkle_tree_page().
 562
 563fsverity_verify_page() returns false if verification failed; in this
 564case, the filesystem must not set the page Uptodate.  Following this,
 565as per the usual Linux pagecache behavior, attempts by userspace to
 566read() from the part of the file containing the page will fail with
 567EIO, and accesses to the page within a memory map will raise SIGBUS.
 568
 569fsverity_verify_page() currently only supports the case where the
 570Merkle tree block size is equal to PAGE_SIZE (often 4096 bytes).
 571
 572In principle, fsverity_verify_page() verifies the entire path in the
 573Merkle tree from the data page to the root hash.  However, for
 574efficiency the filesystem may cache the hash pages.  Therefore,
 575fsverity_verify_page() only ascends the tree reading hash pages until
 576an already-verified hash page is seen, as indicated by the PageChecked
 577bit being set.  It then verifies the path to that page.
 578
 579This optimization, which is also used by dm-verity, results in
 580excellent sequential read performance.  This is because usually (e.g.
 581127 in 128 times for 4K blocks and SHA-256) the hash page from the
 582bottom level of the tree will already be cached and checked from
 583reading a previous data page.  However, random reads perform worse.
 584
 585Block device based filesystems
 586~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 587
 588Block device based filesystems (e.g. ext4 and f2fs) in Linux also use
 589the pagecache, so the above subsection applies too.  However, they
 590also usually read many pages from a file at once, grouped into a
 591structure called a "bio".  To make it easier for these types of
 592filesystems to support fs-verity, fs/verity/ also provides a function
 593fsverity_verify_bio() which verifies all pages in a bio.
 594
 595ext4 and f2fs also support encryption.  If a verity file is also
 596encrypted, the pages must be decrypted before being verified.  To
 597support this, these filesystems allocate a "post-read context" for
 598each bio and store it in ``->bi_private``::
 599
 600    struct bio_post_read_ctx {
 601           struct bio *bio;
 602           struct work_struct work;
 603           unsigned int cur_step;
 604           unsigned int enabled_steps;
 605    };
 606
 607``enabled_steps`` is a bitmask that specifies whether decryption,
 608verity, or both is enabled.  After the bio completes, for each needed
 609postprocessing step the filesystem enqueues the bio_post_read_ctx on a
 610workqueue, and then the workqueue work does the decryption or
 611verification.  Finally, pages where no decryption or verity error
 612occurred are marked Uptodate, and the pages are unlocked.
 613
 614Files on ext4 and f2fs may contain holes.  Normally, ``->readpages()``
 615simply zeroes holes and sets the corresponding pages Uptodate; no bios
 616are issued.  To prevent this case from bypassing fs-verity, these
 617filesystems use fsverity_verify_page() to verify hole pages.
 618
 619ext4 and f2fs disable direct I/O on verity files, since otherwise
 620direct I/O would bypass fs-verity.  (They also do the same for
 621encrypted files.)
 622
 623Userspace utility
 624=================
 625
 626This document focuses on the kernel, but a userspace utility for
 627fs-verity can be found at:
 628
 629        https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/fsverity-utils.git
 630
 631See the README.md file in the fsverity-utils source tree for details,
 632including examples of setting up fs-verity protected files.
 633
 634Tests
 635=====
 636
 637To test fs-verity, use xfstests.  For example, using `kvm-xfstests
 638<https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md>`_::
 639
 640    kvm-xfstests -c ext4,f2fs -g verity
 641
 642FAQ
 643===
 644
 645This section answers frequently asked questions about fs-verity that
 646weren't already directly answered in other parts of this document.
 647
 648:Q: Why isn't fs-verity part of IMA?
 649:A: fs-verity and IMA (Integrity Measurement Architecture) have
 650    different focuses.  fs-verity is a filesystem-level mechanism for
 651    hashing individual files using a Merkle tree.  In contrast, IMA
 652    specifies a system-wide policy that specifies which files are
 653    hashed and what to do with those hashes, such as log them,
 654    authenticate them, or add them to a measurement list.
 655
 656    IMA is planned to support the fs-verity hashing mechanism as an
 657    alternative to doing full file hashes, for people who want the
 658    performance and security benefits of the Merkle tree based hash.
 659    But it doesn't make sense to force all uses of fs-verity to be
 660    through IMA.  As a standalone filesystem feature, fs-verity
 661    already meets many users' needs, and it's testable like other
 662    filesystem features e.g. with xfstests.
 663
 664:Q: Isn't fs-verity useless because the attacker can just modify the
 665    hashes in the Merkle tree, which is stored on-disk?
 666:A: To verify the authenticity of an fs-verity file you must verify
 667    the authenticity of the "fs-verity file digest", which
 668    incorporates the root hash of the Merkle tree.  See `Use cases`_.
 669
 670:Q: Isn't fs-verity useless because the attacker can just replace a
 671    verity file with a non-verity one?
 672:A: See `Use cases`_.  In the initial use case, it's really trusted
 673    userspace code that authenticates the files; fs-verity is just a
 674    tool to do this job efficiently and securely.  The trusted
 675    userspace code will consider non-verity files to be inauthentic.
 676
 677:Q: Why does the Merkle tree need to be stored on-disk?  Couldn't you
 678    store just the root hash?
 679:A: If the Merkle tree wasn't stored on-disk, then you'd have to
 680    compute the entire tree when the file is first accessed, even if
 681    just one byte is being read.  This is a fundamental consequence of
 682    how Merkle tree hashing works.  To verify a leaf node, you need to
 683    verify the whole path to the root hash, including the root node
 684    (the thing which the root hash is a hash of).  But if the root
 685    node isn't stored on-disk, you have to compute it by hashing its
 686    children, and so on until you've actually hashed the entire file.
 687
 688    That defeats most of the point of doing a Merkle tree-based hash,
 689    since if you have to hash the whole file ahead of time anyway,
 690    then you could simply do sha256(file) instead.  That would be much
 691    simpler, and a bit faster too.
 692
 693    It's true that an in-memory Merkle tree could still provide the
 694    advantage of verification on every read rather than just on the
 695    first read.  However, it would be inefficient because every time a
 696    hash page gets evicted (you can't pin the entire Merkle tree into
 697    memory, since it may be very large), in order to restore it you
 698    again need to hash everything below it in the tree.  This again
 699    defeats most of the point of doing a Merkle tree-based hash, since
 700    a single block read could trigger re-hashing gigabytes of data.
 701
 702:Q: But couldn't you store just the leaf nodes and compute the rest?
 703:A: See previous answer; this really just moves up one level, since
 704    one could alternatively interpret the data blocks as being the
 705    leaf nodes of the Merkle tree.  It's true that the tree can be
 706    computed much faster if the leaf level is stored rather than just
 707    the data, but that's only because each level is less than 1% the
 708    size of the level below (assuming the recommended settings of
 709    SHA-256 and 4K blocks).  For the exact same reason, by storing
 710    "just the leaf nodes" you'd already be storing over 99% of the
 711    tree, so you might as well simply store the whole tree.
 712
 713:Q: Can the Merkle tree be built ahead of time, e.g. distributed as
 714    part of a package that is installed to many computers?
 715:A: This isn't currently supported.  It was part of the original
 716    design, but was removed to simplify the kernel UAPI and because it
 717    wasn't a critical use case.  Files are usually installed once and
 718    used many times, and cryptographic hashing is somewhat fast on
 719    most modern processors.
 720
 721:Q: Why doesn't fs-verity support writes?
 722:A: Write support would be very difficult and would require a
 723    completely different design, so it's well outside the scope of
 724    fs-verity.  Write support would require:
 725
 726    - A way to maintain consistency between the data and hashes,
 727      including all levels of hashes, since corruption after a crash
 728      (especially of potentially the entire file!) is unacceptable.
 729      The main options for solving this are data journalling,
 730      copy-on-write, and log-structured volume.  But it's very hard to
 731      retrofit existing filesystems with new consistency mechanisms.
 732      Data journalling is available on ext4, but is very slow.
 733
 734    - Rebuilding the Merkle tree after every write, which would be
 735      extremely inefficient.  Alternatively, a different authenticated
 736      dictionary structure such as an "authenticated skiplist" could
 737      be used.  However, this would be far more complex.
 738
 739    Compare it to dm-verity vs. dm-integrity.  dm-verity is very
 740    simple: the kernel just verifies read-only data against a
 741    read-only Merkle tree.  In contrast, dm-integrity supports writes
 742    but is slow, is much more complex, and doesn't actually support
 743    full-device authentication since it authenticates each sector
 744    independently, i.e. there is no "root hash".  It doesn't really
 745    make sense for the same device-mapper target to support these two
 746    very different cases; the same applies to fs-verity.
 747
 748:Q: Since verity files are immutable, why isn't the immutable bit set?
 749:A: The existing "immutable" bit (FS_IMMUTABLE_FL) already has a
 750    specific set of semantics which not only make the file contents
 751    read-only, but also prevent the file from being deleted, renamed,
 752    linked to, or having its owner or mode changed.  These extra
 753    properties are unwanted for fs-verity, so reusing the immutable
 754    bit isn't appropriate.
 755
 756:Q: Why does the API use ioctls instead of setxattr() and getxattr()?
 757:A: Abusing the xattr interface for basically arbitrary syscalls is
 758    heavily frowned upon by most of the Linux filesystem developers.
 759    An xattr should really just be an xattr on-disk, not an API to
 760    e.g. magically trigger construction of a Merkle tree.
 761
 762:Q: Does fs-verity support remote filesystems?
 763:A: Only ext4 and f2fs support is implemented currently, but in
 764    principle any filesystem that can store per-file verity metadata
 765    can support fs-verity, regardless of whether it's local or remote.
 766    Some filesystems may have fewer options of where to store the
 767    verity metadata; one possibility is to store it past the end of
 768    the file and "hide" it from userspace by manipulating i_size.  The
 769    data verification functions provided by ``fs/verity/`` also assume
 770    that the filesystem uses the Linux pagecache, but both local and
 771    remote filesystems normally do so.
 772
 773:Q: Why is anything filesystem-specific at all?  Shouldn't fs-verity
 774    be implemented entirely at the VFS level?
 775:A: There are many reasons why this is not possible or would be very
 776    difficult, including the following:
 777
 778    - To prevent bypassing verification, pages must not be marked
 779      Uptodate until they've been verified.  Currently, each
 780      filesystem is responsible for marking pages Uptodate via
 781      ``->readpages()``.  Therefore, currently it's not possible for
 782      the VFS to do the verification on its own.  Changing this would
 783      require significant changes to the VFS and all filesystems.
 784
 785    - It would require defining a filesystem-independent way to store
 786      the verity metadata.  Extended attributes don't work for this
 787      because (a) the Merkle tree may be gigabytes, but many
 788      filesystems assume that all xattrs fit into a single 4K
 789      filesystem block, and (b) ext4 and f2fs encryption doesn't
 790      encrypt xattrs, yet the Merkle tree *must* be encrypted when the
 791      file contents are, because it stores hashes of the plaintext
 792      file contents.
 793
 794      So the verity metadata would have to be stored in an actual
 795      file.  Using a separate file would be very ugly, since the
 796      metadata is fundamentally part of the file to be protected, and
 797      it could cause problems where users could delete the real file
 798      but not the metadata file or vice versa.  On the other hand,
 799      having it be in the same file would break applications unless
 800      filesystems' notion of i_size were divorced from the VFS's,
 801      which would be complex and require changes to all filesystems.
 802
 803    - It's desirable that FS_IOC_ENABLE_VERITY uses the filesystem's
 804      transaction mechanism so that either the file ends up with
 805      verity enabled, or no changes were made.  Allowing intermediate
 806      states to occur after a crash may cause problems.
 807