2Miscellaneous Device control operations for the autofs4 kernel module
   5The problem
   8There is a problem with active restarts in autofs (that is to say
   9restarting autofs when there are busy mounts).
  11During normal operation autofs uses a file descriptor opened on the
  12directory that is being managed in order to be able to issue control
  13operations. Using a file descriptor gives ioctl operations access to
  14autofs specific information stored in the super block. The operations
  15are things such as setting an autofs mount catatonic, setting the
  16expire timeout and requesting expire checks. As is explained below,
  17certain types of autofs triggered mounts can end up covering an autofs
  18mount itself which prevents us being able to use open(2) to obtain a
  19file descriptor for these operations if we don't already have one open.
  21Currently autofs uses "umount -l" (lazy umount) to clear active mounts
  22at restart. While using lazy umount works for most cases, anything that
  23needs to walk back up the mount tree to construct a path, such as
  24getcwd(2) and the proc file system /proc/<pid>/cwd, no longer works
  25because the point from which the path is constructed has been detached
  26from the mount tree.
  28The actual problem with autofs is that it can't reconnect to existing
  29mounts. Immediately one thinks of just adding the ability to remount
  30autofs file systems would solve it, but alas, that can't work. This is
  31because autofs direct mounts and the implementation of "on demand mount
  32and expire" of nested mount trees have the file system mounted directly
  33on top of the mount trigger directory dentry.
  35For example, there are two types of automount maps, direct (in the kernel
  36module source you will see a third type called an offset, which is just
  37a direct mount in disguise) and indirect.
  39Here is a master map with direct and indirect map entries:
  41/-      /etc/
  42/test   /etc/auto.indirect
  44and the corresponding map files:
  48/automount/dparse/g6  budgie:/autofs/export1
  49/automount/dparse/g1  shark:/autofs/export1
  50and so on.
  54g1    shark:/autofs/export1
  55g6    budgie:/autofs/export1
  56and so on.
  58For the above indirect map an autofs file system is mounted on /test and
  59mounts are triggered for each sub-directory key by the inode lookup
  60operation. So we see a mount of shark:/autofs/export1 on /test/g1, for
  63The way that direct mounts are handled is by making an autofs mount on
  64each full path, such as /automount/dparse/g1, and using it as a mount
  65trigger. So when we walk on the path we mount shark:/autofs/export1 "on
  66top of this mount point". Since these are always directories we can
  67use the follow_link inode operation to trigger the mount.
  69But, each entry in direct and indirect maps can have offsets (making
  70them multi-mount map entries).
  72For example, an indirect mount map entry could also be:
  74g1  \
  75   /        shark:/autofs/export5/testing/test \
  76   /s1      shark:/autofs/export/testing/test/s1 \
  77   /s2      shark:/autofs/export5/testing/test/s2 \
  78   /s1/ss1  shark:/autofs/export1 \
  79   /s2/ss2  shark:/autofs/export2
  81and a similarly a direct mount map entry could also be:
  83/automount/dparse/g1 \
  84    /       shark:/autofs/export5/testing/test \
  85    /s1     shark:/autofs/export/testing/test/s1 \
  86    /s2     shark:/autofs/export5/testing/test/s2 \
  87    /s1/ss1 shark:/autofs/export2 \
  88    /s2/ss2 shark:/autofs/export2
  90One of the issues with version 4 of autofs was that, when mounting an
  91entry with a large number of offsets, possibly with nesting, we needed
  92to mount and umount all of the offsets as a single unit. Not really a
  93problem, except for people with a large number of offsets in map entries.
  94This mechanism is used for the well known "hosts" map and we have seen
  95cases (in 2.4) where the available number of mounts are exhausted or
  96where the number of privileged ports available is exhausted.
  98In version 5 we mount only as we go down the tree of offsets and
  99similarly for expiring them which resolves the above problem. There is
 100somewhat more detail to the implementation but it isn't needed for the
 101sake of the problem explanation. The one important detail is that these
 102offsets are implemented using the same mechanism as the direct mounts
 103above and so the mount points can be covered by a mount.
 105The current autofs implementation uses an ioctl file descriptor opened
 106on the mount point for control operations. The references held by the
 107descriptor are accounted for in checks made to determine if a mount is
 108in use and is also used to access autofs file system information held
 109in the mount super block. So the use of a file handle needs to be
 113The Solution
 116To be able to restart autofs leaving existing direct, indirect and
 117offset mounts in place we need to be able to obtain a file handle
 118for these potentially covered autofs mount points. Rather than just
 119implement an isolated operation it was decided to re-implement the
 120existing ioctl interface and add new operations to provide this
 123In addition, to be able to reconstruct a mount tree that has busy mounts,
 124the uid and gid of the last user that triggered the mount needs to be
 125available because these can be used as macro substitution variables in
 126autofs maps. They are recorded at mount request time and an operation
 127has been added to retrieve them.
 129Since we're re-implementing the control interface, a couple of other
 130problems with the existing interface have been addressed. First, when
 131a mount or expire operation completes a status is returned to the
 132kernel by either a "send ready" or a "send fail" operation. The
 133"send fail" operation of the ioctl interface could only ever send
 134ENOENT so the re-implementation allows user space to send an actual
 135status. Another expensive operation in user space, for those using
 136very large maps, is discovering if a mount is present. Usually this
 137involves scanning /proc/mounts and since it needs to be done quite
 138often it can introduce significant overhead when there are many entries
 139in the mount table. An operation to lookup the mount status of a mount
 140point dentry (covered or not) has also been added.
 142Current kernel development policy recommends avoiding the use of the
 143ioctl mechanism in favor of systems such as Netlink. An implementation
 144using this system was attempted to evaluate its suitability and it was
 145found to be inadequate, in this case. The Generic Netlink system was
 146used for this as raw Netlink would lead to a significant increase in
 147complexity. There's no question that the Generic Netlink system is an
 148elegant solution for common case ioctl functions but it's not a complete
 149replacement probably because its primary purpose in life is to be a
 150message bus implementation rather than specifically an ioctl replacement.
 151While it would be possible to work around this there is one concern
 152that lead to the decision to not use it. This is that the autofs
 153expire in the daemon has become far to complex because umount
 154candidates are enumerated, almost for no other reason than to "count"
 155the number of times to call the expire ioctl. This involves scanning
 156the mount table which has proved to be a big overhead for users with
 157large maps. The best way to improve this is try and get back to the
 158way the expire was done long ago. That is, when an expire request is
 159issued for a mount (file handle) we should continually call back to
 160the daemon until we can't umount any more mounts, then return the
 161appropriate status to the daemon. At the moment we just expire one
 162mount at a time. A Generic Netlink implementation would exclude this
 163possibility for future development due to the requirements of the
 164message bus architecture.
 167autofs4 Miscellaneous Device mount control interface
 170The control interface is opening a device node, typically /dev/autofs.
 172All the ioctls use a common structure to pass the needed parameter
 173information and return operation results:
 175struct autofs_dev_ioctl {
 176        __u32 ver_major;
 177        __u32 ver_minor;
 178        __u32 size;             /* total size of data passed in
 179                                 * including this struct */
 180        __s32 ioctlfd;          /* automount command fd */
 182        __u32 arg1;             /* Command parameters */
 183        __u32 arg2;
 185        char path[0];
 188The ioctlfd field is a mount point file descriptor of an autofs mount
 189point. It is returned by the open call and is used by all calls except
 190the check for whether a given path is a mount point, where it may
 191optionally be used to check a specific mount corresponding to a given
 192mount point file descriptor, and when requesting the uid and gid of the
 193last successful mount on a directory within the autofs file system.
 195The fields arg1 and arg2 are used to communicate parameters and results of
 196calls made as described below.
 198The path field is used to pass a path where it is needed and the size field
 199is used account for the increased structure length when translating the
 200structure sent from user space.
 202This structure can be initialized before setting specific fields by using
 203the void function call init_autofs_dev_ioctl(struct autofs_dev_ioctl *).
 205All of the ioctls perform a copy of this structure from user space to
 206kernel space and return -EINVAL if the size parameter is smaller than
 207the structure size itself, -ENOMEM if the kernel memory allocation fails
 208or -EFAULT if the copy itself fails. Other checks include a version check
 209of the compiled in user space version against the module version and a
 210mismatch results in a -EINVAL return. If the size field is greater than
 211the structure size then a path is assumed to be present and is checked to
 212ensure it begins with a "/" and is NULL terminated, otherwise -EINVAL is
 213returned. Following these checks, for all ioctl commands except
 215AUTOFS_DEV_IOCTL_CLOSEMOUNT_CMD the ioctlfd is validated and if it is
 216not a valid descriptor or doesn't correspond to an autofs mount point
 217an error of -EBADF, -ENOTTY or -EINVAL (not an autofs descriptor) is
 221The ioctls
 224An example of an implementation which uses this interface can be seen
 225in autofs version 5.0.4 and later in file lib/dev-ioctl-lib.c of the
 226distribution tar available for download from in directory
 229The device node ioctl operations implemented by this interface are:
 235Get the major and minor version of the autofs4 device ioctl kernel module
 236implementation. It requires an initialized struct autofs_dev_ioctl as an
 237input parameter and sets the version information in the passed in structure.
 238It returns 0 on success or the error -EINVAL if a version mismatch is
 245Get the major and minor version of the autofs4 protocol version understood
 246by loaded module. This call requires an initialized struct autofs_dev_ioctl
 247with the ioctlfd field set to a valid autofs mount point descriptor
 248and sets the requested version number in structure field arg1. These
 249commands return 0 on success or one of the negative error codes if
 250validation fails.
 256Obtain and release a file descriptor for an autofs managed mount point
 257path. The open call requires an initialized struct autofs_dev_ioctl with
 258the the path field set and the size field adjusted appropriately as well
 259as the arg1 field set to the device number of the autofs mount. The
 260device number can be obtained from the mount options shown in
 261/proc/mounts. The close call requires an initialized struct
 262autofs_dev_ioct with the ioctlfd field set to the descriptor obtained
 263from the open call. The release of the file descriptor can also be done
 264with close(2) so any open descriptors will also be closed at process exit.
 265The close call is included in the implemented operations largely for
 266completeness and to provide for a consistent user space implementation.
 272Return mount and expire result status from user space to the kernel.
 273Both of these calls require an initialized struct autofs_dev_ioctl
 274with the ioctlfd field set to the descriptor obtained from the open
 275call and the arg1 field set to the wait queue token number, received
 276by user space in the foregoing mount or expire request. The arg2 field
 277is set to the status to be returned. For the ready call this is always
 2780 and for the fail call it is set to the errno of the operation.
 284Set the pipe file descriptor used for kernel communication to the daemon.
 285Normally this is set at mount time using an option but when reconnecting
 286to a existing mount we need to use this to tell the autofs mount about
 287the new kernel pipe descriptor. In order to protect mounts against
 288incorrectly setting the pipe descriptor we also require that the autofs
 289mount be catatonic (see next call).
 291The call requires an initialized struct autofs_dev_ioctl with the
 292ioctlfd field set to the descriptor obtained from the open call and
 293the arg1 field set to descriptor of the pipe. On success the call
 294also sets the process group id used to identify the controlling process
 295(eg. the owning automount(8) daemon) to the process group of the caller.
 301Make the autofs mount point catatonic. The autofs mount will no longer
 302issue mount requests, the kernel communication pipe descriptor is released
 303and any remaining waits in the queue released.
 305The call requires an initialized struct autofs_dev_ioctl with the
 306ioctlfd field set to the descriptor obtained from the open call.
 312Set the expire timeout for mounts within an autofs mount point.
 314The call requires an initialized struct autofs_dev_ioctl with the
 315ioctlfd field set to the descriptor obtained from the open call.
 321Return the uid and gid of the last process to successfully trigger a the
 322mount on the given path dentry.
 324The call requires an initialized struct autofs_dev_ioctl with the path
 325field set to the mount point in question and the size field adjusted
 326appropriately as well as the arg1 field set to the device number of the
 327containing autofs mount. Upon return the struct field arg1 contains the
 328uid and arg2 the gid.
 330When reconstructing an autofs mount tree with active mounts we need to
 331re-connect to mounts that may have used the original process uid and
 332gid (or string variations of them) for mount lookups within the map entry.
 333This call provides the ability to obtain this uid and gid so they may be
 334used by user space for the mount map lookups.
 340Issue an expire request to the kernel for an autofs mount. Typically
 341this ioctl is called until no further expire candidates are found.
 343The call requires an initialized struct autofs_dev_ioctl with the
 344ioctlfd field set to the descriptor obtained from the open call. In
 345addition an immediate expire, independent of the mount timeout, can be
 346requested by setting the arg1 field to 1. If no expire candidates can
 347be found the ioctl returns -1 with errno set to EAGAIN.
 349This call causes the kernel module to check the mount corresponding
 350to the given ioctlfd for mounts that can be expired, issues an expire
 351request back to the daemon and waits for completion.
 356Checks if an autofs mount point is in use.
 358The call requires an initialized struct autofs_dev_ioctl with the
 359ioctlfd field set to the descriptor obtained from the open call and
 360it returns the result in the arg1 field, 1 for busy and 0 otherwise.
 366Check if the given path is a mountpoint.
 368The call requires an initialized struct autofs_dev_ioctl. There are two
 369possible variations. Both use the path field set to the path of the mount
 370point to check and the size field adjusted appropriately. One uses the
 371ioctlfd field to identify a specific mount point to check while the other
 372variation uses the path and optionally arg1 set to an autofs mount type.
 373The call returns 1 if this is a mount point and sets arg1 to the device
 374number of the mount and field arg2 to the relevant super block magic
 375number (described below) or 0 if it isn't a mountpoint. In both cases
 376the the device number (as returned by new_encode_dev()) is returned
 377in field arg1.
 379If supplied with a file descriptor we're looking for a specific mount,
 380not necessarily at the top of the mounted stack. In this case the path
 381the descriptor corresponds to is considered a mountpoint if it is itself
 382a mountpoint or contains a mount, such as a multi-mount without a root
 383mount. In this case we return 1 if the descriptor corresponds to a mount
 384point and and also returns the super magic of the covering mount if there
 385is one or 0 if it isn't a mountpoint.
 387If a path is supplied (and the ioctlfd field is set to -1) then the path
 388is looked up and is checked to see if it is the root of a mount. If a
 389type is also given we are looking for a particular autofs mount and if
 390a match isn't found a fail is returned. If the the located path is the
 391root of a mount 1 is returned along with the super magic of the mount
 392or 0 otherwise.
 394 kindly hosted by Redpill Linpro AS, provider of Linux consulting and operations services since 1995.