/run

Introduce a new top-level directory, /run, and transition packages to use it.

+++ WARNING: Incomplete DRAFT. Please, do *not* migrate any packages to use /run until initscripts (>= 2.88dsf-13.3) is uploaded. +++

+++ 17-Apr-2011: sysvinit (2.88dsf-13.4) entered experimental +++

+++ 13-May-2011: initramfs-tools (0.99) entered unstable +++

Overview

/run is a new cross-distribution location for the storage of transient writable files needed from system startup to shutdown, but which do not require preserving across reboots. This location is a replacement for several existing locations in the Filesystem Hierarchy Standard.

Standardised locations:

Non-standard locations:

Note that files placed as dotfiles in /dev and /dev/shm, which have clearly-defined purposes in the FHS, are in contravention of the FHS; /run provides a standard location for them in the hierarchy. It also provides a location for non-standard distribution-specific solutions such as Debian's /lib/init/rw. /run will be a cross-distribution location and is now also used by Fedora and SuSE and others will also be using it. It has also been proposed to the FHS for standardisation.

This has already been adopted by Fedora, and it's something we've wanted in Debian for years. Debian settled on /lib/init/rw as a writable location in early boot, but /lib is not really an appropriate place, though it was FHS compliant by not introducing a top-level directory. /run is wider in scope than /lib/init/rw, and will be used for transient writable data present from system startup to shutdown, which is not preserved across reboots, and will therefore supersede /var/run, /var/lock, /lib/init/rw and existing use of /dev and /etc as a place to put writable program state.

Once /run is set up by initscripts, /var/run and /var/lock are replaced with symlinks, when possible, to complete the transition. See the "Implementation" section, below for further details.

Note that a bigger issue that /run goes a long way to solving is having a completely read-only root filesystem. Current approaches involve writable aufs/unionfs overlays, which work around the problem, but do not solve it. In addition to migrating /var/run and /var/lock, the Debian initscripts implementation also migrates /dev/shm and (optionally, if the user makes it a symlink to /run/tmp) /tmp.

Policy

An update to debian-policy has been proposed (620870). This will be included in Policy once /run is actually present in Debian unstable. The proposed text is included below:

§9.1. File system hierarchy

§9.1.1. File System Structure

7. The following directories in the root filesystem are additionally allowed: '/run', '/sys' and '/selinux'. [2]

[2] The '/run' directory is a replacement for '/var/run', and its subdirectory '/run/lock' is a replacement for '/var/lock'. These changes have been adopted by most distributions and have been proposed for inclusion in a future revision of the FHS. Files and directories residing in '/run' should be stored on a temporary filesystem, the purpose of which is storage of ephemeral system state which should not be persistent across a reboot. The '/sys' and '/selinux' directories are used as mount points to mount virtual filesystems to get access to kernel information.

§9.3. System run levels and 'init.d' scripts

§9.3.2. Writing the scripts

'/var/run' and '/var/lock' should be symlinks to '/run' and '/run/lock', respectively. This arrangement may also be satisfied through equivalent means, for example bind or nullfs mounts. Files and directories residing in '/run' should be stored on a temporary filesystem and not be persistent across a reboot, and hence the presence of files or directories in any of these directories is not guaranteed and 'init.d' scripts must handle this correctly. This will typically amount to creating any required subdirectories dynamically when the 'init.d' script is run, rather than including them in the package and relying on 'dpkg' to create them.

Annotation to §9.3.2.: Bind mount (Debian GNU/Linux) and nullfs mount (Debian GNU/kFreeBSD).

Implementation

Support for /run is being implemented in initscripts (620191). In order to ensure that /run is both present and functional following package installation and configuration, the setup is done in two stages:

Stage #1: Initial package install

This ensures that the /run hierarchy is present, but using the actual contents of /var/run and /var/lock.

Stage #2: After system reboot

At this point the migration is complete, with /run being a tmpfs, and the old locations /var/run, /var/lock and /dev/shm being converted to symlinks. Note that the fallback to bind mounts will only occur if the user has deliberately mounted something on /var/run, /var/lock or /dev/shm in /etc/fstab, and this will need cleaning up by the sysadmin in order to transition to symlinks at the next reboot (or they can clean up the mounts and create the symlinks by hand). If bind mounts are required, warning messages directing the user to remove the offending entries from /etc/fstab are displayed.

Configuration

The tmpfs filesystems to mount are controlled by settings in /etc/default/rcS:

For each of these mounts, the size and access mode may be controlled via settings in /etc/default/tmpfs. For example, the size and mode of /run/lock are set by the LOCK_SIZE and LOCK_MODE variables. If RAMLOCK=no, then /run/lock will use the underlying /run tmpfs, rather than a separate tmpfs mount.

Discussion is currently in progress to decide the most appropriate tmpfs mounts to use by default, and their optimal default sizes.

Other init systems

upstart

The upstart package in Debian still uses the compat layer (/etc/rcS.d) to initialize the system during early boot, i.e. for mount and fsck it relies on initscripts and as a consequence there are no special steps to be taken to support /run.

In Ubuntu, mount and fsck is handled by "mountall". Support for /run needs to be added to that package.

systemd

The current version of systemd in unstable (20-1) has no support for /run yet. Newer upstream releases (v21+) have native support for /run, i.e. systemd ships a mount unit to setup a tmpfs for /run and two units to setup a bind mount for /var/lock and /var/run. These two unit files var-run.mount and var-lock.mount use ?ConditionPathIsDirectory=/var/run which means, if /var/lock and /var/run are symlinks, the bind mounts are not established.

The initscripts package will not convert /var/run and /var/lock to symlinks in its postinst, but uses /lib/init/mount-functions.sh to convert them to symlinks on the next reboot. As systemd does not use the mount facilities provided by sysvinit / initscripts, this conversion to symlinks will need to be done in the systemd package itself.

Transition of packages

Packages using /lib/init/rw

This list was generated by running grep "/lib/init/rw" over all unpacked source packages from main, contrib and non-free. lib-init-rw.txt

  1. aide (using sendsigs.omit.d)

  2. bibledit (mentioning /lib/init/rw in documentation, no blocker)

  3. bup (using sendsigs.omit.d in examples, no blocker)

  4. chkrootkit

  5. console-common (splashy progress interface)

  6. cruft (filters files from sendsigs.omit.d)

  7. debootstrap

  8. eeepc-acpi-scripts

  9. fcheck

  10. hostapd (using sendsigs.omit.d)

  11. src:linux-atm (using sendsigs.omit.d)

  12. live-build

  13. logcheck (using sendsigs.omit.d)

  14. ltsp (using sendsigs.omit.d)

  15. lxc

  16. mdadm

  17. nbd (using sendsigs.omit.d)

  18. src:nfs-utils (using sendsigs.omit.d)

  19. pbuilder

  20. portmap (using sendsigs.omit.d)

  21. readahead-fedora

  22. src:refpolicy (SELinux policy needs to be updated for /run)

  23. resolvconf (621503) <--- migrated with 1.49 (experimental)

  24. rpcbind (using sendsigs.omit.d)

  25. rsyslog (using sendsigs.omit.d)

  26. splashy (using sendsigs.omit.d)

  27. sysklogd (using sendsigs.omit.d)

  28. systemd

  29. sysvinit (using sendsigs.omit.d)

  30. tiger

  31. unionfs-fuse (example file using sendsigs.omit.d, no blocker)

  32. wpasupplicant (using sendsigs.omit.d)

  33. xymon

Packages using /dev/.*

This list is incomplete! Please extend as far as possible.

  1. udev - /dev/.udev/, v167+ will automatically use /run/udev if /run is available and writable. (168-1 available from unstable)

  2. initramfs-tools - /dev/.initramfs/, /dev/.initramfs-tools - (621803)

  3. live-boot-initramfs-tools - /dev/.initramfs/varrun/sendsigs.omit

  4. dracut - /dev/.dracut/

  5. mdadm - /dev/.mdadm/

  6. systemd - /dev/.systemd/ (v25 uses /run/systemd)

  7. util-linux (mount) - /dev/.mount (v2.19.1 uses /run/mount)

Packages using /etc

This list is incomplete! Please extend as far as possible.

  1. /etc/lvm/cache/ - lvm2

  2. /etc/adjtime
  3. /etc/mtab (symlink to /proc/self/mounts 494001)

  4. /etc/network/run/ifstate - ifupdown

Packages using /var/run and /var/lock

The compatibility symlinks will automatically make any files created in /var/run and /var/lock available under /run and /run/lock, respectively. Therefore the only change needed is to switch to using the new paths in any code creating or referencing files in these directories. Before wheezy, a versioned depends upon initscripts (>= 2.88dsf-13.3) will be required to ensure the presence of a functional /run. After wheezy, /run will always be present, and no dependency will be needed.

It is not currently planned to require transition from /var/run and /var/lock for wheezy. However, if you wish to do so, please see below. A transition may occur in wheezy+1.

Packages using /dev/shm

If you're using /dev/shm directly, then your package is broken. You should only be accessing it via the eglibc shm_* and sem_* functions implementing the POSIX SHM and SEM features. If we do eventually deprecate /dev/shm in favour of /run/shm, eglibc will need configuring to use /run/shm directly. In consequence, if you are using the standard interfaces such as shm_open, sem_open etc., no transition is required.

Packages using /tmp

/tmp will always be available whether used directly or if it is symlinked to a location under /run. Programs should continue to use /tmp. (If configured to use /tmp/run, this should be considered purely an implementation detail, not something programs should need to know or care about).

Transition of filesystem locations

How to transition from /lib/init/rw to /run?

/lib/init/rw is scheduled to be removed. All users must transition to /run for wheezy.

General

There is no automatic migration of files from /lib/init/rw to /run. Packages will need to handle the transition by doing the following:

  1. Depend on initscripts (>= 2.88dsf-13.3)

  2. Replace all usage of /lib/init/rw with /run
  3. Move all files in /lib/init/rw to /run in the package postinst

Note that it is not possible to do this in the preinst. Because the initscripts dependency ensures that /run is available once initscripts is configured, /run can't be used until running the package postinst. Note that if open files are present, it may be necessary to stop the service before moving the files, and restart it after the move is complete.

sendsigs.omit transition

The sendsigs.omit.d interface is used by sysvinit to let applications register PIDs which should not be killed by the sendigs sysv init script on shutdown/reboot.

Packages using that interface currently do that by placing a file in /lib/init/rw/sendsigs.omit.d/ containing the PID. The new location is /run/sendsigs.omit.d/. As detailed in the General section, above, packages need to move their sendsigs file in the postinst. This is as simple as doing

if [ -f /lib/init/rw/sendsigs.omit.d/foo ]; then

  • mv /lib/init/rw/sendsigs.omit.d/foo /run/sendsigs.omit.d/foo

fi

in your package postinst.

How to transition from /dev, /dev/shm or /etc to /run?

All users must transition to /run for wheezy.

As for /lib/init/rw, there is no automatic migration of files from these locations to /run. Packages will need to handle the transition by doing the following:

  1. Depend on initscripts (>= 2.88dsf-13.3)

  2. Replace all usage of /dev/.* /dev/shm/.* or etc with /run
  3. Move all files in /dev/.* /dev/shm/.* or etc to /run in the package postinst

Note that it is not possible to do this in the preinst. Because the initscripts dependency ensures that /run is available once initscripts is configured, /run can't be used until running the package postinst. Note that if open files are present, it may be necessary to stop the service before moving the files, and restart it after the move is complete.

How to transition from /var/run to /run and /var/lock to /run/lock?

Files are automatically transitioned to the new location, and are available via both the old and new paths. As a result, packages do not need to move their files, they just need to start using the new paths by doing the following:

  1. Depend on initscripts (>= 2.88dsf-13.3)

  2. Replace all usage of /var/run with /run, and /var/lock with /run/lock.

Note that the initscripts dependency is only required up to the release of wheezy. If you transition after wheezy, /run will be guaranteed to be present, and the initscripts dependency may be omitted.

FAQ

Is /run FHS compliant?

/run is not yet described in the Filesystem Hierarchy Standard (FHS), though a proposal does exist to add it. The FHS specifies a minimum set of directories which must be present in order to be FHS compliant. Adding additional top-level directories is not permitted by individual applications, but it is permitted for distributions to do so. So while not yet in the standard, adding /run is technically in compliance with the standard.

Why do we need /run?

There is a need for a writable location to store data during early boot, before / is made writable (and it might be read only even during normal operation). Currently, no cross-distribution standardised location exists for this purpose. Debian uses /lib/init/rw; Ubuntu apparently does some very complex stuff linking /var/run to /lib/init/rw and using showthrough mounts. Other distributions do their own thing. Several programs chose not to use /lib/init/rw due to it being non-standard, and continued to use /dev/.foo due to udev mounting a tmpfs there which could be abused as a data store during early boot. /run provides a standard place for these use cases.

Why can't /var/run and /var/lock stay under /var?

/var/run and /var/lock have never really fitted into the /var hierarchy. The contents of /var/run and /var/lock are not preserved across a system reboot, while the rest of /var is preserved. /var stores variable data for the system and applications, but /var/run and /var/lock are ephemeral data. /run as a new top level directory is responsible for storing data with this lifetime (ephemeral system state), and unifies all the separate places this type of data was stored previously (with the exception of /tmp, which is for ephemeral user data rather than system state).

Why put /dev/shm and /tmp under /run?

The remit of /run is the storage of ephemeral system state. /dev/shm is ephemeral shared memory and semaphores, so /run is a more approprate location than /dev. Importantly, it permits the sharing of a single tmpfs for all ephemeral state (though this is not enabled by default, it is now a configurable option). /tmp is less suited to /run, and does not use /run by default. However, it is possible to make it use /run if you want to, e.g. by symlinking /tmp to /run/tmp. While this will not be useful for most systems, it may be useful for small embedded systems which have a single writable filesystem (which need not be a tmpfs) on /run.

Making it possible to put all these filesystems under /run is an important step in making a read-only root filesystem a reality.

Bug reports

When filing bugs, please use User: rleigh@debian.org, Usertag: run-transition so they are properly tagged. User tagged bugs

References

  1. http://thread.gmane.org/gmane.linux.redhat.fedora.devel/146976

  2. https://lwn.net/Articles/436012/

  3. http://bugs.freestandards.org/show_bug.cgi?id=718

  4. http://lists.debian.org/debian-devel/2011/03/msg01118.html

  5. http://lists.debian.org/debian-devel/2011/03/msg01119.html

  6. 620157

  7. 620191

  8. Read FHS-2.3 online