/run

Introduce a new top-level directory, /run, and transition packages to use it.

+++ 24-May-2011: initramfs-tools (0.99) entered testing +++

+++ 20-Jun-2011: sysvinit (2.88dsf-13.6) entered testing +++

+++ 17-Dec-2011: sysvinit (2.88dsf-16) entered unstable, removing /lib/init/rw and moving /etc/mtab to be a symlink to /proc/mounts +++

Overview

This is a release goal for Wheezy.

/run is a new cross-distribution location for the storage of transient state files—that is, files containing run-time information that may or may not need to be written early in the boot process and which does not require preserving across reboots.

/run is a tmpfs.

/run replaces several existing locations described in the Filesystem Hierarchy Standard:

/run also replaces some other locations that have been used for transient files:

Dotfiles in /dev and files stored in /dev/shm make other than the originally intended use of those locations and their names and locations are often not the same across distributions. /run provides a better, standard location for them. It also provides a location for distribution-specific purposes such as those for which Debian earlier created /lib/init/rw.

Making the /run directory available brings us a step closer to the point where it is possible to use the system normally with the root filesystem mounted read-only, without requiring any clunky workarounds such as aufs/unionfs overlays.

/run is being introduced by Fedora and SuSE and others will follow. It has been proposed to the FHS for standardization.

In addition to migrating /var/run and /var/lock, the Debian initscripts implementation also migrates /dev/shm and (optionally, if the user makes it a symlink to /run/tmp) /tmp.

Policy

An update to debian-policy has been proposed (620870). This will be included in Policy once /run is actually present in Debian unstable. The proposed text is included below:

§9.1. File system hierarchy

§9.1.1. File System Structure

7. The following directories in the root filesystem are additionally allowed: '/run', '/sys' and '/selinux'. [2]

[2] The '/run' directory is a replacement for '/var/run', and its subdirectory '/run/lock' is a replacement for '/var/lock'. These changes have been adopted by most distributions and have been proposed for inclusion in a future revision of the FHS. Files and directories residing in '/run' should be stored on a temporary filesystem, the purpose of which is storage of ephemeral system state which should not be persistent across a reboot. The '/sys' and '/selinux' directories are used as mount points to mount virtual filesystems to get access to kernel information.

§9.3. System run levels and 'init.d' scripts

§9.3.2. Writing the scripts

'/var/run' and '/var/lock' should be symlinks to '/run' and '/run/lock', respectively. This arrangement may also be satisfied through equivalent means, for example bind or nullfs mounts. Files and directories residing in '/run' should be stored on a temporary filesystem and not be persistent across a reboot, and hence the presence of files or directories in any of these directories is not guaranteed and 'init.d' scripts must handle this correctly. This will typically amount to creating any required subdirectories dynamically when the 'init.d' script is run, rather than including them in the package and relying on 'dpkg' to create them.

Annotation to §9.3.2.: Bind mount (Debian GNU/Linux) and nullfs mount (Debian GNU/kFreeBSD).

Implementation

Support for /run has been implemented in initscripts and initramfs-tools. In order to ensure that /run is both present and functional following package installation and configuration, the setup is done in two stages by initscripts:

Stage #1: Initial package install

This ensures that the /run hierarchy is present, but /var/run and /var/lock are still used for storing the files and directories accessed via /run.

Stage #2: After system reboot

At this point the migration is complete, with /run being a tmpfs, and the old locations /var/run, /var/lock and /dev/shm being converted to symlinks. Note that the fallback to bind mounts will only occur if the user has deliberately mounted something on /var/run, /var/lock or /dev/shm in /etc/fstab, and this will need cleaning up by the sysadmin in order to transition to symlinks at the next reboot (or they can clean up the mounts and create the symlinks by hand). If bind mounts are required, warning messages directing the user to remove the offending entries from /etc/fstab are displayed.

If the new version of initramfs-tools is present, it will mount /run in the early initramfs and move this over to the rootfs /run. This is not essential, but it means that any data created by programs such as udev in the early initramfs is preserved when it hands over control to the real init. udev will make use of /run if present.

Configuration

/run is a tmpfs.

Additional tmpfs filesystems are mounted if variables in /etc/default/rcS are set as follows.

If RAMLOCK=no, then /run/lock will use the underlying /run tmpfs, rather than a separate tmpfs mount.

For each of these mounts, the size and access mode may be controlled via settings in /etc/default/tmpfs. For example, the size and mode of /run/lock are set using LOCK_SIZE and LOCK_MODE.

There is ongoing discussion about the most appropriate choice of default values for these variables.

Other init systems

upstart

The upstart package in Debian still uses the compat layer (/etc/rcS.d) to initialize the system during early boot, i.e. for mount and fsck it relies on initscripts and as a consequence there are no special steps to be taken to support /run.

In Ubuntu, mount and fsck is handled by "mountall". Support for /run has been added to that package in version 2.28.

systemd

Version (25-1) of systemd in unstable has now native support for /run, i.e. systemd ships a mount unit to setup a tmpfs for /run and two units to setup a bind mount for /var/lock and /var/run. These two unit files var-run.mount and var-lock.mount use ?ConditionPathIsDirectory=/var/run which means, if /var/lock and /var/run are symlinks, the bind mounts are not established.

The initscripts package will not convert /var/run and /var/lock to symlinks in its postinst, but uses /lib/init/mount-functions.sh to convert them to symlinks on the next reboot. As systemd does not use the mount facilities provided by sysvinit / initscripts, this conversion to symlinks will need to be done in the systemd package itself.

Transition of packages

Packages using /lib/init/rw

This list was generated by running grep "(/lib/init/rw|sendsigs.omit)" over all unpacked source packages from main, contrib and non-free. lib-init-rw.txt

Updated list is available from http://people.debian.org/~biebl/run-transition/lib-init-rw.archive-scan.2011-07-08.log

User tagged bugs

Not blocking:

  1. bibledit (mentioning /lib/init/rw in documentation, no blocker 633046)

  2. chkrootkit (633047). Was pending /run/mdadm migration. Patch available. Only used in example text.

  3. debootstrap (unmounts /lib/init/rw if present; no /run handling, 633049).

  4. pbuilder (633055). Only used in hook scripts and testsuite logs.

  5. systemd (633059). For compatibility; does not block transition.

  6. tiger (633060). No maintainer response. Patch available. Only used for security checking; does not block transition.

  7. unionfs-fuse (633039). No maintainer response. Patch available. Only used in example script; does not block transition.

Fixed:

  1. aide (using sendsigs.omit.d 633028). Was pending /run/mdadm migration.

  2. console-common (splashy progress interface 633048)

  3. console-setup (633153)

  4. cruft (filters files from sendsigs.omit.d 633025)

  5. eeepc-acpi-scripts (633050)

  6. fcheck (633051)

  7. hostapd (using sendsigs.omit.d 633026). No maintainer response. Fixed in SVN in July, but not uploaded.

  8. src:linux-atm (using sendsigs.omit.d 633029)

  9. live-build (633052)

  10. ltsp (using sendsigs.omit.d 633031).

  11. mdadm (633054)

  12. nbd (using sendsigs.omit.d 633032)

  13. src:nfs-utils (using sendsigs.omit.d 633034). No maintainer response. Patch available.

  14. portmap (using sendsigs.omit.d; removed from wheezy/unstable, but needs Breaks adding to initscripts to allow /lib/init/rw removal)

  15. readahead-fedora (633056)

  16. src:refpolicy (SELinux policy needs to be updated for /run 626720 and 632561)

  17. resolvconf (621503)

  18. rpcbind (using sendsigs.omit.d 633035). No maintainer response; patch available.

  19. rsyslog (using sendsigs.omit.d 633036)

  20. splashy (using sendsigs.omit.d 633037|splashy progress interface/lib/init/rw/splashy). Removed from Debian, but will require Breaks adding to initscripts to allow /lib/init/rw removal.

  21. sysklogd (using sendsigs.omit.d 633038). No maintainer response. Patch available.

  22. sysvinit (using sendsigs.omit.d, but needs removing last since it's the package actually providing it along with Breaks on all the other packages if this won't break squeeze upgrades. Fixed in git; /lib/init/rw support removed from initscripts, and Breaks added for all fixed packages.)

  23. wpasupplicant (using sendsigs.omit.d 633040)

  24. xymon (633061)

Fixed, but don't need initscripts Breaks:

  1. bup (using sendsigs.omit.d in examples, no blocker 633027)

  2. logcheck (using sendsigs.omit.d 633030)

  3. lxc (633053)

Packages using /dev/.*

This list is incomplete!

In need of fixing:

Fixed:

  1. udev - /dev/.udev/, v167+ will automatically use /run/udev if /run is available and writable. (168-2 available from unstable)

  2. initramfs-tools - /dev/.initramfs/, /dev/.initramfs-tools - (621803)

  3. dracut - /dev/.dracut/ - (628190)

  4. mdadm - /dev/.mdadm/ - (633054)

  5. systemd - /dev/.systemd/ (25-1 onwards uses /run/systemd)

  6. util-linux (mount) - /dev/.mount (v2.19.1 uses /run/mount)

Packages using /dev/.udev

This list was generated by running grep "(/dev/\.udev|/run/udev)" over all unpacked source packages from main, contrib and non-free. http://people.debian.org/~biebl/run-transition/dev-udev.archive-scan.2011-09-27.log

User tagged bugs

In need of fixing:

  1. alsa-driver (620786). Patch available. No maintainer response.

  2. aoetools (644308). Patch available. No maintainer response.

  3. ccstools (644310) Patch available. No maintainer response.

  4. fusioninventory-agent (644312). Broken use of udev internals (db directory no longer present)

  5. isdnutils (644315) Patch available.

  6. kvpnc (644316)

  7. makedev (644318) Patch available. No maintainer response.

  8. mdadm (644319). Patch available.

  9. mouseemu (622210)

  10. ocsinventory-agent (644320)

  11. openvpn (644321)

  12. pmud (644322)

  13. refpolicy (644325) Patch available. No maintainer response.

  14. samhain (644327)

  15. tomoyo-tools (644328)

  16. tpb (644329) Patch available. No maintainer response.

  17. udftools (622208). Patch available. No maintainer response.

  18. userdevfs (644330)

Partially fixed (uses old and new locations):

  1. util-linux (629274 , 620787)

Fixed:

  1. aide (644307)

  2. alsa-utils (false positive, changelog only)

  3. bootcd (644309)

  4. bup (false positive, sample data/examples)

  5. cryptmount (644311)

  6. debian-installer (fixed in d-i Git repository)

  7. fuse (fixed in 2.8.5-3)

  8. gnome-packagekit (644313)

  9. haskell-unixutils (644314)

  10. hw-detect (fixed in 1.87)

  11. initramfs-tools (fixed in 0.99)

  12. irda-utils (Md: false positive)

  13. isdnactivecards (false positive, changelog only)

  14. lilo (644317)

  15. lm-sensors (fixed in 1:3.3.0-3)

  16. metainit (false positive, sample data/examples)

  17. open-iscsi (622209)

  18. pyudev (644323)

  19. qemu-kvm (644324)

  20. rkhunter (644326)

  21. system-config-printer (Md: false positive)

  22. sysvinit (saves /dev/.udev.log via bootlogs, fixed in 2.88dsf-13.11)

False positive:

  1. pyudev (644323)

Packages using /dev/.mdadm

This list was generated by running grep "(/dev/\.mdadm)" over all unpacked source packages from main, contrib and non-free. http://people.debian.org/~biebl/run-transition/dev-mdadm.archive-scan.2011-07-11.log

In need of fixing:

  1. refpolicy (643490)

  2. rkhunter (harmless)

Fixed:

  1. dracut

  2. mdadm

Packages using /dev/.initramfs

This list was generated by running grep "(/dev/\.initramfs)" over all unpacked source packages from main, contrib and non-free. http://people.debian.org/~biebl/run-transition/dev-initramfs.archive-scan.2011-07-11.log

In need of fixing:

  1. rkhunter (harmless)

Fixed:

  1. dracut (fixed in 011+36+gd727c5a-1)

  2. bup (false positive, sample data)

  3. initramfs-tools (fixed in 0.99)

  4. live-boot (fixed in 3.0~a17-1)

  5. mandos (643554)

  6. sysvinit (643558)

Packages using /etc

This list is incomplete!

In need of fixing:

  1. /etc/adjtime

Fixed:

  1. /etc/lvm/cache/ - lvm2 - (562234). Now migrated to /run and /run/lock.

  2. /etc/mtab - initscripts - symlink to /proc/self/mounts - (494001). Fixed in git, and will enter unstable at the same time as /lib/init/rw removal.

  3. /etc/network/run/ifstate - ifupdown - (623523). Now using /run/network/ifstate.

Packages using /var/run and /var/lock

The compatibility symlinks will automatically make any files created in /var/run and /var/lock available under /run and /run/lock, respectively. Therefore the only change needed is to switch to using the new paths in any code creating or referencing files in these directories. Before wheezy, a versioned depends upon initscripts (>= 2.88dsf-13.3) will be required to ensure the presence of a functional /run. After wheezy, /run will always be present, and no dependency will be needed.

It is not currently planned to require transition from /var/run and /var/lock for wheezy. However, if you wish to do so, please see below. A transition may occur in wheezy+1.

Packages using /dev/shm

If you're using /dev/shm directly, then your package is broken. You should only be accessing it via the eglibc shm_* and sem_* functions implementing the POSIX SHM and SEM features. If we do eventually deprecate /dev/shm in favour of /run/shm, eglibc will need configuring to use /run/shm directly. In consequence, if you are using the standard interfaces such as shm_open, sem_open etc., no transition is required.

Packages using /tmp

/tmp will always be available whether used directly or if it is symlinked to a location under /run. Programs should continue to use /tmp. (If configured to use /run/tmp, this should be considered purely an implementation detail, not something programs should need to know or care about).

Transition of filesystem locations

How to transition from /lib/init/rw to /run?

/lib/init/rw is scheduled to be removed. All users must transition to /run for wheezy.

General

There is no automatic migration of files from /lib/init/rw to /run. Packages will need to handle the transition by doing the following:

  1. Depend on initscripts (>= 2.88dsf-13.3)

  2. Replace all usage of /lib/init/rw with /run
  3. Move all files in /lib/init/rw to /run in the package postinst

Note that it is not possible to do this in the preinst. Because the initscripts dependency ensures that /run is available once initscripts is configured, /run can't be used until running the package postinst. Note that if open files are present, it may be necessary to stop the service before moving the files, and restart it after the move is complete.

sendsigs.omit transition

The sendsigs.omit.d interface is used by sysvinit to let applications register PIDs which should not be killed by the sendigs sysv init script on shutdown/reboot.

Packages using that interface currently do that by placing a file in /lib/init/rw/sendsigs.omit.d/ containing the PID. The new location is /run/sendsigs.omit.d/. As detailed in the General section, above, packages need to move their sendsigs file in the postinst. This is as simple as doing

if [ -f /lib/init/rw/sendsigs.omit.d/foo ]; then

  • mv /lib/init/rw/sendsigs.omit.d/foo /run/sendsigs.omit.d/foo

fi

in your package postinst.

How to transition from /dev, /dev/shm or /etc to /run?

All users must transition to /run for wheezy.

As for /lib/init/rw, there is no automatic migration of files from these locations to /run. Packages will need to handle the transition by doing the following:

  1. Depend on initscripts (>= 2.88dsf-13.3)

  2. Replace all usage of /dev/.* /dev/shm/.* or etc with /run
  3. Move all files in /dev/.* /dev/shm/.* or etc to /run in the package postinst

Note that it is not possible to do this in the preinst. Because the initscripts dependency ensures that /run is available once initscripts is configured, /run can't be used until running the package postinst. Note that if open files are present, it may be necessary to stop the service before moving the files, and restart it after the move is complete.

Early initramfs

The initramfs-tools init shows how udev deals with this. /run is guaranteed to be present in the initramfs with initramfs-tools (>= 0.99), so it's safe to use /run unconditionally in the initramfs and also on the host (you won't need to migrate files from /run in the initramfs to /run on the host, since it's the same tmpfs).

rootfs

With an initscripts dependency, /run is guaranteed to be present, but we may or may not have /dev/.xxx or /run/xxx (depending on the initramfs which may or may not have been generated with the new initramfs-tools). As a result, you'll want to move /dev/.xxx to /run/xxx if appropriate.

Summary

With the initscripts dependency, you can be sure of having /run on the rootfs. But due to version differences in initramfs-tools, /run may not be handed over to the rootfs cleanly from the initramfs, so you'll need the above two fallbacks to ensure that the files end up where you want them to be. With up-to-date versions of both initramfs-tools and initscripts the process is entirely transparent, but you do need to cater for upgrades. In wheezy, this logic can be removed, and you can just use /run without any messing around.

How to transition from /var/run to /run and /var/lock to /run/lock?

Files are automatically transitioned to the new location, and are available via both the old and new paths. As a result, packages do not need to move their files, they just need to start using the new paths by doing the following:

  1. Depend on initscripts (>= 2.88dsf-13.3)

  2. Replace all usage of /var/run with /run, and /var/lock with /run/lock.

Note that the initscripts dependency is only required up to the release of wheezy. If you transition after wheezy, /run will be guaranteed to be present, and the initscripts dependency may be omitted.

FAQ

Is /run FHS compliant?

/run is not yet described in the Filesystem Hierarchy Standard (FHS), though a proposal does exist to add it. The FHS specifies a minimum set of directories which must be present in order to be FHS compliant. Adding additional top-level directories is not permitted by individual applications, but it is permitted for distributions to do so. So while not yet in the standard, adding /run is technically in compliance with the standard.

Why do we need /run?

There is a need for a writable location to store data during early boot, before / is made writable (and it might be read only even during normal operation). Currently, no cross-distribution standardized location exists for this purpose. Debian uses /lib/init/rw; Ubuntu apparently does some very complex stuff linking /var/run to /lib/init/rw and using showthrough mounts. Other distributions do their own thing. Several programs chose not to use /lib/init/rw due to it being non-standard, and continued to use /dev/.foo due to udev mounting a tmpfs there which could be abused as a data store during early boot. /run provides a standard place for these use cases.

Why can't /var/run and /var/lock stay under /var?

/var/run and /var/lock have never really fitted into the /var hierarchy. The contents of /var/run and /var/lock are not preserved across a system reboot, while the rest of /var is preserved. /var stores variable data for the system and applications, but /var/run and /var/lock are ephemeral data. /run as a new top level directory is responsible for storing data with this lifetime (ephemeral system state), and unifies all the separate places this type of data was stored previously (with the exception of /tmp, which is for ephemeral user data rather than system state).

Why put /dev/shm and /tmp under /run?

The remit of /run is the storage of ephemeral system state. /dev/shm is ephemeral shared memory and semaphores, so /run is a more approprate location than /dev. Importantly, it permits the sharing of a single tmpfs for all ephemeral state (though this is not enabled by default, it is now a configurable option). /tmp is less suited to /run, and does not use /run by default. However, it is possible to make it use /run if you want to, e.g. by symlinking /tmp to /run/tmp. While this will not be useful for most systems, it may be useful for small embedded systems which have a single writable filesystem (which need not be a tmpfs) on /run.

Making it possible to put all these filesystems under /run is an important step in making a read-only root filesystem a reality.

Bug reports

When filing bugs, please use

User: rleigh@debian.org
Usertags: run-transition

User tagged bugs

References

  1. http://thread.gmane.org/gmane.linux.redhat.fedora.devel/146976

  2. https://lwn.net/Articles/436012/

  3. http://bugs.freestandards.org/show_bug.cgi?id=718

  4. http://lists.debian.org/debian-devel/2011/03/msg01118.html

  5. http://lists.debian.org/debian-devel/2011/03/msg01119.html

  6. 620157

  7. 620191

  8. Read FHS-2.3 online