Description

The main goal of the merged-/usr proposal is to merge the contents of several top root directories (/bin, /sbin, /lib*) into their counterparts in /usr. On Debian systems it is expected that during boot /usr will be available, because it is either not a separate partition, or it has already been mounted by an initramfs. This can reduce the overhead of distinct directories and where to put what. It also makes it easier to have a read-only /usr filesystem.

Parts

This proposal is composed of three parts:

  1. Where to store the objects in the filesystem.
  2. How to move them.
  3. How to provide backwards compatibility.

Approaches

merged-/usr-via-aliased-dirs (was merged-/usr-via-symlinks)

  1. The objects end up stored under /usr/*.
  2. The move is performed at install-time by an unaware dpkg, through the aliased directories.
  3. The backward compatibility is provided by the aliased directories.

This approach goes behind dpkg's back, and has caused and do now cause problems due to the aliased directories, as multiple pathnames canonicalize into different pathnames that point to the same dentry, which can mess up anything that handles pathnames in databases and similar. Note: dpkg has supported for a long time symlinked directories as a way to allow local admins to manage filesystem size constraints, by moving directories contents into other partitions, but has never supported aliased directories via symlinks pointing to other directories already tracked by dpkg. This approach makes it impossible to know what is the canonical (from dpkg's PoV) pathname for an object just by looking at the filesystem, so it is trivial for users use the wrong one. One of the reasons for this approach was to make a migration quick and to require less effort in converting pathnames one-by-one, but has resulted instead in making the individual pathname migration more difficult and less declarative, and as such more fragile, and has created a bleeding problem due to the aliased directories:

This approach is considered broken by design and is unsupported by dpkg. dpkg-buildinfo marks packages built on these systems as tainted in the .buildinfo file.

Unfortunately debootstrap defaults to using this approach, and by extension d-i installs broken systems since Debian buster, but there's a bug report 923091 with a patch to at least make it selectable in expert mode. For now you can pass --no-merged-usr to debootstrap (or you can use instead mmdebstrap which is not affected by this, is way faster, and it intends to replace all bootstrapping tools by making them completely unnecessary anyway, or cdebootstrap); or for d-i install in export mode, select to use cdebootstrap and proceed with the installation, or alternatively step until before the "Install the base system", spawn a shell, and edit the /var/lib/dpkg/info/bootstrap-base.postinst, add the --no-merged-usr option to the run-debootstrap invocation, then exit the shell and proceed with the installation. Other systems might have used this approach via the usrmerge hack.

For already installed systems (since dpkg 1.20.6) you can also use the dpkg-fsys-usrunmess program to revert the breakage from these systems (but beware that it should not be used in systemd's emergency mode).

  1. The objects end up stored under /usr/*.
  2. The move is performed at build-time with full collaboration with dpkg.
  3. The backward compatibility (if necessary, for binaries or ABI providing pathnames) can be done with symlinks.

This approach could have been fully automated via debhelper, in a safe way for every debhelper-using package, with no maintainer scripts involved. This would imply currently a ton of maintainer script due to having to support the broken merged-/usr-via-aliased-dirs approach that has been deployed on new systems or via the usrmerge hack. This approach does not break dpkg at all. Compatibility symlinks are explicit, and can be dropped once nothing uses those pathnames, making it obvious what still remains, and not confusing users that look into the filesystem.

This still suffers mild aliasing problems, but only for pathnames that will end up in both locations due to backwards compatibility symlinks, which should keep decreasing, and any such issue will self-heal over time, eventually ending up being just a handful of them. The big difference is that in the long term this is a tiny and decreasing bounded set of pathnames that might end up causing problems, while the other layout is an unbounded set affecting all pathnames permanently. Another thing that makes a difference is that the object is a symlink, and not the same object being accessed from different pathnames.

Properties

Property

merged-/usr-via-aliased-dirs

merged-/usr-via-moves-and-symlink-farms

Pathnames can be found in both places

Yes

Partially 1

Can be deployed right away, with a quick transition

Partially/No 2

No 3

Requires a flag day

Yes

No

Transition can be automated by debhelper

Partially 4

Yes 5

Transition can be helped by dpkg

No 6

Yes 7

Sharing /usr via the network, can get / and /usr directories out-of-sync

No

Yes 8

dpkg stack, u-a fails to notice file conflicts 9

Yes 10

No

dpkg, dpkg-divert can disappear aliased pathnames

Yes

No

dpkg-deb/tar, f.ex., extracting .debs into «/» will convert the symlinked dirs into real ones 11

Yes 12

No

Aliasing problems

Yes 13

No

  1. Only for pathnames that were previously under /, and have not been removed yet after a transition period, or no API/standard requirement. (1)

  2. Moving the pathnames in .deb will take the same amount of time as the other approach. Shipping the aliased symlinks in a package might require waiting for all packages having moved their pathnames under /usr or there will be ordering problems during bootstrapping. (2)

  3. Only requires transitioning the .debs, no other transition is required. (3)

  4. Moving the files could be done, making something ship the symlinked directories, guaranteeing dependency ordering or they might end up being directories instead of symlinks, will require special handling. (4)

  5. Except for the few packages not using debhelper of course. (5)

  6. Switching the aliased directories cannot be done safely by dpkg. (6)

  7. A new dpkg-maintscript-helper action will be implemented, and a new dpkg command added to register the symlinks. (7)

  8. But /etc and /var can still do, and this optimizes for admins that might know what they are doing instead of end users that might not, due to the unreliability introduced. (8)

  9. Can end up overwriting or disappearing files silently. (9)

  10. This might be controllable by experienced admins, but users installing 3rd-party repos can very easily be affected. (10)

  11. Pretty much messing up the system. (11)

  12. Experienced admins might know what they are doing, end users following instructions from the net, will easily get their system completely damaged. (12)

  13. Because all pathnames are visible in both hierarchies, but some of these pathnames are part of public APIs/standards and cannot be easily changed, we will keep having accesses through the aliased directories (even after all objects have been migrated), but programs managing/tracking the filesystem, such as dpkg/dpkg-query or any of its callers, will only know about one of them, and will become unreliable, as the correct answer will not be given. (13)