= Description =

The main goal of the merged-/usr proposal is to merge the contents of several top root directories (/bin, /sbin, /lib*) into their counterparts in /usr. On Debian systems it is expected that during boot /usr will be available, because it is either not a separate partition, or it has already been mounted by an initramfs. This can reduce the overhead of distinct directories and where to put what. It also makes it easier to have a read-only /usr filesystem.

 FIXME: expand, rationale, etc.

== Parts ==

This proposal is composed of three parts:

  a. Where to store the objects in the filesystem.
  a. How to move them.
  a. How to provide backwards compatibility.

== Approaches ==

=== merged-/usr-via-aliased-dirs (was merged-/usr-via-symlinks) ===

  a. The objects end up stored under /usr/*.
  a. The move is performed at install-time by an unaware dpkg, through the aliased directories.
  a. The backward compatibility is provided by the aliased directories.

This approach goes behind dpkg's back, and has caused and do now cause problems due to the aliased directories, as multiple pathnames canonicalize into different pathnames that point to the same dentry, which can mess up anything that handles pathnames in databases and similar. Note: dpkg has supported for a long time symlinked directories as a way to allow local admins to manage filesystem size constraints, by moving directories contents into other partitions, but has '''never''' supported aliased directories via symlinks pointing to other directories already tracked by dpkg. This approach makes it impossible to know what is the canonical (from dpkg's PoV) pathname for an object just by looking at the filesystem, so it is trivial for users use the wrong one. One of the reasons for this approach was to make a migration quick and to require less effort in converting pathnames one-by-one, but has resulted instead in making the individual pathname migration more difficult and less declarative, and as such more fragile, and has created a bleeding problem due to the aliased directories:

  * :-( all packages that shipped in the .deb compatibility symlinks between / and /usr/ or the reverse, required these to be moved into maintainer script and conditionalized on whether the system had aliased directories.
  * {OK} apparmor required fixes to its policies.
  * {OK} selinux required fixes to its policies.
  * <insert other projects handling filesystem pathnames here...>.
  * {OK} dpkg-shlibdeps required fixes due to dpkg-query breaking, which then required more fixes due to the "fix" breaking cross-compilation.
  * {X} dpkg-query -S is currently broken by this approach, will fail to print entries for pathname queries not matching what is on the database (even if present on the filesystem).
  * {X} dpkg-trigger is currently broken by this approach, will fail to trigger pathnames even if present on the filesystem but not matching what is on the database.
    * Assuming that no packages install activation files in / when the trigger is for /usr (which would be broken anyway), this presently affects three packages in Debian unstable: ''orphan-sysvinit-scripts'', ''runit'' and ''udev''.
  * {X} dpkg and dpkg-divert are currently broken by this approach, will fail to notice file conflicts and will overwrite files.
  * {X} dpkg and dpkg-divert are currently broken by this approach, can disappear files (depending on the installation order).
    * {X} moving a pathname from a .deb (say /bin/foo in pkg A) to another .deb while changing the pathname across an aliased directory (say /usr/bin/foo in pkg B), is broken and can make (depending on the unpack order) the new pathname disappear from the filesystem.
  * {X} dpkg-repack is currently broken by this approach, and repacked packages will not match what was installed.
  * {X} update-alternatives is currently broken by this approach, will fail to notice file conflicts and will overwrite alternatives.
  * {X} update-alternatives is currently broken by this approach, can disappear alternatives.
  * {X} tar -x on the root directory will break systems using this approach, by overwriting the symlinked directories with actual ones.
  * {X} dpkg-deb -x on the root directory will break systems using this approach, by overwriting the symlinked directories with actual ones. DebianBug:989602
  * {OK} DebianPkg:cruft has been broken by this approach and has thus been replaced by DebianPkg:cruft-ng. DebianBug:941998
  * {OK} Since ''/etc/shells'' is managed using ''update-shells'', it contains aliased entries automatically. ''bash'' and ''dash'' have been switched to use ''update-shells''. [[https://binarycontrol.debian.net/?q=add-shell&path=%2Funstable%2F*|Remaining users]] of ''add-shell'' are often violating Debian policy 10.7.3.
  * {X} dpkg-statoverride may fail to apply its changes due to differences in describing the location.

{{{#!wiki warning
This approach is considered broken by design and is unsupported by dpkg. dpkg-buildinfo marks packages built on these systems as tainted in the .buildinfo file.
}}}

Unfortunately DebianPkg:debootstrap defaults to using this approach, and by extension d-i installs broken systems since Debian buster, but there's a bug report DebianBug:923091 with a [[https://salsa.debian.org/installer-team/base-installer/merge_requests/1|patch]] to at least make it selectable in expert mode. For now you can pass ''--no-merged-usr'' to DebianPkg:debootstrap (or you can use instead DebianPkg:mmdebstrap which is not affected by this, is way faster, and it intends to replace all bootstrapping tools by making them completely unnecessary anyway, or DebianPkg:cdebootstrap); or for d-i install in export mode, select to use DebianPkg:cdebootstrap and proceed with the installation, or alternatively step until before the "Install the base system", spawn a shell, and edit the ''/var/lib/dpkg/info/bootstrap-base.postinst'', add the '''--no-merged-usr''' option to the ''run-debootstrap'' invocation, then exit the shell and proceed with the installation. Other systems might have used this approach via the DebianPkg:usrmerge hack.

{{{#!wiki note
For already installed systems (since dpkg 1.20.6) you can also use the DebianMan:dpkg-fsys-usrunmess program to revert the breakage from these systems (but beware that it should not be used in systemd's emergency mode).
}}}

=== merged-/usr-via-moves-and-symlink-farms ===

  a. The objects end up stored under /usr/*.
  a. The move is performed at build-time with full collaboration with dpkg.
  a. The backward compatibility (if necessary, for binaries or ABI providing pathnames) can be done with symlinks.

This approach could have been fully automated via debhelper, in a safe way for every debhelper-using package, with no maintainer scripts involved. This would imply currently a ton of maintainer script due to having to support the broken '''merged-/usr-via-aliased-dirs''' approach that has been deployed on new systems or via the DebianPkg:usrmerge hack. This approach does not break dpkg at all. Compatibility symlinks are explicit, and can be dropped once nothing uses those pathnames, making it obvious what still remains, and not confusing users that look into the filesystem.

This still suffers mild aliasing problems, but only for pathnames that will end up in both locations due to backwards compatibility symlinks, which should keep decreasing, and any such issue will self-heal over time, eventually ending up being just a handful of them. The big difference is that in the long term this is a tiny and decreasing bounded set of pathnames that might end up causing problems, while the other layout is an unbounded set affecting all pathnames permanently. Another thing that makes a difference is that the object is a symlink, and not the same object being accessed from different pathnames.

  * /!\ Code querying ''/etc/shells'' might stop matching on entries that exist on the system, but are not listed with their expected and aliased forms. This cannot be "fixed" centrally in glibc, as the file itself is a public interface. But this only affects a subset of the shells, as several reside already under /usr.

== Properties ==

||<rowstyle="background-color: #424242; color: #f1f1f1; font-weight: bold;">Property || merged-/usr-via-aliased-dirs || merged-/usr-via-moves-and-symlink-farms ||
||Pathnames can be found in both places || Yes || Partially <<FootNote(Only for pathnames that were previously under /, and have not been removed yet after a transition period, or no API/standard requirement.)>> ||
||Can be deployed right away, with a quick transition|| Partially/No <<FootNote(Moving the pathnames in .deb will take the same amount of time as the other approach. Shipping the aliased symlinks in a package might require waiting for all packages having moved their pathnames under /usr or there will be ordering problems during bootstrapping.)>> || No <<FootNote(Only requires transitioning the .debs, no other transition is required.)>> ||
||Requires a flag day ||<#ee2222> Yes ||<#22ee22> No ||
||Transition can be automated by debhelper||Partially <<FootNote(Moving the files could be done, making something ship the symlinked directories, guaranteeing dependency ordering or they might end up being directories instead of symlinks, will require special handling.)>> || Yes <<FootNote(Except for the few packages not using debhelper of course.)>> ||
||Transition can be helped by dpkg||<#ee2222> No <<FootNote(Switching the aliased directories cannot be done safely by dpkg.)>> ||<#22ee22> Yes <<FootNote(A new dpkg-maintscript-helper action will be implemented, and a new dpkg command added to register the symlinks.)>>||
||Sharing /usr via the network, can get / and /usr directories out-of-sync ||<#22ee22> No ||<#bb9999> Yes <<FootNote(But /etc and /var can still do, and this optimizes for admins that might know what they are doing instead of end users that might not, due to the unreliability introduced.)>> ||
||dpkg stack, u-a fails to notice file conflicts <<FootNote(Can end up overwriting or disappearing files silently.)>> ||<#ee2222> Yes <<FootNote(This might be controllable by experienced admins, but users installing 3rd-party repos can very easily be affected.)>>||<#22ee22> No ||
||dpkg, dpkg-divert can disappear aliased pathnames ||<#ee2222> Yes ||<#22ee22> No ||
||dpkg-deb/tar, f.ex., extracting .debs into «/» will convert the symlinked dirs into real ones <<FootNote(Pretty much messing up the system.)>> ||<#ee2222> Yes <<FootNote(Experienced admins might know what they are doing, end users following instructions from the net, will easily get their system completely damaged.)>> ||<#22ee22> No ||
|| Aliasing problems ||<#ee8888> Yes <<FootNote(Because all pathnames are visible in both hierarchies, but some of these pathnames are part of public APIs/standards and cannot be easily changed, we will keep having accesses through the aliased directories (even after all objects have been migrated), but programs managing/tracking the filesystem, such as dpkg/dpkg-query or any of its callers, will only know about one of them, and will become unreliable, as the correct answer will not be given.)>> ||<#22ee22> No ||