Differences between revisions 11 and 12
Revision 11 as of 2019-03-15 03:13:25
Size: 10518
Editor: PaulWise
Comment: typo
Revision 12 as of 2019-03-15 03:14:43
Size: 10519
Editor: PaulWise
Comment: typo
Deletions are marked like this. Additions are marked like this.
Line 30: Line 30:
This has the nicer property of requiring no execution of maintainer scripts, so no complications with chrootless runs, and ending up with a finished image not required a second-stage configuration for foreign architectures. This has the nicer property of requiring no execution of maintainer scripts, so no complications with chrootless runs, and ending up with a finished image not requiring a second-stage configuration for foreign architectures.

Status: draft

Summary

Currently the logic and implicit dependencies to bootstrap a distribution installation is hardcoded in packages such as debootstrap or cdebootstrap (but not mmdebstrap or multistrap which are trying to solve this in a cleaner way). This is a problem because it is fragile, needs to be duplicated on each project that wants to bootstrap a distribution, it is not self-contained, it is also distribution specific, and can easily get out-of-sync with the packages found in the archive, their implied dependencies or the order they might get unpacked or configured by dpkg.

Recent examples showing the fragility of this approach are 760568, 766459, 767999, regardless of what package was at fault here.

Further detailed description of the problem.

Proposal: full declarative pseudo-Essential

Remove any maintainer script usage for the bootstrap case ("configure"/"install" w/o a previous version argument) in all of the pseudo-Essential set. And rely on declarative metadata. This requires supporting:

  • Declarative alternatives.
  • Declarative diversions.
  • Declarative file metadata.
    • With support for ghost files.
    • With support for files initialized from a template.
  • Declarative config files / conffiles.
    • With support for files initialized from a template.

The rest might require fixing code to:

  • Add support for /etc/shells.d/ fragment files, to avoid needing add-shell.
  • Fixing programs to generate required, but missing files.

This has the nicer property of requiring no execution of maintainer scripts, so no complications with chrootless runs, and ending up with a finished image not requiring a second-stage configuration for foreign architectures.

Current maintscript usage in pseudo-Essential set

So, when all of the above have been removed, we could update policy to state that the bootstrap case requires 0 code execution from maintscripts. To track progress, for maintscripts that need to handle also non-bootstrap code, once the bootstrap code has been removed we could mark it with something like:

  • # Package in pseudo-Essential set with no bootstrap maintscript code.

    if [ "$1" = configure ] && [ -z "$2" ]; then

    • exit 0
    fi

The current list of uses is:

  • $ dpkg-query -f '${binary:Package}\n' -W \
    • $(/usr/sbin/debootstrap --variant=minbase --print-debs \
      • --exclude=apt,e2fsprogs sid /tmp/sid-tmp 2>/dev/null) \

      | grep -v :i386 \ | while read pkg; do
      • ls -1 /var/lib/dpkg/info/$pkg.*inst 2>/dev/null; \

      • done
  • /var/lib/dpkg/info/base-files.postinst
    • Creates directories with proper owner/permissions.
    • Creates symlinks for the default dpkg origin file.
    • Creates compat symlinks for some directories.
    • Initializes config files from templates.
    • Initializes state files from templates.
    • Initializes "log" files.
    • Initializes /run file. This should be removed, as /run is supposed to be clean after each boot.
    • Initializes various dpkg files. This should probably be removed.
  • /var/lib/dpkg/info/base-passwd.postinst
    • Ininitializes config files from templates.
    • Runs update-passwd, although I think this should not need to be run on the bootstrap case.
  • /var/lib/dpkg/info/base-passwd.preinst
    • Ininitializes config files from embedded templates in the maintscript. Duplicates the logic in the postinst script.
  • /var/lib/dpkg/info/bash.postinst
    • Installs alternatives.
    • Adds itself into /etc/shells via add-shell.
  • /var/lib/dpkg/info/bash.preinst (compiled binary)
    • Handles the /bin/sh symlink creation if missing, and diversion installation. But perhaps this can be removed now, ISTR a bug report requesting just that, but cannot find it.
  • /var/lib/dpkg/info/coreutils.postinst
    • Creates /usr/bin/ to /bin/ transition symlink. [ Due to merged-usr-via-symlinks. :( … ]

  • /var/lib/dpkg/info/dash.postinst
    • Handles /bin/sh symlink creation if missing, and diversion installation.
    • Adds itself into /etc/shells via add-shell.
  • /var/lib/dpkg/info/debconf.postinst
    • Removes an obsolete conffile.
  • /var/lib/dpkg/info/debconf.preinst
    • Removes an obsolete conffile.
    • Renames a conffile.
  • /var/lib/dpkg/info/debianutils.postinst
    • Initializes /etc/shells from a template.
    • Creates /usr/bin/ to /bin/ transition symlink. [ Due to merged-usr-via-symlinks. :( … ]

  • /var/lib/dpkg/info/dpkg.postinst
    • (./) Initializes its database and log files with owner and permissions. This should probably be done at run-time instead, there's already a report about this. This is fixed now in the next/bootstrap branch.

  • /var/lib/dpkg/info/libc-bin.postinst
    • Initialize config file from template.
    • Runs ldconfig.
  • /var/lib/dpkg/info/libc6:amd64.postinst
    • Removes hwcappkgs registry. Only relevant on upgrades.
    • Handles nohwcap file removal. Only relevant on upgrades.
    • Restarts init. Only relevant on upgrades.
  • /var/lib/dpkg/info/libc6:amd64.preinst
    • LD_LIBRARY_PATH sanity checks. Only relevant on upgrades.
    • Kernel version requirement checks.
  • /var/lib/dpkg/info/libgcc1:amd64.postinst
    • Handles directory to symlink transition. Only relevant on upgrades.
  • /var/lib/dpkg/info/libpam-modules:amd64.postinst
    • Creates empty config files.
  • /var/lib/dpkg/info/libpam-modules:amd64.preinst
    • Does nothing during bootstrapping.
  • /var/lib/dpkg/info/libpam-runtime.postinst
    • Run pam-auth-update.
    • Initialize config file from template.
  • /var/lib/dpkg/info/libpam0g:amd64.postinst
    • Does nothing during bootstrapping.
  • /var/lib/dpkg/info/login.postinst
    • Obsolete init script conffile removal.
    • Create config files with owner and permissions.
  • /var/lib/dpkg/info/login.preinst
    • Does nothing during bootstrapping.
  • /var/lib/dpkg/info/mawk.postinst
    • Installs alternatives.
  • /var/lib/dpkg/info/passwd.postinst
    • Fixes permissions for log files, should only be needed on upgrade, and it should actually be removed now.
    • Handles shadow group addition, only relevant for upgrades.
    • Runs shadownconfig.
    • Runs systemd-tmpfiles.
  • /var/lib/dpkg/info/passwd.preinst
    • Does nothing during bootstrapping.
  • /var/lib/dpkg/info/tar.postinst
    • Installs alternatives.
  • /var/lib/dpkg/info/tzdata.postinst
    • Does nothing during bootstrapping. The actual initial timezone setup is off-loaded to the bootstrapping programs. :/
  • /var/lib/dpkg/info/util-linux.postinst
    • Installs alternatives.
    • Removes obsolete alternatives.
    • Runs helpers for sysvinit/systemd setup.

Proposal: chrootless maintscripts

The installation bootstrap logic for any pseudo-essential package currently handled in tools such as debootstrap or cdebootstrap would be moved into a new package maintainer script or similar. Those would need to be run from outside the chroot, so that we are not back to the problem of implicit assumptions and ordering though. And the expectations on the external environment would need to be specified, for example assuming just POSIX utilities (or a subset of it).

This could also be used to bootstrap a foreign architecture, as the setup would be done by the native system, but in this case it would require not-chroot'ing and passing to the maintainer script the path of the root directory.

Detached chroot handling

A related topic, is the handling of chroots, be them native or foreign, without requiring chroot(2)ing into the directory. This required adding a new environment variable set by dpkg named DPKG_ROOT, which can be used by maintainer scripts when needing to interact with the chroot filesystem.

Starting with dpkg 1.18.5, maintainer scripts get the environment variable DPKG_ROOT set by default to the empty string. With older dpkg it will be unset, so if the script uses set -u you might need to do something like «: "${DPKG_ROOT=}"» for backwards compatibility. If the user requests the new mode of operation the DPKG_ROOT environment variable will contain a chroot path that can be prepended to the pathnames accessed. For example

  • echo config > $DPKG_ROOT/etc/package/package.conf

The only way to test this currently is to force the new mode with the --root and --force-script-chrootless, but that will force the mode even when the affected packages do not support it, which might damage your host system, do not use without very careful consideration; running dpkg as a nonpriviledged user can mitigate that issue (possibly requiring fakeroot).

Support for DPKG_ROOT utilization is being added to dpkg's own tools first (eg. update-alternatives), and being tracked under the dpkg-root-support usertag of debian-dpkg@lists.debian.org.

Maintainer script dependencies

Because this is not limited to packages in the Essential set, any Depends and Pre-Depends would need to be installed on the host systems. To avoid having to install all of those in the host, Helmut Grohne proposed adding a new Maint-Depends field, which would declare dependencies required by the maintainer scripts, and those and only those would need to be present on the host system.

This is problematic though, and several reasons have been presented in 804624, but the hope is to eventually find a solution to this problem.