Differences between revisions 10 and 11
Revision 10 as of 2008-04-01 21:47:17
Size: 4961
Editor: JoeyHess
Revision 11 as of 2009-03-16 03:31:49
Size: 4961
Editor: anonymous
Comment: converted to 1.6 markup
No differences found!

Dpkg Postponing Tasks Mechanism (DPTM)

Something very similar has been implemented as DpkgTriggers.

See the spec: http://lists.debian.org/debian-dpkg/2007/04/msg00076.html


During Debian package installation there exist several tasks that are executed by many packages repeatedly because:

  • each package has to provide complete and sany installation state when postinst is done
  • there is no way of communication between packages and dpkg, so a package cannot tell whether a program will be executed later or not, or how

For this reason we are going to develop a mechanism for integration with dpkg designed to minimize the number of redundant program calls, especially WRT IO/CPU time consuming applications like ldconfig.

Integration Policy

There are two types of applications covered by the postpone mechanism. Those that have dependents (ie. packages that rely on a certain state of installation which is not reached if a postponed operation is not executed yet, eg. modprobe on "depmod -a" calls) and those that don't (eg. menu-update).

Case1: strong dependency

When other packages require a certain state and therefore depend on a feature/state providing package, the postinst scripts of the client packages need to make sure that relevant outstanding operations are executed before the dependent operations can begin.

Decission to make use of DPTM to reach a certain goal must be met by consens between all involved stakeholders: maintainers of state providing package as well as their "customers", maintainers of packages relying on them. The issue should be brought up on a medium where most involved partys are listening/reading (eg. pkg-FOO-group-devel mailingl list on alioth) or via private email communication.

When a consens is reached, the preparation phase begins. Client package maintainers need to begin to integrate the execution triggering command (see Usage policy) into their postinst script, if required. (FIXME: changes to integrate DPTM can be declared as BSP goals justifying 0-day severity).

When all or a sufficient majority of client packages is adapted, the postponing hooks can be installed into the feature providing packages. Note that those packages need to conflict with all versions of client packages which may become broken because of that change (eg. foo (<= adapted_version) for all client packages). In the case of a large number of such client packages, the begin of this phase can be coupled with a Debian release (at which point all client packages must be converted).

Case2: no dependency

Without strong dependencies, the packages can run the hook client (ie. install postponed tasks) as done with usual program calls. To speedup the transition, DPTM package can install wrappers that emulate the behaviour of programs (eg. ldconfig) while postponing them.

Usage Policy

The provisoric name for the DPTM client command is "dpkg-hook". Basic usage:

  • dpkg-hook [ --background ] <postponed command cmd> [ <command arguments> ] - run cmd only once in total. With background, detach the process later.

  • dpkg-hook --add-option cmd option(s) - add argument 'path' to a previous cmd hook

  • dpkg-hook --run-pending cmd [ --add-option <options>] - run postponed tasks for the command cmd if there are any. To force the command to run at least one, use --add-option without arguments. And add options to add aditional options. Also accepts --background to detach the process.

  • dpkg-hook --on-dependents triggerpkg cmd - run once before depends of foobar (fixme: Dependency tracking sucks and may be unreliable, see integration policy instead)

Implementation Ideas

DTPM has main these tasks:

  • start the collecting daemon with dpkg (or maybe later, with the first hook client). (Ideas: Perl, Fork, Unix Domain Socket, diverted dpkg executable, functionality can be moved to dpkg later)
  • dpkg-hook (client) sends the commands to the collecting daemon (Ideas: Unix Domain Socket)
  • when dpkg is done, its wrapper sends the "trigger-all" command to the daemon, waits for the daemon to send the return code and exits. (FIXME: return that code or just 0?). Foreground processes are executed in the order of their scheduling, background process is then run independently.

The post-execution process should display status message like: Executing pending tasks:

  • fooconfig (requested by libfoo, libbar, libbaz) menu-update (requested by 23 packages, see dpkg.log for details)

And dpkg.log would contain things like:

2006-03-14 08:04:15 status post-configured libfoo 2.6.15-8 fooconfig (+option: "/lib/a /lib/b")

2006-03-14 08:04:15 status post-configured libbar 2.6.15-8 fooconfig (+option: "/lib/bar")

2006-03-14 08:04:15 status post-configured libbaz 2.6.15-8 fooconfig (+option: "/lib/baz")