Differences between revisions 12 and 13
Revision 12 as of 2011-06-06 23:33:07
Size: 7147
Editor: ?Sam Dunne
Comment:
Revision 13 as of 2011-06-07 19:35:03
Size: 7659
Editor: ?Sam Dunne
Comment: Formatting
Deletions are marked like this. Additions are marked like this.
Line 10: Line 10:
<<BR>>
Line 11: Line 12:
-----------------------
Line 20: Line 22:
<<BR>>
Line 21: Line 24:
-----------------------
Line 32: Line 36:
<<BR>>
Line 34: Line 38:
---------------------------------
Line 46: Line 51:
<<BR>>
Line 47: Line 53:
-----------------------------------
Line 62: Line 69:
<<BR>>
Line 63: Line 71:
-----------------------------------------------
Line 73: Line 82:
<<BR>>
Line 74: Line 84:
----------------------------------------------------
Line 89: Line 100:
       ----
Line 107: Line 118:
----
Line 116: Line 127:

<<BR>>
Line 117: Line 130:
------------------------------
Line 124: Line 138:
<<BR>>
Line 125: Line 140:
------------------------------------
Line 133: Line 149:
<<BR>>
Line 135: Line 151:
-----------------------------------
Line 140: Line 157:
----
Line 145: Line 162:
----
Line 146: Line 164:
<<BR>>
Line 147: Line 166:
-----------------------------------
Line 151: Line 171:
----
Line 155: Line 175:
----
Line 158: Line 178:
<<BR>>
Line 160: Line 180:
-------------------------------

Declarative diversions

  • Mentor: Steve Langasek; supported by Raphaël Hertzog for dpkg maintainer review and sign-off

  • Student: Sam Dunne

  • Summary: implement support for declarative diversions in dpkg, to obsolete manual calls to dpkg-divert in maintainer scripts



Introduction


A diversion is when it is possible to have dpkg not overwrite a file when it reinstalls the package it belongs to, and to have it put the file from the package somewhere else instead.

Declarative diversions involves a new control file with a declarative syntax which dpkg will parse and process directly as part of the package unpack and removal phases, eliminating the problems resulting from non-atomic handling of diversions.


Topics


There are a number of topics involved in implementing this kind of project

  • What syntax do we use for the new control file?
  • What dpkg does with the control file
  • How do we handle diversions to a non-existant directory?
  • How do we order unpacking a new package which adds a diversion?
  • How do we order unpacking a new package that removes a diversion?
  • How do we order removing a package which had a diversion?
  • How do we handle errors?
  • What happens to dpkg-divert?


Details - Control File Syntax


It will conform to RFC2822 style with the following format:

  • Divert-From:
  • Divert-To:
  • Blank lines and lines beginning with '#' will be comments

'Divert-To' will be optional and if it is ommitted then files being diverted will have their filename changed to 'file.distrib'

The above style has it's advantages, one of the main ones being that there is no need to escape whitespace within filenames. It also allows for more commands to be added at a later date.


Details - Control File Handling


Within control.tar.gz the file should be named 'diversions' This file is then copied to /var/lib/info/$package.diversions We need to store the control file for the currently-installed package so that we can detect declarative diversions that have been removed from the control file between versions (On-disk version and the to-be-unpacked version).

If there is no previous version of a control file then all current diversions within the package are left in place and any new ones declared within the control file are added. If there is a previous version of the control file then the diversions listed within the file are compared with the diversions currently in place. If a diversion exists that is no longer listed in the control file then it is removed. If there is a diversion listed in the control file that doesn't currently exist then it is added.


Details - Handling Diversions to non-existant directories


Diverting files to directories that don't exist can cause a number of problems. If the package does not 'own' the directory it may be left orphaned on removal of the package The package is responsible for ensuring the availability of the target directory in the unpack phase.

If a declarative diversion attempts to divert a file to a folder that neither exists already on the filesystem at unpack time, nor is shipped within the package, dpkg will return an error at unpack time.


Details - Ordering Requirements


Unpacking a new package that adds a diversion

  • 1. Check if package has a previous diversion control file
  • 2. Copy control file to tmp directory
  • 3. Calculate diversions to be added
  • 4. Unpack folders
  • 5. Add diversions
  • 6. Unpack files
  • 7. Copy control file to /var/lib/dpkg/info/$package.diversion
  • Performing diversions this way avoids trouble with creating folders and
    • accidentally overwriting the wrong version of files.
  • Another major problem would be extracting a new version of the file and
    • overwriting the old one.
  • This might cause the wrong version of the file to be diverted and the
    • package to break.

Unpacking a new package that removes a diversion

  • 1. Check if package has a previous diversion control file
  • 2. Copy control file to tmp directory
  • 3. Calculate diversions to be removed
  • 4. Check if dropped diversion results in a file conflict
  • 5. Unpack folders
  • 6. Unpack files
  • 7. Remove old files no longer needed in the package
  • 8. Remove diversions
  • 9. Copy control file to /var/lib/dpkg/info/$package.diversion
  • If a package has no previous diversion control file then no diversions
    • will be removed and existing diversions can only be removed using dpkg-divert.
  • If removing a diversion results in a file conflict then dpkg will return
    • an error at unpack time but before anything is unpacked. If an error is returned the unpack is aborted


Removing a package which had a diversion

  • 1. Remove files
  • 2. Remove diversions
  • 3. Remove folders
  • 4. Remove control file at /var/lib/dpkg/info/$package.diversion
  • This ensures that all files, diversions and folders are removed correctly


Details - Error handling


Errors in diversions will have to handled with a great deal of care due to the fact that if they are not the package could be broken. This means that a great deal of checks must be done to ensure that all the files can be diverted properly before any actual diverting takes place. If they can't the package installation/update must be stopped and rolled back to avoid the package being installed incorrectly or broken.


Details - 'dpkg-divert'


When we impliment the new diversion method we should keep the current dpkg-divert. This allows maintainers to catch up with the new method without breaking their packages. It also allows maintainers to perform some operations that aren't support by the new method. All current functionality of dpkg-divert will be left intact for use by sysadmins and for maintainers to catch up with the new system. The two systems will work together without directly conflicting with each other.


Example Usage #1


The file to be diverted is '/usr/share/foo'

It needs to be moved to '/usr/share/bar'

The syntax of the control file would be:


#Start File
Divert-From: /usr/share/foo
Divert-To: /usr/share/bar
#End File



Example Usage #2


In this example the maintainer doesn't want to move the file to any specific folder

The syntax of the control file would be:


#Start File
Divert-From: /usr/share/foo
#End File


This would divert the file to '/usr/share/foo.distrib'


Footnotes


RFC2822 Guide:

First Email Thread on Declarative Diversions (First Message in Thread):

Email thread on first design draft (First message in Thread):

Declarative Diversions Wiki:

My Blog for this Project: