Debian Grip script processing details

Database queries

ProjectB provides a table describing the changes in the archive along with a timestamp. From this table, emdebian-blavet.pl retrieves the changes since the last run and organises the data:

  1. suite - primary key, default is unstable which maps to unstable-grip
  2. source - secondary key, the source package name for these changes, according to projectb
  3. architecture - tertiary key. For the purposes of these scripts, source and all are distinct architectures. source is always handled first but the lack of source changes does not prevent uploads of binary packages from that source. This covers the multi-architecture approach by allowing individual architecture uploads to happen consecutively as the buildds complete their tasks.

  4. package - a list of binary package names which other buildds would build from this source package and which need processing for Emdebian Grip.
  5. version - a list of binary versions (some source packages build binaries with versions which differ from each other and from the source package itself).
  6. The name of the dscfile - obtained from an rsync'd Packages.gz file from ftp.debian.org

  7. The path to the named dscfile (i.e. the directory in which all the specified packages and the dscfile itself can be found).

Build operations

The data from projectb is passed to the build subroutines, one source package at a time. emdebian-buildd.pl prepares the Emdebian versions of the packages:

  1. Downloads the source package, checking the hashes against the Sources.gz file from the rsync'd data. If a source upload is required, uses dscextract to get the debian/control and debian/changelog files and prepares a source.changes file which is then uploaded using dput as a source-only upload. The original .dsc signature is not changed (because the source version has not, cannot, change) and the source.changes is signed with the Emdebian buildd signing key.

  2. Downloads each binary package for this source package, checking hashes against the Packages file from the rsync'd data, one architecture at a time. For these purposes, all is a distinct architecture and is uploaded as a $HOSTARCH.changes file. Architecture dependent packages for the host architecture are uploaded separately.

  3. Uses emgrip to prepare the Emdebian Grip version of the binary packages.

  4. Uses dpkg-architecture -a$ARCH -c to wrap the call to dpkg-genchanges -b along with data from dscextract to prepare a $ARCH.changes file which is then uploaded. ($ARCH = $HOSTARCH if $ARCH is all, so uploads from $HOSTARCH includes the architecture independent and architecture dependent binaries for that package).

Automatic package additions and removals

Packages are only added or removed from unstable-grip.

emdebian-depends.pl is now being run in the crontab on blavet.debian.org to use [edos-debcheck] to identify packages which need to be added to Emdebian Grip to satisfy dependencies of updated packages. When new package are added, the source upload made by emdebian-depends.pl uses the -sa option to dpkg-genchanges as the newly introduced package needs the complete source uploaded to the Emdebian suite.

emdebian-sync.pl is also run in the crontab to catch missed uploads or retry failed uploads.

Manual interventions

Scripts now use [edos-debcheck] to handle situations where the updated package has gained a dependency on a binary package which is not yet included in Emdebian Grip. In such situations, packages will need to be added directly to testing-grip as the package itself may already have migrated into testing a long time before becoming of interest to Emdebian.

Adding extra packages

Packages which are not already a dependency of a package currently in Emdebian Grip are not added to Emdebian Grip without a bug report against buildd.emdebian.org and discussion on that bug about why the package should be in Emdebian Grip and what role it serves for embedded devices.

Criteria for adding new packages include but are not limited to:

  1. Packages which are already in testing
  2. Stable API
  3. Clear usage for embedded devices
  4. Lack of unwarranted dependencies (e.g. most meta-packages do not qualify.)
  5. Presence of a useful tasksel configuration available in Debian Installer.

i.e. Emdebian will refuse to add packages which just make more work than they are worth - as was the case with enlightenment packages for Wheezy.

Removing packages

Once added to Emdebian Grip, packages are only removed from Emdebian Grip for the following reasons:

  1. Package is removed from the equivalent Debian suite by ftp-master
  2. A specific request is made to remove the package via a bug report against buildd.emdebian.org, with reasons and all reverse dependencies (including build dependencies where appropriate) have already been removed.


CategoryEmdebian