Integrating Emdebian Grip into Debian

Emdebian has long described itself as an "official sub project of Debian" but this status has not actually been granted by Debian yet. The discrepancy recently led to discussions with the Debian Project Leader and on the debian-embedded mailing list about integrating Emdebian into Debian so that the status of Emdebian can be decided properly.

It rapidly became clear that Emdebian could integrate the Emdebian Grip flavour more closely with Debian and that such integration was desirable both within Emdebian and within Debian.

At DebConf11, discussions took place with representatives of the ftp team, wanna-buld team and release team around the mechanisms by which Emdebian Grip could be integrated directly into the main Debian archive and Debian release process.

Currently, the proposal is for the following changes:

  1. Emdebian would run a buildd which would run the emgrip script instead of using schroot. This buildd would receive all packages moving through the normal Debian buildd system, including source uploads, other buildd uploads, binary NMU's and security updates. Alternatively, the buildd could receive notifications via the ftp process-uploads action which would make it easier for the Emdebian buildd to work on binary packages from 6 or more architectures as wanna-build expects buildd's to care about source packages and a single architecture.
  2. The buildd would control the list of packages in which Emdebian is interested and this list would be modifiable via bug reports against buildd.emdebian.org. The list itself may end up in the wanna-build postgresql database for the Emdebian Grip buildd or some other mechanism.
  3. The buildd would use some form of dependency resolution to bring in extra packages when an existing package changes dependencies. (This is necessary for the release team migration process to work correctly and does not need to happen immediately, as long as the dependencies are resolved before the package tries to migrate. The expectation is that the package list would be altered automatically in time for the next buildd operation. However, this may involve some level of "back-tracking" to bring in a package which was last built for Debian some period previously.)
  4. The ftp team have agreed to take care of removing old dependencies or packages removed from Debian using the same mechanisms as are used for the current Debian archive.
  5. The ftp team have also agreed to push the Emdebian Grip packages to all existing primary Debian mirrors which do not explicitly ask not to take them. Currently, this is about 25Gb of data.
  6. The release team have agreed to work with the Emdebian team to synchronise all releases, including stable updates. The Emdebian Grip processes would need to ensure that the release process is not delayed by missing packages or other discrepancies.
  7. The release team have agreed to run the testing migrations of Emdebian Grip packages alongside their Debian counterparts, possibly using a second britney process.
  8. Emdebian has agreed to drop the dev, debug, java and doc components from Grip unstable and Wheezy in order to make this process work - all packages will revert to the "main" component and Emdebian would gain support for contrib and non-free (although no packages currently exist for those components in Emdebian Grip).
  9. The ftp team require that the Emdebian packages are separated from the Debian packages by two mechanisms, the current em1 version suffix and a distinctive suite name. Emdebian is free to choose how to make suite names (and therefore codenames) distinctive and in order to preserve the option of doing something in the future with Emdebian Crush, the suggestion at this stage is something like unstable-grip or grip-unstable, alongside sid-grip or grip-sid respectively.
  10. Still to be decided is how to manage existing packages but the suggestion from Emdebian would be that the ftp team synchronise the new suites with the existing Emdebian packages for the existing releases (Lenny and Squeeze) and that www.emdebian.org continues to provide those releases under the existing suites. Similarly, the existing packages on ftp.uk.debian.org/emdebian/grip will be retained for Lenny and Squeeze but will eventually migrate to being symlinks to the new suite names in ftp.uk.debian.org/debian in time for the Wheezy release and ongoing for unstable. Packages in the new suites for Lenny and Squeeze would only use the "main" component, with the old components remaining available via the current Emdebian Grip repository and mirror as symlinks. i.e. Emdebian will retain the links only until Wheezy becomes old-stable.
  11. At this stage, Emdebian does not expect to need support for experimental.
  12. stable-proposed-updates and testing-proposed-updates could work exactly as in Debian, instead of the previous best-effort support available via the Emdebian team.
  13. The list of architectures supported by Emdebian Grip and the names of those architectures do not change.
  14. Unofficial ports cannot be handled via this integrated method and the Emdebian Grip support for armhf, sh4 and powerpcspe will continue to be processed using the existing scripts from the emdebian-grip-server package but this package would then drop the buggy support for testing migrations. If an unofficial port is adopted by Debian, that architecture will migrate to using the wanna-build method for Emdebian Grip under the control of the ftp team and release team.
  15. packages.debian.org and packages.qa.debian.org will be able to list the new suites and the versions available within those suites as well as a comparison of the file list and package sizes.
  16. Emdebian will continue to forcibly truncate the long description of packages as part of the emgrip process, subject to other changes in Debian. Other processing of Emdebian Grip packages will remain, including the removal of manpages, translations and documentation, the compression of the copyright file and removal of changelogs. The fate of Emdebian TDebs is undecided as these are not compatible with Debian but Debian TDeb support is in development.
  17. emgrip itself will be modified to support setting the new suite names in the changelog entry which adds the em1 suffix. This will require support for the new suite names in the debchange script from the devscripts package.
  18. Existing lintian profile support can be used to check the packages.
  19. Documentation will be required for maintainers who may not currently know that their package has already been released as part of Emdebian Grip 1.0 (Lenny) and Emdebian Grip 2.0 (Squeeze). This documentation will need to cover the changes made within the package and how to deal with bug reports which refer to the Emdebian Grip version. (Reportbug already has support for directing bug reports which claim a Version: with the em1 suffix to the buildd.emdebian.org pseudo-package in the BTS but this has not been extensively tested.) Maintainers need to be clear that the Emdebian Grip package is binary compatible with the Debian version of the same package without the suffix. Emdebian has not come across packages which change behaviour as a result of the conversion to an Emdebian Grip package.
  20. Maintainers also need to be clear that Emdebian has two main flavours and these changes only affect the binary-compatible Grip flavour currently. Just describing these packages as "Emdebian" may lead to confusion in bug reports. This is why the proposed suite names include "grip" rather than "emdebian".

  21. For Emdebian, once development of Emdebian Crush restarts after the MultiArch transition, consideration will need to be given to how best to handle the necessary changes. Emdebian Crush development is currently stalled but will involve functional changes in selected packages which break binary compatibility with Debian and Emdebian Grip in order to achieve reductions in the dependency chains of those packages. e.g. Crush will support dropping LDAP dependencies from packages like curl and similar changes to cairo, openssh and dpkg as well as a fully functional rebuild of busybox to replace coreutils. Other packages which do not require changes will be drawn from Emdebian Grip. It is currently proposed that Emdebian Crush adopts new source package names and new binary package names for the modified packages and that all consequent problems of dependency resolution and meta packages is left entirely to Emdebian Crush to sort out and must not affect Emdebian Grip or Debian at this stage.


CategoryEmdebian