Differences between revisions 3 and 4
Revision 3 as of 2006-05-16 05:01:20
Size: 2195
Editor: wookey
Comment:
Revision 4 as of 2006-10-31 13:28:35
Size: 6595
Editor: NeilWilliams
Comment: Reformat and include overview of LinuxWorld Expo discussions
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
The biggest remaining issue in making Embedded Debian is how to store and control package metadata which describes how to reduce package size, increase granularity, reduce the size of the essential package set, etc. = EmdebianMetaData =

The biggest remaining issue in making Embedded Debian is how to store and control
package metadata which describes how to reduce package size, increase granularity,
reduce the size of the essential package set, etc.
Line 5: Line 9:
$DEBIAN_DIR The issues were discussed with various Debian Developers at LinuxWorld Expo 2006.

== Composite method. ==

=== Principles: ===

 1 Use of an appended emdebian version string with the 'em' prefix and a sequential digit as well as a distinct file suffix: .mdeb

 Example: if epoch = 1, upstream version = 2.3.4, debian version = 5, NMU number = 6, emdebian version 7, built for emdebian on arm:
{{{ foo_1:2.3.4-5.6em7_arm.mdeb
foo_1:2.3.4-5.6em7.diff.gz
foo_1:2.3.4-5.6em7.dsc
foo_1:2.3.4-5.6em7_arm.changes
}}}

 In common with Debian versioning itself, the emdebian version resets to 1 each time a new package is available. Emdebian considers Debian as upstream so a new Debian version is still a new package as far as emdebian is concerned.

 This achieves three distinct ends.
 a. Separates the emdebian archive files from other Debian archives and associated files being built from the same source tree on the developer system.
 a. Identifies the archive as an adapted, stripped out, archive that is '''not necessarily''' compatible with any other .deb or .udeb files or programs.
 a. Allows emdebian developers to have a version number that is independent of the *full* Debian version string.

'''Status''': Mostly already achievable with normal Debian tools - except that the final archive needs to be renamed to use the .mdeb suffix at the final stage.

'''Problems''': May require small changes to some .deb handling programs like reprepro. There has been a request to separate the emdebian diff.gz contents from the Debian .diff.gz contents - treating Debian as upstream of emdebian. This needs tweaking in the build scripts, possibly via the wrapper script below. '''Note''' that building such a package from source would then need both .diff.gz files and this could require further changes to the debhelper scripts, e.g. in the generation of the .changes file. In some (most?) situations, the emdebian .diff.gz will be empty - most changes being implemented by the build scripts.

 1.#2 Wherever possible, emdebian will use standard Debian packaging scripts - only tweaking those related to the content needing to be removed or ignored in emdebian packages. i.e. the burden of changes has moved from Debian developers to emdebian developers. Packages like cross-get and dpkg-cross can be further enhanced to ensure the correct versions of the debhelper tools are used. Overall, this leads to changes in less files than using the $DEBIAN_DIR approach, as well as less duplication.

 1. Those packages that need further customisation can have patches stored in the emdebian SVN repository. An emdebian build script - a wrapper for pdebuild - will checkout and apply the patches *before* debian/rules is called. This is vital because using existing patch methods like CDBS and dpatch relies on debian/rules being called to perform the patch. In order to successfully patch debian/rules itself, the patch process needs to begin before debian/rules is called. The repository needs to hold patches on a per-source basis.

'''Status''': cross-get can become apt-cross with normal apt functionality so that it can support a wrapper for pdebuild (empbuild) that downloads, cross-builds and installs dependencies in the chroot when building emdebian packages. This is in current development (Nov 2006).

 1. Unlike Debian packages, emdebian packages will not have individual emdebian maintainers. Instead, any emdebian developer will be able to update the emdebian customisations for a particular Debian package, e.g. when a new version is uploaded to unstable.

 1. An emdebian buildd can then package and upload archives to the emdebian repository to remove the delay between packages reaching testing and being built for emdebian.

Overall, this method uses the best of each of the previous ideas but large amounts of work remain.

One necessary precursor for this approach is some form of emdebian BTS. Repackaging Debian archives through automated systems can cause mysterious bugs.

== $DEBIAN_DIR ==
Line 7: Line 52:
 * Needs updated tools (dpkg, debhelper, etc)
'''con''':
 * Needs updated tools (dpkg, debhelper, etc) This is a show-stopper. Too many packages will need changing and although the change appears minor, at LinuxWorld Expo 2006, discussions showed that $DEBIAN_DIR is unlikely to be supported by maintainers of the existing Debian package maintainers.
 * Repetition of info in two or more places
 * Generates packages with same names as standard packages but different files/functionality/interface - will cause problems if mixed with Debian proper, or with different distros using the same mechanism.
 * Debian packages will install, but may not work due to missing files.
 * Easy for package maintainers to ignore so will get bitrot

'''pro''':
Line 13: Line 66:
 * Repetition of info in two or more places
 * Easy for package maintainers to ignore so will get bitrot
 * Generates packages with same names as standard packages but different files/functionality/interface - will cause problems if mixed with Debian proper, or with different distros using the same mechanism.
 * Debian packages will install, but may not work due to missing files.
== udebs ==
Line 18: Line 68:
udebs
Line 20: Line 69:

'''con''':
 * Harder to make specific variants due to close integration with Debian proper / no overlay mechanism
 * Changed package names require different sets of packages for 'embedded' base system.

pro:
 * Easy to get changes into standard packages - mechanism is accepted, separation is clear
Line 21: Line 77:
 * Easy to get changes into standard packages - mechanism is accepted, separation is clear
Line 23: Line 78:
   * Changed package names require different sets of packages for 'embedded' base system.
Line 27: Line 80:
 * Harder to make specific variants due to close integration with Debian proper/ no overlay mechanism
Line 29: Line 81:
Just change /debian files
 <explanation/example> - used by SLIND
 In fact this is really a variant of the $DEBIAN_DIR example and could be implmented that way too (this assertion needs testing)
 * Simple - easy to understand
 * Standard tools will work
== Just change /debian files ==
Line 35: Line 83:
'''con''':
Line 38: Line 87:
'''pro''':

<explanation/example> - used by SLIND

In fact this is really a variant of the $DEBIAN_DIR example and could be implmented that way too(this assertion needs testing)

 * Simple - easy to understand
 * Standard tools will work

EmdebianMetaData

The biggest remaining issue in making Embedded Debian is how to store and control package metadata which describes how to reduce package size, increase granularity, reduce the size of the essential package set, etc.

This page tries to document the issues and the pros and cons of the current active proposals.

The issues were discussed with various Debian Developers at ?LinuxWorld Expo 2006.

Composite method.

Principles:

  • 1 Use of an appended emdebian version string with the 'em' prefix and a sequential digit as well as a distinct file suffix: .mdeb Example: if epoch = 1, upstream version = 2.3.4, debian version = 5, NMU number = 6, emdebian version 7, built for emdebian on arm:

{{{ foo_1:2.3.4-5.6em7_arm.mdeb foo_1:2.3.4-5.6em7.diff.gz foo_1:2.3.4-5.6em7.dsc foo_1:2.3.4-5.6em7_arm.changes }}}

  • In common with Debian versioning itself, the emdebian version resets to 1 each time a new package is available. Emdebian considers Debian as upstream so a new Debian version is still a new package as far as emdebian is concerned. This achieves three distinct ends.
  • Separates the emdebian archive files from other Debian archives and associated files being built from the same source tree on the developer system.
  • Identifies the archive as an adapted, stripped out, archive that is not necessarily compatible with any other .deb or .udeb files or programs.

  • Allows emdebian developers to have a version number that is independent of the *full* Debian version string.

Status: Mostly already achievable with normal Debian tools - except that the final archive needs to be renamed to use the .mdeb suffix at the final stage.

Problems: May require small changes to some .deb handling programs like reprepro. There has been a request to separate the emdebian diff.gz contents from the Debian .diff.gz contents - treating Debian as upstream of emdebian. This needs tweaking in the build scripts, possibly via the wrapper script below. Note that building such a package from source would then need both .diff.gz files and this could require further changes to the debhelper scripts, e.g. in the generation of the .changes file. In some (most?) situations, the emdebian .diff.gz will be empty - most changes being implemented by the build scripts.

  1. Wherever possible, emdebian will use standard Debian packaging scripts - only tweaking those related to the content needing to be removed or ignored in emdebian packages. i.e. the burden of changes has moved from Debian developers to emdebian developers. Packages like cross-get and dpkg-cross can be further enhanced to ensure the correct versions of the debhelper tools are used. Overall, this leads to changes in less files than using the $DEBIAN_DIR approach, as well as less duplication.
  2. Those packages that need further customisation can have patches stored in the emdebian SVN repository. An emdebian build script - a wrapper for pdebuild - will checkout and apply the patches *before* debian/rules is called. This is vital because using existing patch methods like CDBS and dpatch relies on debian/rules being called to perform the patch. In order to successfully patch debian/rules itself, the patch process needs to begin before debian/rules is called. The repository needs to hold patches on a per-source basis.

Status: cross-get can become apt-cross with normal apt functionality so that it can support a wrapper for pdebuild (empbuild) that downloads, cross-builds and installs dependencies in the chroot when building emdebian packages. This is in current development (Nov 2006).

  1. Unlike Debian packages, emdebian packages will not have individual emdebian maintainers. Instead, any emdebian developer will be able to update the emdebian customisations for a particular Debian package, e.g. when a new version is uploaded to unstable.
  2. An emdebian buildd can then package and upload archives to the emdebian repository to remove the delay between packages reaching testing and being built for emdebian.

Overall, this method uses the best of each of the previous ideas but large amounts of work remain.

One necessary precursor for this approach is some form of emdebian BTS. Repackaging Debian archives through automated systems can cause mysterious bugs.

$DEBIAN_DIR

<explanation/example> - used by STAG and STAGE

con:

  • Needs updated tools (dpkg, debhelper, etc) This is a show-stopper. Too many packages will need changing and although the change appears minor, at ?LinuxWorld Expo 2006, discussions showed that $DEBIAN_DIR is unlikely to be supported by maintainers of the existing Debian package maintainers.

  • Repetition of info in two or more places
  • Generates packages with same names as standard packages but different files/functionality/interface - will cause problems if mixed with Debian proper, or with different distros using the same mechanism.
  • Debian packages will install, but may not work due to missing files.
  • Easy for package maintainers to ignore so will get bitrot

pro:

  • Flexible and hierarchical - easy to apply a set of local changes
  • Easy to build entirely standard packages when $DEBIAN_DIR is set to debian
  • Changes are cleanly separated from standard package
  • Quite easy to understand

udebs

<explanation/example> - used by Debian-installer

con:

  • Harder to make specific variants due to close integration with Debian proper / no overlay mechanism
  • Changed package names require different sets of packages for 'embedded' base system.

pro:

  • Easy to get changes into standard packages - mechanism is accepted, separation is clear
  • Ensures that changed packages have changed names so no confusion over functionality
  • Good acceptance in Debian proper already due to Debian-Installer team work
  • Debian packages will not install due to wrong deps
  • Needs changed tools to understand it (dpkg, debhelper etc) - such tools/changes are already in Debian

Just change /debian files

con:

  • Very difficult to push changes back into standard packages - most are not appropriate
  • Large maintenance effort (due to no mechanism for pushing back into standard Debian packages

pro:

<explanation/example> - used by SLIND

In fact this is really a variant of the $DEBIAN_DIR example and could be implmented that way too(this assertion needs testing)

  • Simple - easy to understand
  • Standard tools will work


CategoryEmdebian