Differences between revisions 1 and 74 (spanning 73 versions)
Revision 1 as of 2005-11-19 02:43:49
Size: 1840
Editor: wookey
Comment: Moved embedian content to page with correctly spelled title
Revision 74 as of 2011-02-23 12:32:20
Size: 13265
Editor: NeilWilliams
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
Adapted Debian tools so you can build/cross-compile DebianPackage s or
adapted packages with an emdebian dir.

As it seems at the moment the ["Scratchbox"] environment seems to be a
primary target to try out these things. With the advantage that we can
ignore some cross-compilation issues, and that the tools are easily
usable natively too.
#language en

=== Emdebian vision ===
In the Emdebian vision someone wishing to build a GNU / Linux based device would:

 1. Port the linux kernel to their hardware (including writing any specific device drivers).
 1. Select the '''''prebuilt''''' emdebian packages needed to support their application.
 1. Package their application as Debian package(s) using Debian and Emdebian tools.
 1. Build a root filesystem using Emdebian tools from the steps above.

Emdebian is involved in steps 2,3,4 above (there are far too many embedded device hardware variations to make prebuilt kernels practical).

Thus EmDebian is a '''''binary''''' distribution for embedded devices (whereas most of the other contenders in this space are '''''source''''' distributions [of course being Debian and open source the source code is still available if required].

=== What emdebian does ===
'''''In short, what EmDebian does is wrap around the regular debian package building tools to provide a more fine grained control over package selection, size, dependencies and content to enable creation of very small and efficient debian packages for use on naturally resource limited embedded targets.'''''

We adapt Debian tools so you can build/cross-compile Debian packages or adapted packages with info on how to cross-build and build smaller packages.

We currently have two separate distributions - a post-processed binary-compatible distribution called [[http://www.emdebian.org/grip/|Emdebian Grip]] for intermediate installations and a much smaller, much more involved cross-built distribution called [[http://www.emdebian.org/crush/|Emdebian Crush]]. Each distribution can be further optimised for systems that would benefit from a static configuration by removing the maintainer scripts - this flavour is called [[http://www.emdebian.org/baked/|Emdebian Baked]].

Emdebian is experimenting with different approaches to cross-building and small-system-friendly packages in order to produce sustainable and robust long-term solutions. For cross-building there are two main approaches: [[http://www.scratchbox.org/|Scratchbox]] ([[DebPkg:scratchbox2|pkg]]), which allows cross-builds to appear to be native to the apps being built, and dpkg-cross ([[DebPkg:dpkg-cross|pkg]]) which allows conventional cross-builds to occur in a Debian-compatible way using emdebian-tools ([[DebPkg:emdebian-tools|pkg]]).

For package meta-data making packages more suitable for shrinking, two approaches are currently being investigated. The STAG approach uses a $(DEBIAN_DIR) directory to overlay changed info from the /debian directory, and is used by STAGE. The udeb approach of Debian-installer gives a tighter integration with Debian, with embedded packages having changed names so they can exist in the debian package namespace. Both mechanisms have pros and cons which are discussed in EmdebianMetaData.

New packages can be added to Grip using the [[DebPkg:emdebian-grip-server|emdebian-grip-server]] package. Adding new packages or new architectures to Crush is currently stalled until the EmdebianCodeAudit can be completed in order to resolve issues around packages with modified functionality.

== Distribution status ==

Crush 2.0 is currently stalled due to build system problems - see EmdebianCrush for details of why, how this could be fixed and current experiments. If things stay as they are, there will be no release of Crush 2.0 (based on Debian 6.0 "squeeze"), instead development will target Crush 3.0 (based on Debian 7.0 - wheezy), reliant upon delivery of an implementation of Multiarch with sufficient support for cross-building.

Grip development is continuing. Grip 2.0 has been released alongside Debian 6.0 "squeeze" on six architectures (armel, mips, mipsel, i386, amd64, powerpc) and with some 2,000 packages. Work is ongoing alongside Debian for Emdebian Grip 3.0 based on Debian GNU/Linux 7.0 Wheezy.

Baked is as ready as it can be - Baked is primarily a do-it-yourself distribution but some starter packages are available.

== Project Website ==
http://www.emdebian.org/

August 17th 2006: The server has moved to a new machine in Europe instead of the US.

== Wiki SiteMap ==
[[EmdebianWebLayout|Emdebian Web Layout]]

== Frequently Asked Questions (FAQ) ==
[[EmbeddedDebianFAQ]]

== Quick Start Guide ==
EmdebianQuickStart

== Tracker ==
EmdebianTracker - quick notes about issues that need a fix within the Emdebian packages (issues unrelated to cross building). e.g. a postinst script that calls a binary that is not available or with an unknown option, conflicts between packages that only show up when Essential is ignored, extra dependencies needed when Essential is ignored, etc.)

== Code Audit ==
EmdebianCodeAudit - coordinating a complete audit of all patches required for Emdebian Crush 1.0 to support the implementation of the patches within the relevant Debian packages.

== Policy ==
EmdebianPolicy

== Example workflows ==
EmdebianExamples

== Wiki pages in continuous devel ==
CategoryEmdebian

== Hardware ==
 * [[/Smartphone]] - Debian in [[http://en.wikipedia.org/wiki/Smartphone|smart phones]].
 * PDA - [[http://en.wikipedia.org/wiki/Personal_digital_assistant|Debian in Personal digital assistant]].
 * Domotics - [[http://en.wikipedia.org/wiki/X10_(industry_standard)|Debian in X10 and other domotic standards]].
 * DebianRobot .
== Cross Toolchains ==
Emdebian builds [[EmdebianToolchain|cross-toolchains]] for use on standard Debian development machines. [[http://www.emdebian.org/crosstools.html|Repository details]]. [[EtchCrossToolchains|Toolchains for Etch]].

== Documentation ==
EmdebianPackagingGuidelines is a document explaining to package maintainers the sorts of things they should be aware of in their packaging to make their packages 'Embedded-friendly'. This includes cross-building and small-system-building.

[[Embedded_Debian_packaging_infrastructure|Embedded Debian Packaging Infrastructure]] explains how the Emdebian repositories are organised, where the repositories are located and how to setup your system for emdebian uploads.

[[Embedded_Debian_packaging_rules|Emdebian packaging rules]] details how emdebian packages should be built and organised.

== How Grip relates to Crush ==

Building packages for Crush (for the EmdebianCodeAudit) now bases the final package on the Grip processing. i.e. the package is cross-built for Crush with limited changes and then the cross-built package is run through the Grip scripts to remove documentation, examples, handle Emdebian TDebs etc. This reduces the number of changes that need to be made in the patches for Crush, simplifying the patches and the build process. As the Audit proceeds, the patches necessary to allow the packages to be cross-built at all will be fed back to Debian via the [[http://bugs.debian.org/|BTS]] and included into the packages via maintainer uploads or, if necessary, with NMU's by Emdebian developers. The aim is to reduce the number of patches to only those necessary for the functional changes needed for Crush:

 * Avoid calling perl scripts that cannot be replaced by busybox
 * Avoid using perl in maintainer scripts
 * Reduce long dependency chains by turning off optional build components
 * Make maintainer scripts compatible with the busybox shell

This should allow many Debian packages to be cross-built without special tools and without external patches, with compliance to EmdebianPolicy being achieved '''after''' the cross-build is complete.

== Multiarch and cross-compile ==

The goal is that both native and cross-compile behave the same, use files in the same location and use the same compiler flags. This is comparable to setting sysroot=/ now.

MultiArchExperiments

=== Current situation: all -*-cross debs, no multiarch ===

||includes:||/usr/triplet/include ||
||libraries:||/usr/triplet/lib ||
||pkg-config:||/usr/triplet/lib/pkg-config ||

=== Soon: all -*-cross debs, some multiarch -*-cross debs ===

||includes:||/usr/triplet/include ||
||libraries:||/usr/triplet/lib ||
||pkg-config:||/usr/triplet/lib/pkg-config ||

Since multiarch libfoo for foreign archs can't be installed with current dpkg they must still be converted to -*-cross packages by dpkg-cross. That involves moving files out of the multiarch directories into the old cross compile dirs and adjusting .pc and .la files.

=== Multiarch transition: some -*-cross debs, some -*-cross dummy debs, some multiarch debs ===

||includes:||/usr/include + /usr/include/triplet + /usr/triplet/include ||
||libraries:||/lib/triplet + /usr/lib/triplet + /usr/triplet/lib ||
||pkg-config:||/usr/lib/triplet/pkg-config + /usr/triplet/lib/pkg-config ||

This starts when dpkg can install multiarch libfoo for foreign archs. Dpkg-cross will convert multiarch libfoo to a dummy libfoo-arch-cross that depends on libfoo:arch (= ver) and is completly empty. The dummy libfoo-arch-cross has 2 jobs: Pull in libfoo:arch on upgrade and satisfy reverse dependencies. Over time the reverse dependencies will change to use libfoo:arch directly.

For gcc you have this behaviour already with sysroot=/. But libtool, pkg-config and probably some more will need to be adjusted to this.

=== Long term goal: no -*-cross debs, all multiarch ===

||includes:||/usr/include + /usr/include/triplet ||
||libraries:||/lib/triplet + /usr/lib/triplet ||
||pkg-config:||/usr/lib/triplet/pkg-config ||
Line 10: Line 129:

As Ed seems to have a working environment I guess we should start
working with that and improve it on the go.

Seems you got the "build base system" item finished Ed. Thanks!

And the second item "ability to build a base ["rootfs"] too!

One good thing would that he did not mention in his mail was adding
support for ignoring packaging documentation and man-pages in the
packages. A harder thing to do is actually making sure that the docs
don't even get build. And a way to avoid the circular dependencies that
exist due to this documentation building tools.

Set-up a buildd inside scratchbox (which should be possible as I heard
that it has already been done) to start building more packages. (This
will hopefully be possible when the emdebian server is back up). (Once I
find out how to silence my proliant server I might hook it up to my
internet line and provide a secondary build env)

Document setting up the buildd so ["people"]/ ["dev"]s can try it out for
themselves.

Integrate ["uclibc"] in the story. This will definitely give some size boost
toward smaller footprint to the packages.

Once we have proven that this works start pushing things into Debian
proper.

Optional :
-Make sure everything works with the successors of scratchbox\
-Check the ARM build of Ubuntu

Well I hope I can open a fruitfull discussion with this and hope to see
some more people starting to work on this.

See also: BootUsb , DebianRobot .
Things that need doing are:

 * Emdebianising more packages for Crush so that they
  a. cross-build correctly
  a. split packages where the default has too many dependencies
 * Work on Emdebian policy and actual packages to:
  * Speed up boot time
  * Reduce dependencies
  * Differentiate between install time and run-time dependencies
  * Differentiate between build and target build-dependencies
  * Shrink minimum rootfs sizes
   * Busybox-based rootfs as well as GNU-based EmdebianRootfs
   * Dash instead of bash
   * Remove perl from essential
 * Work on Debian and emdebian packages to:
  * Support changes due to above policy
  * Push changes back into Debian as much as possible (e.g faster boot)
 * Infrastructure Work:
  * Completing auto-build mechansism that will keep emdebian cross-toolchains uptodate with debian toolchain versions
  * Incorporate full 3-stage bootstrap changes from slind
  * Auto-build emdebian packages, tracking debian versions and including support for developers to track packages which have build failures
 * Document:
  * Making Emdebian systems
  * Emdebianising new packages
  * The infrastructure
  * Developed Policy
 * create configurations for various devices and document howto do it
 * promote DEB_BUILD_OPTIONS=nodocs flag for not building docs
 * promote DEB_BUILD_OPTIONS=notest or DEB_BUILD_OPTIONS=nocheck for skipping tests
 * co-operate with i386-uclibc team and embedded arch porters
 * Kill scratchbox in its current form and make a better scratchbox2
Set-up a buildd inside scratchbox

Document setting up the buildd so developers and non-developers can try it out for themselves.

Integrate [[uclibc]] in the story. This will definitely give some size boost toward smaller footprint to the packages.

Once we have proven that this works start pushing things into Debian proper.

Optional :

 * Make sure everything works with the successors of scratchbox
 * Check the ARM build of Ubuntu
See also: EmDebian/DeBootstrap and EmDebian/CrossDebootstrap .

== Wacky Ideas ==
There are also a few ideas that may need to be investigated further to decide whether it is a good idea to make them a new TODO item. These are collected on the EmdebianWackyIdeas page.

== Donations / Donors ==
[[EmdebianDonations|Emdebian Donations and Donors]]

== Meetings, Conferences, Work Sessions ==

There are IRC meetings roughly every two weeks, for details please see [[Emdebian/Meetings]]

==== 2009 ====
February 7th-8th in Brussels, Belgium [[http://fosdem.org/2009/|Fosdem2009]]

==== 2008 ====
September, Extremadura Work Meeting EmdebianWorkSessionExtremadura2008

June, [[http://www.toby-churchill.com/|TCL]] work session

February 23rd-24th in Brussels, Belgium [[http://fosdem.org/2008/]]

==== 2007 ====
June 17-23rd, in Edinburgh, Scotland. [[https://debconf7.debconf.org/wiki/Main_Page|DebConf7]]

February 24&25th, in Brussels, Belgium. [[http://www.fosdem.org/2007/|Fosdem2007]]

==== 2006 ====
October 25&26th, in London, UK. [[http://www.linuxworldexpo.co.uk/|London Linux World Expo]]

May 14th - May 21st, in Mexico. DebConf6EmdebianStuff

April 12-16th, in Extremadura, Spain. DebianEmbeddedWorkSessionExtremadura2006

February 25&26th, in Brussels, Belgium. [[http://wiki.debian.org/EmDebian/Fosdem2006|Fosdem]] [[http://www.fosdem.org/2006/|Fosdem2006]]

== External links ==
 * http://en.wikipedia.org/wiki/embedded_system - Embedded systems
 * http://en.wikipedia.org/wiki/cross-compilation - cross compiling
 * http://www.netrino.com/Publications/Glossary/ - Embedded systems glossary
----
## This page is referenced from http://www.emdebian.org/
CategoryPermalink
CategoryEmdebian

Emdebian vision

In the Emdebian vision someone wishing to build a GNU / Linux based device would:

  1. Port the linux kernel to their hardware (including writing any specific device drivers).
  2. Select the prebuilt emdebian packages needed to support their application.

  3. Package their application as Debian package(s) using Debian and Emdebian tools.
  4. Build a root filesystem using Emdebian tools from the steps above.

Emdebian is involved in steps 2,3,4 above (there are far too many embedded device hardware variations to make prebuilt kernels practical).

Thus EmDebian is a binary distribution for embedded devices (whereas most of the other contenders in this space are source distributions [of course being Debian and open source the source code is still available if required].

What emdebian does

In short, what EmDebian does is wrap around the regular debian package building tools to provide a more fine grained control over package selection, size, dependencies and content to enable creation of very small and efficient debian packages for use on naturally resource limited embedded targets.

We adapt Debian tools so you can build/cross-compile Debian packages or adapted packages with info on how to cross-build and build smaller packages.

We currently have two separate distributions - a post-processed binary-compatible distribution called Emdebian Grip for intermediate installations and a much smaller, much more involved cross-built distribution called Emdebian Crush. Each distribution can be further optimised for systems that would benefit from a static configuration by removing the maintainer scripts - this flavour is called Emdebian Baked.

Emdebian is experimenting with different approaches to cross-building and small-system-friendly packages in order to produce sustainable and robust long-term solutions. For cross-building there are two main approaches: Scratchbox (pkg), which allows cross-builds to appear to be native to the apps being built, and dpkg-cross (pkg) which allows conventional cross-builds to occur in a Debian-compatible way using emdebian-tools (pkg).

For package meta-data making packages more suitable for shrinking, two approaches are currently being investigated. The STAG approach uses a $(DEBIAN_DIR) directory to overlay changed info from the /debian directory, and is used by STAGE. The udeb approach of Debian-installer gives a tighter integration with Debian, with embedded packages having changed names so they can exist in the debian package namespace. Both mechanisms have pros and cons which are discussed in EmdebianMetaData.

New packages can be added to Grip using the emdebian-grip-server package. Adding new packages or new architectures to Crush is currently stalled until the EmdebianCodeAudit can be completed in order to resolve issues around packages with modified functionality.

Distribution status

Crush 2.0 is currently stalled due to build system problems - see EmdebianCrush for details of why, how this could be fixed and current experiments. If things stay as they are, there will be no release of Crush 2.0 (based on Debian 6.0 "squeeze"), instead development will target Crush 3.0 (based on Debian 7.0 - wheezy), reliant upon delivery of an implementation of Multiarch with sufficient support for cross-building.

Grip development is continuing. Grip 2.0 has been released alongside Debian 6.0 "squeeze" on six architectures (armel, mips, mipsel, i386, amd64, powerpc) and with some 2,000 packages. Work is ongoing alongside Debian for Emdebian Grip 3.0 based on Debian GNU/Linux 7.0 Wheezy.

Baked is as ready as it can be - Baked is primarily a do-it-yourself distribution but some starter packages are available.

Project Website

http://www.emdebian.org/

August 17th 2006: The server has moved to a new machine in Europe instead of the US.

Wiki SiteMap

Emdebian Web Layout

Frequently Asked Questions (FAQ)

EmbeddedDebianFAQ

Quick Start Guide

EmdebianQuickStart

Tracker

EmdebianTracker - quick notes about issues that need a fix within the Emdebian packages (issues unrelated to cross building). e.g. a postinst script that calls a binary that is not available or with an unknown option, conflicts between packages that only show up when Essential is ignored, extra dependencies needed when Essential is ignored, etc.)

Code Audit

EmdebianCodeAudit - coordinating a complete audit of all patches required for Emdebian Crush 1.0 to support the implementation of the patches within the relevant Debian packages.

Policy

EmdebianPolicy

Example workflows

EmdebianExamples

Wiki pages in continuous devel

CategoryEmdebian

Hardware

Cross Toolchains

Emdebian builds cross-toolchains for use on standard Debian development machines. Repository details. Toolchains for Etch.

Documentation

EmdebianPackagingGuidelines is a document explaining to package maintainers the sorts of things they should be aware of in their packaging to make their packages 'Embedded-friendly'. This includes cross-building and small-system-building.

Embedded Debian Packaging Infrastructure explains how the Emdebian repositories are organised, where the repositories are located and how to setup your system for emdebian uploads.

Emdebian packaging rules details how emdebian packages should be built and organised.

How Grip relates to Crush

Building packages for Crush (for the EmdebianCodeAudit) now bases the final package on the Grip processing. i.e. the package is cross-built for Crush with limited changes and then the cross-built package is run through the Grip scripts to remove documentation, examples, handle Emdebian TDebs etc. This reduces the number of changes that need to be made in the patches for Crush, simplifying the patches and the build process. As the Audit proceeds, the patches necessary to allow the packages to be cross-built at all will be fed back to Debian via the BTS and included into the packages via maintainer uploads or, if necessary, with NMU's by Emdebian developers. The aim is to reduce the number of patches to only those necessary for the functional changes needed for Crush:

  • Avoid calling perl scripts that cannot be replaced by busybox
  • Avoid using perl in maintainer scripts
  • Reduce long dependency chains by turning off optional build components
  • Make maintainer scripts compatible with the busybox shell

This should allow many Debian packages to be cross-built without special tools and without external patches, with compliance to EmdebianPolicy being achieved after the cross-build is complete.

Multiarch and cross-compile

The goal is that both native and cross-compile behave the same, use files in the same location and use the same compiler flags. This is comparable to setting sysroot=/ now.

?MultiArchExperiments

Current situation: all -*-cross debs, no multiarch

includes:

/usr/triplet/include

libraries:

/usr/triplet/lib

pkg-config:

/usr/triplet/lib/pkg-config

Soon: all -*-cross debs, some multiarch -*-cross debs

includes:

/usr/triplet/include

libraries:

/usr/triplet/lib

pkg-config:

/usr/triplet/lib/pkg-config

Since multiarch libfoo for foreign archs can't be installed with current dpkg they must still be converted to -*-cross packages by dpkg-cross. That involves moving files out of the multiarch directories into the old cross compile dirs and adjusting .pc and .la files.

Multiarch transition: some -*-cross debs, some -*-cross dummy debs, some multiarch debs

includes:

/usr/include + /usr/include/triplet + /usr/triplet/include

libraries:

/lib/triplet + /usr/lib/triplet + /usr/triplet/lib

pkg-config:

/usr/lib/triplet/pkg-config + /usr/triplet/lib/pkg-config

This starts when dpkg can install multiarch libfoo for foreign archs. Dpkg-cross will convert multiarch libfoo to a dummy libfoo-arch-cross that depends on libfoo:arch (= ver) and is completly empty. The dummy libfoo-arch-cross has 2 jobs: Pull in libfoo:arch on upgrade and satisfy reverse dependencies. Over time the reverse dependencies will change to use libfoo:arch directly.

For gcc you have this behaviour already with sysroot=/. But libtool, pkg-config and probably some more will need to be adjusted to this.

Long term goal: no -*-cross debs, all multiarch

includes:

/usr/include + /usr/include/triplet

libraries:

/lib/triplet + /usr/lib/triplet

pkg-config:

/usr/lib/triplet/pkg-config

TODO

Things that need doing are:

  • Emdebianising more packages for Crush so that they
    1. cross-build correctly
    2. split packages where the default has too many dependencies
  • Work on Emdebian policy and actual packages to:
    • Speed up boot time
    • Reduce dependencies
    • Differentiate between install time and run-time dependencies
    • Differentiate between build and target build-dependencies
    • Shrink minimum rootfs sizes
      • Busybox-based rootfs as well as GNU-based EmdebianRootfs

      • Dash instead of bash
      • Remove perl from essential
  • Work on Debian and emdebian packages to:
    • Support changes due to above policy
    • Push changes back into Debian as much as possible (e.g faster boot)
  • Infrastructure Work:
    • Completing auto-build mechansism that will keep emdebian cross-toolchains uptodate with debian toolchain versions
    • Incorporate full 3-stage bootstrap changes from slind
    • Auto-build emdebian packages, tracking debian versions and including support for developers to track packages which have build failures
  • Document:
    • Making Emdebian systems
    • Emdebianising new packages
    • The infrastructure
    • Developed Policy
  • create configurations for various devices and document howto do it
  • promote DEB_BUILD_OPTIONS=nodocs flag for not building docs
  • promote DEB_BUILD_OPTIONS=notest or DEB_BUILD_OPTIONS=nocheck for skipping tests
  • co-operate with i386-uclibc team and embedded arch porters
  • Kill scratchbox in its current form and make a better scratchbox2

Set-up a buildd inside scratchbox

Document setting up the buildd so developers and non-developers can try it out for themselves.

Integrate ?uclibc in the story. This will definitely give some size boost toward smaller footprint to the packages.

Once we have proven that this works start pushing things into Debian proper.

Optional :

  • Make sure everything works with the successors of scratchbox
  • Check the ARM build of Ubuntu

See also: EmDebian/DeBootstrap and EmDebian/CrossDebootstrap .

Wacky Ideas

There are also a few ideas that may need to be investigated further to decide whether it is a good idea to make them a new TODO item. These are collected on the EmdebianWackyIdeas page.

Donations / Donors

Emdebian Donations and Donors

Meetings, Conferences, Work Sessions

There are IRC meetings roughly every two weeks, for details please see Emdebian/Meetings

2009

February 7th-8th in Brussels, Belgium Fosdem2009

2008

September, Extremadura Work Meeting EmdebianWorkSessionExtremadura2008

June, TCL work session

February 23rd-24th in Brussels, Belgium http://fosdem.org/2008/

2007

June 17-23rd, in Edinburgh, Scotland. DebConf7

February 24&25th, in Brussels, Belgium. Fosdem2007

2006

October 25&26th, in London, UK. London Linux World Expo

May 14th - May 21st, in Mexico. DebConf6EmdebianStuff

April 12-16th, in Extremadura, Spain. DebianEmbeddedWorkSessionExtremadura2006

February 25&26th, in Brussels, Belgium. Fosdem Fosdem2006


CategoryPermalink CategoryEmdebian