1840
Comment: Moved embedian content to page with correctly spelled title
|
13265
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
Adapted Debian tools so you can build/cross-compile DebianPackage s or adapted packages with an emdebian dir. As it seems at the moment the ["Scratchbox"] environment seems to be a primary target to try out these things. With the advantage that we can ignore some cross-compilation issues, and that the tools are easily usable natively too. |
#language en === Emdebian vision === In the Emdebian vision someone wishing to build a GNU / Linux based device would: 1. Port the linux kernel to their hardware (including writing any specific device drivers). 1. Select the '''''prebuilt''''' emdebian packages needed to support their application. 1. Package their application as Debian package(s) using Debian and Emdebian tools. 1. Build a root filesystem using Emdebian tools from the steps above. Emdebian is involved in steps 2,3,4 above (there are far too many embedded device hardware variations to make prebuilt kernels practical). Thus EmDebian is a '''''binary''''' distribution for embedded devices (whereas most of the other contenders in this space are '''''source''''' distributions [of course being Debian and open source the source code is still available if required]. === What emdebian does === '''''In short, what EmDebian does is wrap around the regular debian package building tools to provide a more fine grained control over package selection, size, dependencies and content to enable creation of very small and efficient debian packages for use on naturally resource limited embedded targets.''''' We adapt Debian tools so you can build/cross-compile Debian packages or adapted packages with info on how to cross-build and build smaller packages. We currently have two separate distributions - a post-processed binary-compatible distribution called [[http://www.emdebian.org/grip/|Emdebian Grip]] for intermediate installations and a much smaller, much more involved cross-built distribution called [[http://www.emdebian.org/crush/|Emdebian Crush]]. Each distribution can be further optimised for systems that would benefit from a static configuration by removing the maintainer scripts - this flavour is called [[http://www.emdebian.org/baked/|Emdebian Baked]]. Emdebian is experimenting with different approaches to cross-building and small-system-friendly packages in order to produce sustainable and robust long-term solutions. For cross-building there are two main approaches: [[http://www.scratchbox.org/|Scratchbox]] ([[DebPkg:scratchbox2|pkg]]), which allows cross-builds to appear to be native to the apps being built, and dpkg-cross ([[DebPkg:dpkg-cross|pkg]]) which allows conventional cross-builds to occur in a Debian-compatible way using emdebian-tools ([[DebPkg:emdebian-tools|pkg]]). For package meta-data making packages more suitable for shrinking, two approaches are currently being investigated. The STAG approach uses a $(DEBIAN_DIR) directory to overlay changed info from the /debian directory, and is used by STAGE. The udeb approach of Debian-installer gives a tighter integration with Debian, with embedded packages having changed names so they can exist in the debian package namespace. Both mechanisms have pros and cons which are discussed in EmdebianMetaData. New packages can be added to Grip using the [[DebPkg:emdebian-grip-server|emdebian-grip-server]] package. Adding new packages or new architectures to Crush is currently stalled until the EmdebianCodeAudit can be completed in order to resolve issues around packages with modified functionality. == Distribution status == Crush 2.0 is currently stalled due to build system problems - see EmdebianCrush for details of why, how this could be fixed and current experiments. If things stay as they are, there will be no release of Crush 2.0 (based on Debian 6.0 "squeeze"), instead development will target Crush 3.0 (based on Debian 7.0 - wheezy), reliant upon delivery of an implementation of Multiarch with sufficient support for cross-building. Grip development is continuing. Grip 2.0 has been released alongside Debian 6.0 "squeeze" on six architectures (armel, mips, mipsel, i386, amd64, powerpc) and with some 2,000 packages. Work is ongoing alongside Debian for Emdebian Grip 3.0 based on Debian GNU/Linux 7.0 Wheezy. Baked is as ready as it can be - Baked is primarily a do-it-yourself distribution but some starter packages are available. == Project Website == http://www.emdebian.org/ August 17th 2006: The server has moved to a new machine in Europe instead of the US. == Wiki SiteMap == [[EmdebianWebLayout|Emdebian Web Layout]] == Frequently Asked Questions (FAQ) == [[EmbeddedDebianFAQ]] == Quick Start Guide == EmdebianQuickStart == Tracker == EmdebianTracker - quick notes about issues that need a fix within the Emdebian packages (issues unrelated to cross building). e.g. a postinst script that calls a binary that is not available or with an unknown option, conflicts between packages that only show up when Essential is ignored, extra dependencies needed when Essential is ignored, etc.) == Code Audit == EmdebianCodeAudit - coordinating a complete audit of all patches required for Emdebian Crush 1.0 to support the implementation of the patches within the relevant Debian packages. == Policy == EmdebianPolicy == Example workflows == EmdebianExamples == Wiki pages in continuous devel == CategoryEmdebian == Hardware == * [[/Smartphone]] - Debian in [[http://en.wikipedia.org/wiki/Smartphone|smart phones]]. * PDA - [[http://en.wikipedia.org/wiki/Personal_digital_assistant|Debian in Personal digital assistant]]. * Domotics - [[http://en.wikipedia.org/wiki/X10_(industry_standard)|Debian in X10 and other domotic standards]]. * DebianRobot . == Cross Toolchains == Emdebian builds [[EmdebianToolchain|cross-toolchains]] for use on standard Debian development machines. [[http://www.emdebian.org/crosstools.html|Repository details]]. [[EtchCrossToolchains|Toolchains for Etch]]. == Documentation == EmdebianPackagingGuidelines is a document explaining to package maintainers the sorts of things they should be aware of in their packaging to make their packages 'Embedded-friendly'. This includes cross-building and small-system-building. [[Embedded_Debian_packaging_infrastructure|Embedded Debian Packaging Infrastructure]] explains how the Emdebian repositories are organised, where the repositories are located and how to setup your system for emdebian uploads. [[Embedded_Debian_packaging_rules|Emdebian packaging rules]] details how emdebian packages should be built and organised. == How Grip relates to Crush == Building packages for Crush (for the EmdebianCodeAudit) now bases the final package on the Grip processing. i.e. the package is cross-built for Crush with limited changes and then the cross-built package is run through the Grip scripts to remove documentation, examples, handle Emdebian TDebs etc. This reduces the number of changes that need to be made in the patches for Crush, simplifying the patches and the build process. As the Audit proceeds, the patches necessary to allow the packages to be cross-built at all will be fed back to Debian via the [[http://bugs.debian.org/|BTS]] and included into the packages via maintainer uploads or, if necessary, with NMU's by Emdebian developers. The aim is to reduce the number of patches to only those necessary for the functional changes needed for Crush: * Avoid calling perl scripts that cannot be replaced by busybox * Avoid using perl in maintainer scripts * Reduce long dependency chains by turning off optional build components * Make maintainer scripts compatible with the busybox shell This should allow many Debian packages to be cross-built without special tools and without external patches, with compliance to EmdebianPolicy being achieved '''after''' the cross-build is complete. == Multiarch and cross-compile == The goal is that both native and cross-compile behave the same, use files in the same location and use the same compiler flags. This is comparable to setting sysroot=/ now. MultiArchExperiments === Current situation: all -*-cross debs, no multiarch === ||includes:||/usr/triplet/include || ||libraries:||/usr/triplet/lib || ||pkg-config:||/usr/triplet/lib/pkg-config || === Soon: all -*-cross debs, some multiarch -*-cross debs === ||includes:||/usr/triplet/include || ||libraries:||/usr/triplet/lib || ||pkg-config:||/usr/triplet/lib/pkg-config || Since multiarch libfoo for foreign archs can't be installed with current dpkg they must still be converted to -*-cross packages by dpkg-cross. That involves moving files out of the multiarch directories into the old cross compile dirs and adjusting .pc and .la files. === Multiarch transition: some -*-cross debs, some -*-cross dummy debs, some multiarch debs === ||includes:||/usr/include + /usr/include/triplet + /usr/triplet/include || ||libraries:||/lib/triplet + /usr/lib/triplet + /usr/triplet/lib || ||pkg-config:||/usr/lib/triplet/pkg-config + /usr/triplet/lib/pkg-config || This starts when dpkg can install multiarch libfoo for foreign archs. Dpkg-cross will convert multiarch libfoo to a dummy libfoo-arch-cross that depends on libfoo:arch (= ver) and is completly empty. The dummy libfoo-arch-cross has 2 jobs: Pull in libfoo:arch on upgrade and satisfy reverse dependencies. Over time the reverse dependencies will change to use libfoo:arch directly. For gcc you have this behaviour already with sysroot=/. But libtool, pkg-config and probably some more will need to be adjusted to this. === Long term goal: no -*-cross debs, all multiarch === ||includes:||/usr/include + /usr/include/triplet || ||libraries:||/lib/triplet + /usr/lib/triplet || ||pkg-config:||/usr/lib/triplet/pkg-config || |
Line 10: | Line 129: |
As Ed seems to have a working environment I guess we should start working with that and improve it on the go. Seems you got the "build base system" item finished Ed. Thanks! And the second item "ability to build a base ["rootfs"] too! One good thing would that he did not mention in his mail was adding support for ignoring packaging documentation and man-pages in the packages. A harder thing to do is actually making sure that the docs don't even get build. And a way to avoid the circular dependencies that exist due to this documentation building tools. Set-up a buildd inside scratchbox (which should be possible as I heard that it has already been done) to start building more packages. (This will hopefully be possible when the emdebian server is back up). (Once I find out how to silence my proliant server I might hook it up to my internet line and provide a secondary build env) Document setting up the buildd so ["people"]/ ["dev"]s can try it out for themselves. Integrate ["uclibc"] in the story. This will definitely give some size boost toward smaller footprint to the packages. Once we have proven that this works start pushing things into Debian proper. Optional : -Make sure everything works with the successors of scratchbox\ -Check the ARM build of Ubuntu Well I hope I can open a fruitfull discussion with this and hope to see some more people starting to work on this. See also: BootUsb , DebianRobot . |
Things that need doing are: * Emdebianising more packages for Crush so that they a. cross-build correctly a. split packages where the default has too many dependencies * Work on Emdebian policy and actual packages to: * Speed up boot time * Reduce dependencies * Differentiate between install time and run-time dependencies * Differentiate between build and target build-dependencies * Shrink minimum rootfs sizes * Busybox-based rootfs as well as GNU-based EmdebianRootfs * Dash instead of bash * Remove perl from essential * Work on Debian and emdebian packages to: * Support changes due to above policy * Push changes back into Debian as much as possible (e.g faster boot) * Infrastructure Work: * Completing auto-build mechansism that will keep emdebian cross-toolchains uptodate with debian toolchain versions * Incorporate full 3-stage bootstrap changes from slind * Auto-build emdebian packages, tracking debian versions and including support for developers to track packages which have build failures * Document: * Making Emdebian systems * Emdebianising new packages * The infrastructure * Developed Policy * create configurations for various devices and document howto do it * promote DEB_BUILD_OPTIONS=nodocs flag for not building docs * promote DEB_BUILD_OPTIONS=notest or DEB_BUILD_OPTIONS=nocheck for skipping tests * co-operate with i386-uclibc team and embedded arch porters * Kill scratchbox in its current form and make a better scratchbox2 Set-up a buildd inside scratchbox Document setting up the buildd so developers and non-developers can try it out for themselves. Integrate [[uclibc]] in the story. This will definitely give some size boost toward smaller footprint to the packages. Once we have proven that this works start pushing things into Debian proper. Optional : * Make sure everything works with the successors of scratchbox * Check the ARM build of Ubuntu See also: EmDebian/DeBootstrap and EmDebian/CrossDebootstrap . == Wacky Ideas == There are also a few ideas that may need to be investigated further to decide whether it is a good idea to make them a new TODO item. These are collected on the EmdebianWackyIdeas page. == Donations / Donors == [[EmdebianDonations|Emdebian Donations and Donors]] == Meetings, Conferences, Work Sessions == There are IRC meetings roughly every two weeks, for details please see [[Emdebian/Meetings]] ==== 2009 ==== February 7th-8th in Brussels, Belgium [[http://fosdem.org/2009/|Fosdem2009]] ==== 2008 ==== September, Extremadura Work Meeting EmdebianWorkSessionExtremadura2008 June, [[http://www.toby-churchill.com/|TCL]] work session February 23rd-24th in Brussels, Belgium [[http://fosdem.org/2008/]] ==== 2007 ==== June 17-23rd, in Edinburgh, Scotland. [[https://debconf7.debconf.org/wiki/Main_Page|DebConf7]] February 24&25th, in Brussels, Belgium. [[http://www.fosdem.org/2007/|Fosdem2007]] ==== 2006 ==== October 25&26th, in London, UK. [[http://www.linuxworldexpo.co.uk/|London Linux World Expo]] May 14th - May 21st, in Mexico. DebConf6EmdebianStuff April 12-16th, in Extremadura, Spain. DebianEmbeddedWorkSessionExtremadura2006 February 25&26th, in Brussels, Belgium. [[http://wiki.debian.org/EmDebian/Fosdem2006|Fosdem]] [[http://www.fosdem.org/2006/|Fosdem2006]] == External links == * http://en.wikipedia.org/wiki/embedded_system - Embedded systems * http://en.wikipedia.org/wiki/cross-compilation - cross compiling * http://www.netrino.com/Publications/Glossary/ - Embedded systems glossary ---- ## This page is referenced from http://www.emdebian.org/ CategoryPermalink CategoryEmdebian |
Emdebian vision
In the Emdebian vision someone wishing to build a GNU / Linux based device would:
- Port the linux kernel to their hardware (including writing any specific device drivers).
Select the prebuilt emdebian packages needed to support their application.
- Package their application as Debian package(s) using Debian and Emdebian tools.
- Build a root filesystem using Emdebian tools from the steps above.
Emdebian is involved in steps 2,3,4 above (there are far too many embedded device hardware variations to make prebuilt kernels practical).
Thus EmDebian is a binary distribution for embedded devices (whereas most of the other contenders in this space are source distributions [of course being Debian and open source the source code is still available if required].
What emdebian does
In short, what EmDebian does is wrap around the regular debian package building tools to provide a more fine grained control over package selection, size, dependencies and content to enable creation of very small and efficient debian packages for use on naturally resource limited embedded targets.
We adapt Debian tools so you can build/cross-compile Debian packages or adapted packages with info on how to cross-build and build smaller packages.
We currently have two separate distributions - a post-processed binary-compatible distribution called Emdebian Grip for intermediate installations and a much smaller, much more involved cross-built distribution called Emdebian Crush. Each distribution can be further optimised for systems that would benefit from a static configuration by removing the maintainer scripts - this flavour is called Emdebian Baked.
Emdebian is experimenting with different approaches to cross-building and small-system-friendly packages in order to produce sustainable and robust long-term solutions. For cross-building there are two main approaches: Scratchbox (pkg), which allows cross-builds to appear to be native to the apps being built, and dpkg-cross (pkg) which allows conventional cross-builds to occur in a Debian-compatible way using emdebian-tools (pkg).
For package meta-data making packages more suitable for shrinking, two approaches are currently being investigated. The STAG approach uses a $(DEBIAN_DIR) directory to overlay changed info from the /debian directory, and is used by STAGE. The udeb approach of Debian-installer gives a tighter integration with Debian, with embedded packages having changed names so they can exist in the debian package namespace. Both mechanisms have pros and cons which are discussed in EmdebianMetaData.
New packages can be added to Grip using the emdebian-grip-server package. Adding new packages or new architectures to Crush is currently stalled until the EmdebianCodeAudit can be completed in order to resolve issues around packages with modified functionality.
Distribution status
Crush 2.0 is currently stalled due to build system problems - see EmdebianCrush for details of why, how this could be fixed and current experiments. If things stay as they are, there will be no release of Crush 2.0 (based on Debian 6.0 "squeeze"), instead development will target Crush 3.0 (based on Debian 7.0 - wheezy), reliant upon delivery of an implementation of Multiarch with sufficient support for cross-building.
Grip development is continuing. Grip 2.0 has been released alongside Debian 6.0 "squeeze" on six architectures (armel, mips, mipsel, i386, amd64, powerpc) and with some 2,000 packages. Work is ongoing alongside Debian for Emdebian Grip 3.0 based on Debian GNU/Linux 7.0 Wheezy.
Baked is as ready as it can be - Baked is primarily a do-it-yourself distribution but some starter packages are available.
Project Website
August 17th 2006: The server has moved to a new machine in Europe instead of the US.
Wiki SiteMap
Frequently Asked Questions (FAQ)
Quick Start Guide
Tracker
EmdebianTracker - quick notes about issues that need a fix within the Emdebian packages (issues unrelated to cross building). e.g. a postinst script that calls a binary that is not available or with an unknown option, conflicts between packages that only show up when Essential is ignored, extra dependencies needed when Essential is ignored, etc.)
Code Audit
EmdebianCodeAudit - coordinating a complete audit of all patches required for Emdebian Crush 1.0 to support the implementation of the patches within the relevant Debian packages.
Policy
Example workflows
Wiki pages in continuous devel
Hardware
/Smartphone - Debian in smart phones.
Domotics - Debian in X10 and other domotic standards.
Cross Toolchains
Emdebian builds cross-toolchains for use on standard Debian development machines. Repository details. Toolchains for Etch.
Documentation
EmdebianPackagingGuidelines is a document explaining to package maintainers the sorts of things they should be aware of in their packaging to make their packages 'Embedded-friendly'. This includes cross-building and small-system-building.
Embedded Debian Packaging Infrastructure explains how the Emdebian repositories are organised, where the repositories are located and how to setup your system for emdebian uploads.
Emdebian packaging rules details how emdebian packages should be built and organised.
How Grip relates to Crush
Building packages for Crush (for the EmdebianCodeAudit) now bases the final package on the Grip processing. i.e. the package is cross-built for Crush with limited changes and then the cross-built package is run through the Grip scripts to remove documentation, examples, handle Emdebian TDebs etc. This reduces the number of changes that need to be made in the patches for Crush, simplifying the patches and the build process. As the Audit proceeds, the patches necessary to allow the packages to be cross-built at all will be fed back to Debian via the BTS and included into the packages via maintainer uploads or, if necessary, with NMU's by Emdebian developers. The aim is to reduce the number of patches to only those necessary for the functional changes needed for Crush:
- Avoid calling perl scripts that cannot be replaced by busybox
- Avoid using perl in maintainer scripts
- Reduce long dependency chains by turning off optional build components
- Make maintainer scripts compatible with the busybox shell
This should allow many Debian packages to be cross-built without special tools and without external patches, with compliance to EmdebianPolicy being achieved after the cross-build is complete.
Multiarch and cross-compile
The goal is that both native and cross-compile behave the same, use files in the same location and use the same compiler flags. This is comparable to setting sysroot=/ now.
?MultiArchExperiments
Current situation: all -*-cross debs, no multiarch
includes: |
/usr/triplet/include |
libraries: |
/usr/triplet/lib |
pkg-config: |
/usr/triplet/lib/pkg-config |
Soon: all -*-cross debs, some multiarch -*-cross debs
includes: |
/usr/triplet/include |
libraries: |
/usr/triplet/lib |
pkg-config: |
/usr/triplet/lib/pkg-config |
Since multiarch libfoo for foreign archs can't be installed with current dpkg they must still be converted to -*-cross packages by dpkg-cross. That involves moving files out of the multiarch directories into the old cross compile dirs and adjusting .pc and .la files.
Multiarch transition: some -*-cross debs, some -*-cross dummy debs, some multiarch debs
includes: |
/usr/include + /usr/include/triplet + /usr/triplet/include |
libraries: |
/lib/triplet + /usr/lib/triplet + /usr/triplet/lib |
pkg-config: |
/usr/lib/triplet/pkg-config + /usr/triplet/lib/pkg-config |
This starts when dpkg can install multiarch libfoo for foreign archs. Dpkg-cross will convert multiarch libfoo to a dummy libfoo-arch-cross that depends on libfoo:arch (= ver) and is completly empty. The dummy libfoo-arch-cross has 2 jobs: Pull in libfoo:arch on upgrade and satisfy reverse dependencies. Over time the reverse dependencies will change to use libfoo:arch directly.
For gcc you have this behaviour already with sysroot=/. But libtool, pkg-config and probably some more will need to be adjusted to this.
Long term goal: no -*-cross debs, all multiarch
includes: |
/usr/include + /usr/include/triplet |
libraries: |
/lib/triplet + /usr/lib/triplet |
pkg-config: |
/usr/lib/triplet/pkg-config |
TODO
Things that need doing are:
- Emdebianising more packages for Crush so that they
- cross-build correctly
- split packages where the default has too many dependencies
- Work on Emdebian policy and actual packages to:
- Speed up boot time
- Reduce dependencies
- Differentiate between install time and run-time dependencies
- Differentiate between build and target build-dependencies
- Shrink minimum rootfs sizes
Busybox-based rootfs as well as GNU-based EmdebianRootfs
- Dash instead of bash
- Remove perl from essential
- Work on Debian and emdebian packages to:
- Support changes due to above policy
- Push changes back into Debian as much as possible (e.g faster boot)
- Infrastructure Work:
- Completing auto-build mechansism that will keep emdebian cross-toolchains uptodate with debian toolchain versions
- Incorporate full 3-stage bootstrap changes from slind
- Auto-build emdebian packages, tracking debian versions and including support for developers to track packages which have build failures
- Document:
- Making Emdebian systems
- Emdebianising new packages
- The infrastructure
- Developed Policy
- create configurations for various devices and document howto do it
- promote DEB_BUILD_OPTIONS=nodocs flag for not building docs
- promote DEB_BUILD_OPTIONS=notest or DEB_BUILD_OPTIONS=nocheck for skipping tests
- co-operate with i386-uclibc team and embedded arch porters
- Kill scratchbox in its current form and make a better scratchbox2
Set-up a buildd inside scratchbox
Document setting up the buildd so developers and non-developers can try it out for themselves.
Integrate ?uclibc in the story. This will definitely give some size boost toward smaller footprint to the packages.
Once we have proven that this works start pushing things into Debian proper.
Optional :
- Make sure everything works with the successors of scratchbox
- Check the ARM build of Ubuntu
See also: EmDebian/DeBootstrap and EmDebian/CrossDebootstrap .
Wacky Ideas
There are also a few ideas that may need to be investigated further to decide whether it is a good idea to make them a new TODO item. These are collected on the EmdebianWackyIdeas page.
Donations / Donors
Meetings, Conferences, Work Sessions
There are IRC meetings roughly every two weeks, for details please see Emdebian/Meetings
2009
February 7th-8th in Brussels, Belgium Fosdem2009
2008
September, Extremadura Work Meeting EmdebianWorkSessionExtremadura2008
June, TCL work session
February 23rd-24th in Brussels, Belgium http://fosdem.org/2008/
2007
June 17-23rd, in Edinburgh, Scotland. DebConf7
February 24&25th, in Brussels, Belgium. Fosdem2007
2006
October 25&26th, in London, UK. London Linux World Expo
May 14th - May 21st, in Mexico. DebConf6EmdebianStuff
April 12-16th, in Extremadura, Spain. DebianEmbeddedWorkSessionExtremadura2006
February 25&26th, in Brussels, Belgium. Fosdem Fosdem2006
External links
http://en.wikipedia.org/wiki/embedded_system - Embedded systems
http://en.wikipedia.org/wiki/cross-compilation - cross compiling
http://www.netrino.com/Publications/Glossary/ - Embedded systems glossary