Differences between revisions 15 and 17 (spanning 2 versions)
Revision 15 as of 2010-04-05 15:21:47
Size: 12457
Editor: NeilWilliams
Comment:
Revision 17 as of 2010-04-05 15:23:52
Size: 12502
Editor: NeilWilliams
Comment:
Deletions are marked like this. Additions are marked like this.
Line 28: Line 28:
 * busybox-crush - reconfigured to try and replace coreutils. [[http://www.emdebian.org/svn/browser/current/target/busybox-crush/trunk|source]]  * busybox-crush - reconfigured to try and replace coreutils.
   *
[[http://www.emdebian.org/svn/browser/current/target/busybox-crush/trunk|source]]
Line 51: Line 52:
 * xorg-server-crush -  * xorg-server-crush - Disable aiglx, glx, xglx, xgl and dri.

Emdebian Crush

Definitions

EmdebianCrush - Busybox based root filesystem and packages to support the G Palmtop Environment based on GTK+2. Packages are heavily modified and cross-built - functional changes exist between Emdebian Crush and standard Debian.

Current experiments

The attempt is to carry over the functional changes from Emdebian Crush 1.0 by creating replacement source packages with new binary package names. (Changing the source package name proved complicated.)

Packages are selected on the following criteria:

  1. Avoid packages blacklisted for Emdebian Crush:
  2. Modify busybox to provide the majority of coreutils functionality.

  3. Use the EmdebianCodeAudit to identify packages that break under those conditions and modify them to work.

  4. Provide (usually empty) replacements for commands commonly called by packages but which cannot be supported by Crush.
    • update-rc.d - POSIX shell replacement script replaces the perl one from sysvinit

    • dpkg-divert - empty shell script replaces the perl one from dpkg

    • update-alternatives - empty shell script replaces the perl one from dpkg

  5. Remove optional build components to reduce dependencies:
    • Drop LDAP support
    • Drop PDF support

The current set of packages may change but includes:

  • busybox-crush - reconfigured to try and replace coreutils.
  • cairo-crush - Drop SVG, PDF and PS backends.
  • curl-crush - Drop OpenSSL support.
  • debconf-crush - Drop everything except the confmodule shell script required by cdebconf

  • dpkg-crush - Drop all perl content, drop dependency on coreutils and allow busybox-crush.

  • gnome-vfs-crush - Drop avahi and gnutls support.

  • gnupg-crush - Drop LDAP support.
  • hal-crush - Drop smbios and dependency on adduser

  • libidl-crush - Drop dependency on cpp

  • openssh-crush - Drop use of grep -x in maintainer script - not supportable in busybox. Drop binary-ssh-krb5 build.

  • sysvinit-crush - Replace update-rc.d with a shell version.

  • udev-crush - busybox kill and dash kill options are incompatible.
  • xorg-server-crush - Disable aiglx, glx, xglx, xgl and dri.

Building

  1. svn-buildpackage -us -uc --svn-download-orig --svn-postbuild=/usr/share/emdebian-tools/crush-postbuild.pl
  2. upload the native version
  3. first chroot to build i386
  4. second chroot to cross build for armel, mips, mipsel and powerpc

Usage

Changes are being made in multistrap to use these packages in place of standard packages from ?EmdebianGrip, mainly by omitting all Priority:required packages. This allows users to select exactly which packages are used - it also means that users can easily create completely broken root filesystems. :-)

The goal is that users of Emdebian Crush will all need some core packages:

  • debconf-crush - supports cdebconf
  • dpkg-crush - work with busybox instead of coreutils
  • busybox-crush - drop coreutils

Other Crush packages are optional. When using apt from ?EmdebianGrip, gnupg-crush can be useful. The theory is that packages from Crush provide options for using the packages you need without the dead-end dependencies.

Work list

Seeing as you asked, maybe this little list will give you some idea of why I had to abandon this particular task - I'm not convinced that the problems can be fixed before Squeeze, even if someone else takes on the role. Several stages rely on other tasks within Debian that may or may not make it into Squeeze. The list is by no means complete either! Even before my illness, I had *serious* doubts about whether Crush 2.0 was possible.

0. You'll find plenty of hacks, errors and mistakes that have remained in the current scripts from the earlier development phases. It is worth revisiting things like that and seeing what breaks without them. Ensure you test with more than a few packages - there are various gotchas that only apply to a handful of packages and some of those packages have since had new upstream releases that may well have fixed the issues that led to the hacks in the emdebian-tools scripts. Other improvements elsewhere could also lead to some procedures within the current scripts becoming redundant or new ones needing to be devised.

1. Read the EmdebianCodeAudit wiki page and supporting pages, go through and cross-build each package to work out what each audit point means. Results are for armel, using emsource and emdebuild. Replicate Busybox based root filesystem and packages to support the G Palmtop Environment based on GTK+2. Packages are heavily modified and cross-built - functional changes exist between Emdebian Crush and standard Debian. results using chroots builds (with empdebuild) wherever possible. Various other parts of the wiki also need to be cleaned up, as does the website itself which still has a fair bit of outdated information.

2. Continue discussion on how xcontrol support can actually be implemented. Check how it is used in the EmdebianCodeAudit and tie that into the developments at DebConf9.

  • DebConf result: We want to switch to multiarch. Cross building dependencies can be shoehorned into the current multi-arch specification such that the additional effort for Debian will be minimal, then everything xcontrol can do can be done with multi-arch too as soon as squeeze is released. -- SimonRichter

3. Learn the emdebian-tools scripts and manpages, read the source code and get a handle on how things currently work - taking over maintenance of a native package means taking over the source code too, in effect the "upstream".

4. Struggle with the inherent weaknesses in apt-cross which lie in the horrible apt perl bindings - the most recent version of apt has (once again) broken these bindings and apt-cross needs a patch to continue operating. (Something to do with /etc/apt/preferences.d/). Hope that someone gets apt to understand multiarch so that this whole issue can be removed - you need a C++ expert for that role.

5. Get the toolchains back into installable state - I can't upgrade my own armel boxes today but I don't have time to investigate why. Build scripts that ensure that this persistent problem is fixed for the long term and across all supported architectures. Crush 2.0 abandoned

6. Create a local mirror and use the scripts in emdebian-qa and emdebian-tools to upload cross-built packages into the repo and start building a small set of packages for armel. Don't proceed to other architectures or make the mirror publicly accessible until the rest of the issues in the Code Audit have been fixed.

7. Pick up development of emdebian-tools from existing SVN - there are some changes in SVN that still need testing.

8. Start developing ways to test and fix an existing problem in that the ./configure scripts in some packages insist on looking for stuff in /usr/lib/ and then linking against the shared libraries in /usr/arm-linux-gnueabi/lib/ etc. This is a disaster waiting to happen and must be fixed before Crush can progress to more than one architecture. In the same manner, check builds for /usr/include/ pollution and drop the apt-get build-deps stage in a clean chroot. If no xcontrol file, allow (nay require) native deps.

9. Make notes in EmdebianCodeAudit of packages that fail to rebuild twice in a row and detail why.

10. test and document the proper support for CC_BUILD in autotools.

11. Get Build-Depends split into debian/control and normal Debian? Possibly as well as Depends-Install (installed on build) and Depends-Runtime ? Building a *static* chroot for another machine, only needs runtime stuff. Allows native cross-versions of packages, smallerhttp://lists.debian.org/debian-embedded/2009/08/msg00005.html installed images. No way to put a prefix in front of maintainer scripts for config files. Need a mangling frontend in dpkg before calling the script. Install time scripts vs runtime scripts. (A topic briefly discussed with Wookey, hopefully my notes aren't too brief for the idea to be reconstructed. I knew what I meant at the time.)

12. Create a new wiki page to detail things like this and whatever else comes up as you go. Link it from the EmdebianTracker page?

13. Please don't come to me continually for input, if Crush is to continue, someone needs to take over maintenance and that means filing and fixing the bug reports without looking at me to do it. (You'd alter the Maintainer: field in debian/control to remove my name and enter your own so that the bug messages for the emdebian-tools source package go to you or to this list.)

14. Work out how to fix "X-Build-Cross-Libtool: yes" which is probably tied into other libtool issues identified in the EmdebianCodeAudit to do with refreshing the autoconf and libtool metadata prior to attempting the cross-build.

15. Get CMake build systems cross-building properly.

16. Get a mini-perl and some kind of python support.

17. Solve what ever other issues turn up during these processes.

18. Fix all the packages that used to cross-build for Crush 1.0 but not longer cross-build in unstable. (The EmdebianCodeAudit identifies these packages, one is gcc-4.4). Work out and gain consensus on whether such packages should *not* be cross-built from source and just "borrowed" directly from Grip. This is a Policy decision about whether Crush should be 100% buildable from source or whether compromises are acceptable. Mixing Crush and Grip will *only* work if the xcontrol support is fixed such that changed functionality is always encoded within the binary package name like libgconf-noldap-2-4 etc.

19. Identify *new* packages that have become dependencies of necessary packages in Crush since the release of 1.0 and then work out how to get each of these to cross-build.

20. Test these packages with real systems and see if the rootfs scripts can still operate and what new bugs arise at installation/deployment time.

21. Manage the repository to cope with britney (the script that migrates packages from unstable into testing in Debian).

22. Manage the repository to cope with testing-proposed-updates once the Squeeze release freeze starts to happen.

23. Test and report bugs with the locale support via Emdebian TDebs and langupdate.

24. Fix issues like the bulky gconv files in libc6 and test with the tzdata support.

25. uClibc support

26. Finalise the cache value support in dpkg-cross and add to it if necessary. Consider deploying the cache values into the Debian package such that the files can be created at build time on a per-package-per-architecture basis.

27. Keep an eye on multiarch developments and feed into the process to ensure Debian gets a usable cross-build environment based on multiarch. Get dpkg-cross merged into dpkg as soon as multiarch is supportable.

28. Whatever else I've forgotten.

References


CategoryEmdebian