Debian-packaging the output of crosstool

Update 2006-05-22

I've made a quick and dirty hack to bundle crosstool output into .tgz and .deb packages. See http://freaknet.org/martin/crosstool for the scripts and some packages.

Abstract

Support for cross-compiler packages in Debian is patchy and adhoc at present. The main Debian archive includes

while cross-compilers for individual mainline targets are produced as-and-when by bright individuals for a random selection of architectures, with a random selection of support libraries, invariably running only on i386 machines:

deb http://www.emdebian.org/emdebian-tools/stable ./
deb http://zigzag.lvk.cs.msu.su/~nikita/debian/ sarge/
deb http://zigzag.lvk.cs.msu.su/~nikita/debian/ sid/

Crosstool is able to produce working cross-compilers for C and C++ with a standard set of support libraries (libgcc and glibc) for a huge number of host/target machine combinations, including all the supported Debian architectures. If we could Debian-package the output from crosstool this would be a big step forwards for Debian cross compiler land.

(Note from crosstool author: I have a shell script that uses Alien to create usable .deb packages. Will that do?)

At present (crosstool-0.42), crosstool is not "productized" but is a hackers' playpen. You configure and run it in its own source directory and it installs cross-compilers under /opt/crosstool. There is a script "makerpms.sh" which should build RPM files of the results; if we could write a "makedebs.sh" script that produced .deb files conforming to Debian conventions, we would be laughing. That is what this page is all about.

Is it possible?

Crosstool's demo scripts install into its own idiosyncratic directory structure, for example /opt/crosstool/gcc-4.0.1-glibc-2.3.6/arm-none-linux-gnu/ containing directories arm-none-linux-gnu, bin, distributed, include, info, lib, libexec, man, share and tmp and refuse to install into / or /usr (because it would delete their contents before starting!).

There is already a USE_SYSROOT=1 hook that distributes things between .../lib, .../usr/lib and .../usr/include within its build directory, and it has a script that claims to "fix embedded pathnames". However, installing all the components into the correct places in the Linux filetree may require some study.

Does it satisfy Debian Policy?

Debian standard are stringent. At present crosstool itself is not even installable, let alone packagable, so cannot be specified as a build dependency. Presumable, the binary .debs produced by crosstool would not have corresponding source packages, since crosstool itself is not Debianised (and is not likely to be Debianisable without a major rewrite).

If not, it might still be legal to make an independent repository containing the binary .debs, with apologies and instructions for building them.

Simply packaging up crosstool's usual /opt/crosstool/gcc-4.1.0-glibc-2.4/arm-unknown-linux-gnueabi dir is against the FHS for /opt, which requires user executables in /opt/something/bin and so on.

Has it already been done?

Do toolchain-source, dpkg-cross, binutils-multiarch, scratchbox,the emdebian project (and others?) already do what we are trying to do?

toolchain-source

toolchain-source is a Debian package that "contains everything that is needed to download and compile all packages that are required for a cross compiler (that is to say, gcc, binutils and libc...)"

Actually it contains its own copy of the source code for a specific version of GCC (3.4.5). It is supposed to be able to produce cross-compilers, but is now deprecated, its maintainer wants to abandon it and there is no framework for upgrading to newer version of the toolchain components.

dpkg-cross

dpkg-cross "is a tool for installing libraries and headers for cross compiling in a way similar to dpkg. Furthermore, the functionality of dpkg-buildpackage and dpkg-shlibdeps is enhanced to support cross compiling."

It seems to do this by downloading native C library packages and headers for alien architectures and installing them as cross-development libraries on the native platform.

It looks like a friend, but does not manage the cross-compilers themselves.

binutils-multiarch

Version of binutils programs "to manipulate binary and object files that may have been created on other architectures. ... a cross-assembling version of gas is not included in this package, just the binary utilities."

No good. We need to provide the whole set of cross-binutils utilities.

scratchbox

"Scratchbox is a cross-compilation toolkit designed to make embedded Linux application development easier. It also provides a full set of tools to integrate and cross-compile an entire Linux distribution."

Scratchbox uses either QEMU or a real target hardware (via sbrsh) to execute target architecture binaries. This is extremely useful for handling all the ./configure script tests and other situations where a cross-compile build process requires executing target arch binaries.

It has good Debian support, both as a host and especially for crosscompiling Debian packages. Latest Debian packages of scratchbox can found from the following apt repository.

deb http://scratchbox.org/debian ./

including a scratchbox-devkit-debian. The current Debian (target) devkit includes dpkg and apt equilavent of sarge packages.

Crosstool-created toolchains can be used within scratchbox: see http://www.scratchbox.org/wiki/ForeignToolchains

Scratchbox itself is not in mainline Debian due to it size (it contains practically it's own Linux distribution inside it). "Scratchbox2 intends to fix this" they say...

It's not what we need but if we are lucky they might make actual Debian packages from crosstool output as part of scratchbox's stuff before we do :)

emdebian

The embedded Debian project documents a simple procedure for building binary Debian packages for cross-compilers from existing Debian source packages (gcc, glibc, binutils). It relies on the versions of the sources you need to use already being included in Debian.

Differences

Here is a list of some of the things that need changing to Debianise crosstool's output

Binary names

The demo scripts create arm-unknown-linux-gnueabi-gcc and friends. These need to be installed in Debian as arm-linux-gnueabi-gcc etc (maybe just change the arch in the .dat file?). Mind you, this would conflict with existing cross-compiler names. Would arm-crosstool-linux-gnueabi-gcc be better (using the "vendor" field of the GCC target name)?

Installation directories and embedded pathnames

Needs to install into a separate filesystem tree to then be packaged (using crosstool option USE_SYSROOT=1?). The existing Emdebian packages

pull in

and install to

and strangely also to:

though this may be an error.

Crosstool installs into $RESULT_TOP/gcc-4.1.0-glibc-2.4 with directories

and is also known to favour /opt/crosstool as a system installation directory.

Alternatively, with the USE_SYSROOT option, it installs to the same dir with directories

under arm-unknown-linux-gnueabi:

and under arm-unknown-linux-gnueabi/sys-root:

Making the corresponding set of interdependent Debian packages, with source packages that you can "build" seems a monstrous task. That said, the existing buildsrpms.sh in crosstool claims to make source RPMs that you can "rpmbuild".

Is turning the crosstool output tree into a single all-in binary package for each target under /opt/crosstool Debian-acceptable?

Naming the packages

For instance, crosstool-0.42-arm-linux-gcc-4.0.2-1_i386.deb for the crosscompiler targetting ARM that runs on i386 class machines?

With a little study, the g++ components can be split off into a separate package dependent on the gcc package, and maybe arch-independent bits into *_all files.

External docs