Differences between revisions 51 and 52
Revision 51 as of 2014-11-16 23:37:50
Size: 15567
Editor: wookey
Comment: Add link to crosstoolchain page
Revision 52 as of 2014-12-05 17:55:54
Size: 16655
Editor: wookey
Comment:
Deletions are marked like this. Additions are marked like this.
Line 6: Line 6:
This has been essentially working in Debian since end-2013 Packaging and packages were uploaded to Debian unstable in October 2014. Unfortunately these did not migrate into Jessie due to infrastructure requirements (sbuild, wanna-build, britney versions) only satisfied in Jessie, ([[ReleaseGoals/CrossToolchains]]). The changes needed to allow multiarch builds on buildds before packages using this technique can be built in the archive were done in sbuild 0.64.3. So packages are available in an external repo. See CrossToolchains for installation details. This has been essentially working on Debian since end-2013. Packaging and packages were uploaded to Debian unstable in October 2014. Unfortunately these did not migrate into Jessie due to infrastructure requirements (sbuild, wanna-build, britney versions) only satisfied in Jessie, ([[ReleaseGoals/CrossToolchains]]). The changes needed to allow multiarch builds on buildds before packages using this technique can be built in the archive were done in sbuild 0.64.3. So packages are available in an external repo. See CrossToolchains for installation details.
Line 12: Line 13:
There have been binary cross-toolchains built for Debian for a decade or so now, using various schemes. (toolchain-source, buildcross). The Emdebian project has provided these, and currently builds them using buildcross, which relies on there already being a suitable glibc package for the target architecture, which is transformed using dpkg-cross so that everything exists in one architecture. The buildcross process is not designed for use on normal buildds due to the cross-arch buildd-deps so these packages have never been available in Debian proper. Hector Oron wrote buildcross and maintained There have been binary cross-toolchains built for Debian for a decade or so now, using various schemes. (toolchain-source, buildcross). The Emdebian project has provided these, and until wheezy buildt them using buildcross, which relies on there already being a suitable glibc package for the target architecture, which is transformed using dpkg-cross so that everything exists in one architecture. The buildcross process is not designed for use on normal buildds due to the cross-arch buildd-deps so these packages have never been available in Debian proper. Hector Oron wrote buildcross and maintained the toolchain builds during etch and squeeze.
Line 16: Line 17:
Multiarch, lets us simplify the build process by using the normal source packages, but building them for specified targets and stages, with cross-arch dependencies. Multiarch lets us simplify the build process by using the normal source packages, but building them for specified targets and stages, with cross-arch dependencies.
Line 30: Line 31:
A related issue is having co-installable native compilers. See CoinstallableToolchains for detals. A related issue is having co-installable native compilers. See CoinstallableToolchains for details.
Line 36: Line 37:
If you just just want working binaries, you can now (2014/10/22) get them from the main Debian/unstable repo! When jessie ships, they should be available there as well. If you just just want working binaries, you can now (2014/10/22) get them from the main Debian/unstable repo! Note that they are uninstallable for a while (up to a few days on slow architectures) after a new gcc-4.9, libc or linux upload as the multiarch libraries have to be in sync across architectures, so the cross-toolchains need to be rebuilt against the
Line 53: Line 54:
Integration work on this is uploaded to [[http://people.debian.org/~wookey/tools/debian| Wookey's toolchain repos]] since 2013. Integration work on this is uploaded to [[http://emdebian.org/tools/debian| The Debian cross-toolchain repo]] since Nov 2014 (previously in a people.debian.org personal repo since 2013).
Line 67: Line 68:
There is a cross-gcc-<ver>-<arch> source package for each target <arch>, which builds a set of cross-compiler packages. The set is currently gcc, g++, and gfortran. gobjc and gccgo will likely be enabled soon. It does not build cross library packages, but uses the native system ones. There is a cross-gcc-<ver>-<arch> source package for each target <arch>, which builds a set of cross-compiler packages. The set is currently gcc, g++, and gfortran. gobjc and gccgo are enabled for jessie, and will be in unstable. It does not build cross library packages, but uses the native system ones. Those source packages are generated from the cross-gcc git repo above.

In experimental the cross-gcc package itself is in the archive, and its binary cross-gcc-dev is used offline to generate the cross-gcc-<ver>-<arch> sources and at build time to build the binaries. This keeps everything in the archive, and gives one package in the archive to file cross-build packaging bugs against.
Line 72: Line 75:
 * crossbuild-essential-<arch> - pulls in binutils-<triplet>, gcc-<triplet>, libc-dev:<arch>  * crossbuild-essential-<arch> - pulls in binutils-<triplet>, gcc-<triplet>, libc-dev:<arch> and dpkg-cross
Line 77: Line 80:
(proposed):
Line 91: Line 96:
Support for these builds was disabled in the gcc packaging by the maintainer in 4.9.1-19 and removed in 4.9.2-4 so you need to use the cross-gcc-* packages to patch it back and do the builds - you can't just build from gcc-4.9 sources themselves any more. (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=766924)
Line 98: Line 105:
{{{
Line 99: Line 107:
}}}
Line 100: Line 109:
{{{
Line 101: Line 111:
}}}
Line 109: Line 120:
=== in jessie or unstable=== === In jessie or unstable ===
Line 133: Line 144:
pushd cross-gcc-4.9-4.9.*/ pushd cross-gcc-4.9-armhf-4.9.*/

Multiarch Cross Toolchain Build

Cross Toolchains can be built using multiarch mechanisms. Specifically they need the ability to build-depend on foreign-arch packages. (arm-linux-gnueabi-gcc needs libc:armel and linux-libc-dev:armel to build).

This has been essentially working on Debian since end-2013. Packaging and packages were uploaded to Debian unstable in October 2014. Unfortunately these did not migrate into Jessie due to infrastructure requirements (sbuild, wanna-build, britney versions) only satisfied in Jessie, (ReleaseGoals/CrossToolchains). The changes needed to allow multiarch builds on buildds before packages using this technique can be built in the archive were done in sbuild 0.64.3. So packages are available in an external repo. See CrossToolchains for installation details.

How is this different from before

There have been binary cross-toolchains built for Debian for a decade or so now, using various schemes. (toolchain-source, buildcross). The Emdebian project has provided these, and until wheezy buildt them using buildcross, which relies on there already being a suitable glibc package for the target architecture, which is transformed using dpkg-cross so that everything exists in one architecture. The buildcross process is not designed for use on normal buildds due to the cross-arch buildd-deps so these packages have never been available in Debian proper. Hector Oron wrote buildcross and maintained the toolchain builds during etch and squeeze.

Ubuntu (with Linaro sponsorship) took a different approach in order to get toolchains into the archive. armel-cross-toolchain-base is a package which uses binary source packages from binutils, eglibc, linux-kernel and gcc (binutils-source, eglibc-source, linux-<ver>-source, gcc-<ver>-source) to run a 3-stage bootstrap on a normal buildd and thus generate in-archive toolchains. It works, but it's not pretty. This package has support for doing builds on Debian too, although it is only occasionally maintained.

Multiarch lets us simplify the build process by using the normal source packages, but building them for specified targets and stages, with cross-arch dependencies.

Initial work on this was done as a 2012 GSOC project (by Thibaut Girka), and was moerged into the gcc package in Dec 2012, with all the important bits included from gcc-4.7.2-17 on.

Credits:

  • Nikita Youshchenko for toolchain-source and initial Debian packaging cross support
  • Hector Oron (Zumbi): for Buildcross, toolchain build maintenance, GSOC mentoring
  • Marcin Juscewiekcz for cross-toolchain-base and gcc-cross-defaults, GSOC mentoring
  • Wookey: initial emdebian cross-toolchains, GSOC proposals and general badgering
  • Thibaut Girka: initial multiarch toolchain support
  • Mattias Klose: for merging all of this stuff in recent years
  • Dima Kogan: cross toolchain patches and secretsauce repo
  • Helmut Grohne: toolchain bootstrap testing (and patches) with rebootstrap

A related issue is having co-installable native compilers. See CoinstallableToolchains for details.

Repository and patches

Binaries

If you just just want working binaries, you can now (2014/10/22) get them from the main Debian/unstable repo! Note that they are uninstallable for a while (up to a few days on slow architectures) after a new gcc-4.9, libc or linux upload as the multiarch libraries have to be in sync across architectures, so the cross-toolchains need to be rebuilt against the

Sources and development

The alioth cross-toolchain project is (since Debconf 2013) where things are co-ordinated, and code resides: https://alioth.debian.org/projects/crosstoolchain/

Source git repos:

The mailing lists to use are debian-toolchain (toolchain focus) and debian-cross (crossbuilding focus).

Development co-ordination happens on the #debian-toolchain IRC channel on OFTC

If you want to get commit messages on the development there is the alioth crosstoolchain-logs list

Integration work on this is uploaded to The Debian cross-toolchain repo since Nov 2014 (previously in a people.debian.org personal repo since 2013).

GCC patches were first merged into gcc-4.7.2-17. Old GCC patches are available for 4.7.2-4, 4.7.2.-11, 4.7.2-12, 4.7.2-14 and 4.7.2-16 at time of writing.

Packages

Binutils

The cross-binutils source package builds a set of binutils-<triplet> packages (currently for armel, armhf, arm64, mips, mipsel, powerpc, ppc64el), on a set of faster architectures (currently amd64 i386 and arm64). The source comes from the normal binutils packages (as the binutils-source package).

These package were accepted into unstable in June 2014

GCC

There is a cross-gcc-<ver>-<arch> source package for each target <arch>, which builds a set of cross-compiler packages. The set is currently gcc, g++, and gfortran. gobjc and gccgo are enabled for jessie, and will be in unstable. It does not build cross library packages, but uses the native system ones. Those source packages are generated from the cross-gcc git repo above.

In experimental the cross-gcc package itself is in the archive, and its binary cross-gcc-dev is used offline to generate the cross-gcc-<ver>-<arch> sources and at build time to build the binaries. This keeps everything in the archive, and gives one package in the archive to file cross-build packaging bugs against.

Cross-support

A few other packages are needed to tie this together:

  • crossbuild-essential-<arch> - pulls in binutils-<triplet>, gcc-<triplet>, libc-dev:<arch> and dpkg-cross

    • There is one of these for each supported architecture. These packages are built from the build-essential source package. In experimental since Oct 2014
  • cross-gcc-defaults package to set cross-toochain default versions as is done for the native compilers (by installing unversioned links for each binary). In unstable since Oct 2014

(proposed):

  • cross-support package to generate things like one-link pkg-config-<arch> packages, and the useful autoconf and cmake cross-cache info currently in dpkg cross.

LLVM

LLVM crossing works differently to gcc so cross-specific builds are not needed. The standard llvm packages should work with the above binutils-<triplet> packages using --host and --build options. However there are issues with llvm not having debian ABI and ISA defaults encoded anywhere.

Libraries

All library packages are provided by the normal native builds (linux-libc-dev, libgcc1, libstdc++, libc6, and everything higher up the stack).

Local building of multiarch cross-toolchains

In general local building is discouraged. Just use the packages in the archive. However you may want to do this for various reasons, such as building variants not built in the archive, or development.

Support for these builds was disabled in the gcc packaging by the maintainer in 4.9.1-19 and removed in 4.9.2-4 so you need to use the cross-gcc-* packages to patch it back and do the builds - you can't just build from gcc-4.9 sources themselves any more. (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=766924)

Where the target architecture already exists as a Debian port this is simple as you already have a suitable libc:<arch> on which to depend and libc-dev:<arch> and linux-libc-dev:<arch> on which to build-depend.

You need gcc-4.9, gcc-4.8 or gcc-4.7.2-17 or later.

And binutils 2.23.1-1 or later. See original patch in 695595

Note that the mechanism for selecting the target architecture changed (incompatibly) in dpkg 1.17.17. Prior to that you use:

DEB_TARGET_ARCH=armel dpkg-buildpackage ...

from 1.17.17 you use

dpkg-buildpackage --target-arch armel ...

DEB_TARGET_ARCH gets reset when --target-arch is used so you can't carry on using the older version.

First you need a multiarch dpkg environment (as root/with sudo):

dpkg --add-architecture armel
apt-get update

In jessie or unstable

Binutils: If you have a jessie chroot set up:

sbuild -d jessie cross-binutils_<ver>.dsc

or more manually

apt-get build-dep --no-install-recommends cross-binutils
apt-get source cross-binutils
pushd cross-binutils-*/
dpkg-buildpackage -B

GCC 4.9: If you have a jessie chroot set up:

sbuild -d jessie cross-gcc-4.9-armhf_<ver>.dsc

or more manually

apt-get build-dep --no-install-recommends cross-gcc-4.9-armhf
apt-get source cross-gcc-4.9-armhf
pushd cross-gcc-4.9-armhf-4.9.*/
dpkg-buildpackage -B

in wheezy

Binutils:

apt-get build-dep --no-install-recommends binutils
apt-get source binutils
pushd binutils-2.*/
DEB_TARGET_ARCH=armel TARGET=armel dpkg-buildpackage -d -T control-stamp
dpkg-checkbuilddeps

If any dependencies are reported as missing, install them here

WITH_SYSROOT=/ DEB_TARGET_ARCH=armel TARGET=armel dpkg-buildpackage -b
popd

GCC 4.8:

apt-get build-dep --no-install-recommends gcc-4.8
apt-get source gcc-4.8
pushd gcc-4.8-4.8.*/
DEB_TARGET_ARCH=armel DEB_CROSS_NO_BIARCH=yes with_deps_on_target_arch_pkgs=yes dpkg-buildpackage -d -T control
dpkg-checkbuilddeps

If any dependencies are reported as missing, install them here

DEB_TARGET_ARCH=armel DEB_CROSS_NO_BIARCH=yes with_deps_on_target_arch_pkgs=yes dpkg-buildpackage -b
popd

Bootstrapping the toolchain MA-style

This is the process of building the cross-toolchains using multiarch mechanisms for multiarch use, but when the target architecture doesn't yet exist in the archive. So things are complicated by not having libselinux-dev:arm64, libc6-dev:arm64, libgcc1:arm64 or libstdc++-dev when starting off.

Manual full bootstrap toolchain build

Get suitable binutils source (2.23.1 or later), unpack, install build-deps, then build a BUILD_ARCH package with a specified TARGET_ARCH.

TARGET=arm64 dpkg-buildpackage -b

which should produce: binutils-aarch64-linux-gnu_2.23.90.20131017-1_amd64.deb

Get suitable linux kernel source (from bootstrap repo or see 695241 ), unpack. Source has been modified to allow a stage1 build of just linux-libc-dev using minimal build-deps.

DEB_BUILD_PROFILE=stage1 apt-get --only-source build-dep linux

Then build the package for the target arch:

DEB_BUILD_PROFILE=stage1 dpkg-buildpackage -aarm64

which should get you linux-libc-dev_3.7.11-1_arm64.deb (i.e. linux-libc-dev:arm64)

(Note: in ubuntu raring with linux-3.7.0 you need to use either DEB_STAGE=stage1 or DEB_BUILD_PROFILE=bootstrap - this needs sorting)

Now on to the gcc/eglibc 3-stage bootstrap dance: Using a little bit of BUILD_PROFILE support:

Index: gcc-4.7-4.7.2.MAquilt/debian/rules.defs
===================================================================
--- gcc-4.7-4.7.2.MAquilt.orig/debian/rules.defs        2012-12-10 04:48:11.000000000 +0000
+++ gcc-4.7-4.7.2.MAquilt/debian/rules.defs     2012-12-10 18:08:30.000000000 +0000
@@ -214,7 +214,19 @@
 endif

 # -------------------------------------------------------------------
-# stage options
+# stage/build_profile options
+
+ifneq (,$(findstring stage1,$(DEB_BUILD_PROFILE)))
+  DEB_STAGE := stage1
+  DEB_CROSS_NO_BIARCH := yes
+endif
+
+ifneq (,$(findstring stage2,$(DEB_BUILD_PROFILE)))
+  DEB_STAGE := stage2
+  DEB_CROSS_NO_BIARCH := yes
+endif
+
+
 ifdef DEB_STAGE
   with_cdev := yes
   separate_lang := yes

we should be able to build with: GCC_TARGET=arm64 DEB_BUILD_PROFILE=stage1 debuild --preserve-env but it fails with -12 because that version doesn't build libgcc1 any more but does try to package it.

Then eglibc stage1: Get suitable eglibc source

DEB_BUILD_PROFILE=bootstrap apt-get build-dep -aarm64 eglibc
DEB_BUILD_PROFILE=bootstrap dpkg-buildpackage -aarm64

Slick apt-driven bootstrap

This process can be made very slick by using apt-get --build, as that gets sources, unpacks them, and builds them. apt-get build-dep will get all the build-deps needed (but does only work on deps listed in a Sources file so if you've changed them locally it doesn't do what you want.

This could be the basis for automation of the process.

First add the target architecture

dpkg --add-architecture arm64

Binutils: (no source changes)

 apt-get build-dep binutils
 TARGET=arm64 apt-get --build -oDpkg::Build-options="-kBEA7C52D -b" source binutils

Binutils should be fixed to understand the standard DEB_TARGET_ARCH (see DebBug:#695595)

Kernel-headers: (this works unpatched in jessie (from 3.11.5 onwards))

 DEB_BUILD_PROFILE=stage1 sudo apt-get --only-source build-dep linux
 DEB_BUILD_PROFILE=stage1 apt-get --only-source --build -oDpkg::Build-options="-aarm64 -kBEA7C52D -B" source linux

gcc stage 1 build: (needs at least version 4.7.2-13~multiarch)

 DEB_BUILD_PROFILE=stage1 sudo apt-get --only-source build-dep gcc-4.8
 DEB_BUILD_PROFILE=stage1 DEB_TARGET_ARCH=arm64 apt-get --build -oDpkg::Build-options="-kBEA7C52D -B" source gcc-4.8

What actually happens here is the equivalent of DEB_STAGE=stage1 DEB_CROSS_NO_BIARCH=yes BACKPORT=false DEB_TARGET_ARCH=arm64 dpkg-buildpackage

eglibc stage 1 build: (needs version 2.16)

Consistent target specifications

Packages that are built for a target architecture (because they generate code) need a way to specify that. This has been a little haphazard to date. For this to be automatable by build tools which don't necessarily know the specifics of every package we need a bit more regularity.

binutils uses TARGET=<arch> or a file debian/target. (<arch here can be Debian or GNU name)

gcc uses DEB_GCC_TARGET (or GCC_TARGET)=<arch> or a file debian/target. (<arch> here is Debian name)

Using a consistent variable of the same form as the BUILD and HOST vars makes sense as a way of making this consistent. e.g. DEB_TARGET_ARCH=arm64

It should also be possible to set the target arch in dpkg-buildpackage, as is currently possible for the host arch. a --target-arch=<arch> seems the obvious way. An unfinished patch for this exists.

A bug for binutils to honour DEB_TARGET_ARCH has been filed.

dpkg gained the option to specify the target with --target-arch in version 1.17.17. This overrides DEB_TARGET_ARCH in the environment because it sets it itself so you have to use the correct syntax for the dpkg version. This is a useful make snippet for testing it:

DPKG_HAS_TARGETARCH = $(shell (cd /; dpkg-buildpackage --target-arch; if [ $$? -eq 2 ] ; then echo 'no'; fi))
ifeq ($(DPKG_HAS_TARGETARCH),no)
 use DEB_TARGET_ARCH=arch dpkg-buildpackage
else
 use dpkg-buildpackage --target-arch arch
endif

Consistent BUILD_PROFILE specifications

DEB_STAGE=stage1, stage2 has historically been used in the toolchain builds. It makes sense to continue to use the 'stage1' 'stage2' labels for all staged builds with the more flexible DEB_BUILD_PROFILE mechanism. But we currently (Nov 2013) have some divergence. Linux in Debian uses DEB_BUILD_PROFILE=stage1, linux in ubuntu uses DEB_BUILD_PROFILE=bootstrap. eglibc understands both DEB_BUILD_PROFILE=bootstrap and DEB_STAGE=stage1 (but not DEB_BUILD_PROFILE=stage1). Gcc uses DEB_STAGE=stage, stage2. This page is the spec for build profiles, which includes the set of defined profiles.

Normal cross-toolchains builds

Normally we will already have a target_arch glibc and libgcc1 for the HOST architecture, so the first two stages can be skipped.

Things that go wrong

There are lots of ways that building the cross-toolchain can not work. toolchain/BootstrapIssues collects known issues.