Differences between revisions 31 and 32
Revision 31 as of 2012-11-20 01:56:47
Size: 19558
Editor: wookey
Comment:
Revision 32 as of 2012-11-20 02:27:26
Size: 19624
Editor: wookey
Comment:
Deletions are marked like this. Additions are marked like this.
Line 150: Line 150:
|| eglibc || 2.16-0arm64.1 || 2.16-0ubuntu3 || || eglibc || DebianBug:690873 2.16-0arm64.1 || 2.16-0ubuntu3 ||
Line 154: Line 154:
|| dpkg-cross || 2.6.7arm64 || 2.6.7arm64 || || dpkg-cross || DebianBug:693730 2.6.7arm64 || 2.6.7arm64 ||
Line 168: Line 168:
|| python-2.7 || || || || python-2.7 || DebianBug:683755 || ||
Line 174: Line 174:
|| dctrl-tools || || || || dctrl-tools || DebianBug:693474 || ||

'arm64' is the Debian port name for the 64-bit ARMv8 architecture, referred to as 'aarch64' in upstream toolchains (GNU triplet aarch64-linux-gnu)

There will be no hardware in existence until sometime in 2013 but ARM (via Linaro) are working well with the community and trying to get this port done early so software will exist when there is something to run it on.

Status

Just started, there are cross-toolchains, and base system package building is ongoing. There is not yet a (Debian/Ubuntu) image to run.

Binutils, kernel, gcc and glibc port patches were sent upstream over the summer of 2012, with enough stuff to build a cross-toolchain available by October.

Bootstrap package repos are here. Only the Ubuntu repo currently has much in it.

Bug tracking

Please tag all arm64/aarch64-specific bugs 'arm64' in the BTS (see bugs.debian.org/usertags for instructions), or in launchpad.

Here is the current list: http://bugs.debian.org/cgi-bin/pkgreport.cgi?users=debian-arm@lists.debian.org;tag=arm64

Upstream-relevant bugs should be filed in the linaro-aarch64 bug tracker to avoid distro duplication of effort: https://bugs.launchpad.net/linaro-aarch64 File bugs in the Debian BTS too, then link to them in the Linaro tracker.

Repository

Quantal and unstable armv8 bootstrap repos have been set up here. These contain current state of play for modified tools, sources, cross-tools, and packages built for arm64. Notable packages include eglibc2.16, gcc-4.7, linux-source, arm64-cross-toolchain-base, gcc-4.7-aarch64-linux-gnu, dpkg (with build profile support).

You should just be able to add that repo and apt-get install crossbuild-essential-arm64 to get a working cross environment. If build and host arch packages have got out of sync this won't work.

Cross Toolchain

Before anything useful can happen dpkg needs to have arm64 support. This was done in v1.16.4

Bootstrap

First job is to bootstrap a cross-toolchain so other stuff can be built.

This is relatively easy to do from upstream sources either directly or using a framework like ?OpenEmbedded, but to get a Debian-packaged toolchain easily usable in chroots, sbuild etc requires merging the new port with the Debian packaging, which is frankly a PITA.

When starting from scratch multiarch doesn't help you because you need eglibc:arm64 to make aarch64-linux-gnu-gcc and you need aarch64-linux-gnu-gcc to make eglibc:arm64. So you have to do a 3-stage bootstrap.

Ubuntu has (thanks to Linaro) a package to automate the building of fully-bootstrapped compilers for armhf/el so that was munged to make arm64-cross-toolchain-base, and the delighful task of getting everything working was started.

The process is:

  • Make a binutils with aarch64 port and arm64 packaging in so you can build binutils-source
  • Make a kernel with arm64 stuff in so you can build kernel-source
  • Make a gcc with aarch64 gcc port and arm64 packaging in so you can build gcc-source
  • Make an eglibc with aarch64 port and arm64 packaging in so you canbuild eglibc-source
  • Make arm64-cross-toolchain-base with the right runes in it to build:
    • munged linux-libc-dev kernel headers
    • gcc stage1 bare cross c-compiler
    • eglibc stage1 simple libc
    • gcc stage2 compiler and cross-compiler and libraries
    • eglibc stage 2 (full) build
    • gcc stage3 (full) build

Easy peasy.

This work was initially done in Ubuntu due to better multiarch support, availability of *-cross-toolchain-base and newer eglibc.

Bootstrap Issues

Binutils was straightforward as the Quantal package already had aarch64 support and arm64 packaging. It just builds :-)

Linux-source required some work. linux-3.5.0 is current in quantal. Upstream work was on 3.6 and 3.7, although initially posted patches from July were for 3.5. Only headers are needed and base port is not invasive so version isn't too important. Taking the port from Linaro current armv8 tree http://git.linaro.org/gitweb?p=people/omarrmz/linux-aarch64.git;a=summary and adding Debian arm64 packaging in various places had only one bit of hackery needed (remove mention of drivers/irqchip from arm64/kconfig) to get a linux-source package built. The initial patch is here:

eglibc in Ubuntu is 2.15. supplied glibc patches were for trunk and then 2.16. eglibc2.15 does not build with gcc4.7, only 4.6. Initial gcc patches were for trunk and then 4.7. There was no public 4.6 port. eglibc2.16 was not available in Debian or Ubuntu, but there was a set of packaging in alioth, but no actual glibc source. eglibc Debian source layout is, for historical reasons not like upstream so a load of munging is needed to get the right layout. I did this hard way before dicovering the debian rules target to do it for me better. Then remove the gcc4.6 deps and rules-forcing. Add in glibc aarch64 patch and poke it till it applies. Fortunately glibc and eglibc are very similar here. Grok packaging layout and patch-naming scheme. Fix up submitted-fcntl-bits patch which moves AT_ GNU symbols and needs to put them back in again for each arch specifically. Turn off libselinux when building stage1 (on by default for linux in 2.16 packaging) Spend some time tracking down why builds always fail with EI_foo aliased to external symbol GI_foo errors 680439. Turns out to be gcc4.6 vs 4.7 issue, which boils down to pr33763, the patch for which breaks eglibc builds. So this actually needs to be fixed in gcc. Some hassling got this done quickly with a new pr33763 patch to put in gcc.

gcc packaging is complicated. Lots of patches applied for different arches, distros, options. Only the core C, C++ and fortran compliers are supported in initial port, so libssp, libmudflap, go, java, objectiveC, lidquadmath, ada, multilib etc all need to be disabled. pr33763 fix as above in order to even build eglibc stage1.

Next the gcc stage2 build fails with 'cannot find bits/predefs.h'. Some investigation finds that this is due to xgcc looking in the wrong place for lib headers. These are put in debian/tmp/ and a sysroot is set in order to find them, but the headers path given a) doesn't look in the sysroot and b) is for old /usr/<triplet>/include path instead of new multiarch /usr/include/<triplet>:/usr/include path. Work out where to poke this in multiple packaging layers.

Then libgcc.so ends up in non-multiarch dir. Turns out to be misapplication of part of Thibgs 'use multiarch paths everywhere', which I tried too late at night and forgot about.

The selinux still gets used for eglibc stage build. We still don't have one so patch changed to always skip selinux use if DEB_PROFILE=bootstrap (and set this in the arm64-cross-toolchain-base build for stage2 full build as well as stage1 - this needs doing properly).

Eventually get a toolchain - woohoo.

After this building extra c++ packages was straightforward.

And this can be installed in a chroot and used to build simple things like zlib (woohoo) but there is an issue. The c-library is provided in the old dpkg-cross location, not as libc6:arm64 . This means that anything non-trivial, i.e. that build-depends on another library, which itself has libc as an install dependency can't be built as the bottom of the multiarch library chain is missing. So we need to cross-build a real libc:arm64. (It could be faked up with equivs but this will cause problem with finding headers/libs later - in fact this whole toolchain mess is a bit of a bodge that needs sorting properly).

Building eglibc for arm64 is a simple matter of DEB_BUILD_PROFILE=bootstrap DEB_BUILD_OPTIONS=nocheck dpkg-buildpackage -d -aarm64 -kBEA7C52D -b (the -d is there because otherwise gettext gets in the way (it should be gettext:any in ubuntu))

This fails because the native part of the build (for getent and rpcgen) can't find errno.h. This is because libc6-dev:amd64 isn't installed (because it's not specified as an actual build-dep of eglibc, even though it's needed when crossing - we need to fix this). But this can't just be installed because an attempt to do so causes apt to remove a lot of essential packages (libnih, adduser, mount, rsyslog etc). This turns out to be because libnih1 has these dependencies: Pre-Depends: multiarch-support, libc6 (>> 2.15), libc6 (<< 2.16) which prevents libc 2.16 being installed. This is almost certainly a bug in eglibc's shlibdeps overrides. The easy fix is to build a version of libnih without this restriction, at which point libc6-dev:amd64 can be installed and eglibc can be cross-built for arm64.

Rebuilding eglibc:amd64 mean we now have matching libc6 for amd64 and arm64 and crossbuild-essential-arm64 should be installable, which opens the way for automated sbuild crossbuilds.

However, this wasn't quit enough as libgcc1:arm64 and linux-libc-dev:arm64 are needed to install libc6:arm64. We already have a libgcc1-arm64-cross with the right files in, but not the right layout, architecture, multiarchness or package name. The toolchain has support for a 'reverse-cross' to get the needed libgcc1 but it seemed easier to just mangle libgcc1-arm64-cross to make the right package. Linux-libc-dev is easiest to get by doing DEB_STAGE=stage1 build of the kernel. The kernel package version in quantal has moved on so the amd64 linux-libc-dev is newer. Bah - multi-arch fail. So build one of those too.

The toolchain bootstrap process needs to spit out real multiarch library packages to avoid this faffage. But finally libc6:arm64 is installable which means crossbuild-essential-arm64 is.

Building packages

Given a cross-build chroot, in general you should be able to do

apt-get build-dep -aarm64 <package>
apt-get source <package>
cd <package>-<ver>
CONFIG_SITE=/etc/dpkg-cross/cross-config.arm64  DEB_BUILD_OPTIONS=nocheck dpkg-buildpackage -aarm64

But there are of course some caveats.

Details on setting up an arm64 cross-chroot are given here: https://wiki.linaro.org/Platform/DevPlatform/CrossCompile/arm64bootstrap

There is no libssp (stack protection) support in aarch64 toolchain yet so the -fstack-protector will cause an error:

/usr/lib/gcc/aarch64-linux-gnu/4.7/../../../../aarch64-linux-gnu/bin/ld: cannot find -lssp_nonshared
/usr/lib/gcc/aarch64-linux-gnu/4.7/../../../../aarch64-linux-gnu/bin/ld: cannot find -lssp

For any package that used dpkg-buildflags this is easy to fix by setting DEB_BUILD_MAINT_OPTIONS=hardening=-stackprotector You can set this for the whole build chroot if using one (probably best for now), or for a user, or for a build by doing:

DEB_BUILD_MAINT_OPTIONS=hardening=-stackprotector dpkg-buildpackage -aarm64

Better to fix this permanently in /etc/dpkg/buildflags:

STRIP CFLAGS -fstack-protector

The initial version of the cross-toolchain only searches the old dpkg-cross library paths by default so you need this in /etc/dpkg/buildflags too:

DEB_LDFLAGS_APPEND="-L/usr/lib/aarch64-linux-gnu -L/lib/aarch64-linux-gnu -L/usr/lib -Wl,-rpath-link=/usr/lib/aarch64-linux-gnu:/lib/aarch64-linux-gnu:/usr/lib"

Package porting

Many packages need no more than config.guess and config.sub updating to build for arm64. autotools got aarch64 support upstream in June 2012. So autotools-dev 20120608.1 has the files needed. However many (most) packages don't autoreconfi on build, or otherwise update those files so they have to be patched. A lot of boring patches were filed with the tag arm64, such as 689613

Because there is no hardware everything needs to be cross-built so cross-building patches are also needed in many packages, and multiarch patches in many build-deps. Due to the Wheezy freeze and thus the more progressed state of multiarchification in Ubuntu, this work was initialy done in a private Quantal-based repo, but all bugs are either directly filed, or also pushed upstream to Debian.

A few packages need actual changes. Here is a list so far:

Packages that need arm64-related changes

package

Debian Bug/Fixed version

Ubuntu Bug/Fixed version

dpkg

672408

autotools

20120210.1

20120210.1

binutils

binutils-2.22.90.20120924

gcc

gcc-4.7-4.7.2-3

gcc-4.7-4.7.2-3ubuntu1arm64

eglibc

690873 2.16-0arm64.1

2.16-0ubuntu3

linux

1063895

util-linux

689607

perl

dpkg-cross

693730 2.6.7arm64

2.6.7arm64

(cross)-build-essential

693220

gmp

693467

1079831

libffi

Packages that need crossbuild fixes

package

Debian Bug

Ubuntu Bug

diffutils

1:3.2-3

1:3.2-1ubuntu1

tzdata

gdbm

packages that need mulitarch-related changes

package

Debian Bug

Ubuntu Bug

perl

633884

python-2.7

683755

chrpath

0.14

0.14

less

693318

check

693221

gettext

683751

ed

dctrl-tools

693474

indent

fdupes

Packages that need changes for eglibc 2.16

package

Debian Bug

Ubuntu Bug

diffutils

693346

1079770

tar

693352

1079750

gettext

693361

1079768

cpio

693440

1079748

coreutils

m4

Packages that just needed config.{guess,sub} updates

package

Debian Bug

Ubuntu Bug

acl

689610

Quantal

coreutils

689611

Quantal

cpio

689612

Quantal

db

689613

Quantal

db-4.8

dialog

689615

Quantal

diffutils

689617 1:3.2-7

1:3.2-7

dropbear

expat

689619

Quantal

findutils

689620

Quantal

gnupg

Quantal

libgpg-error

689621

Quantal

libnih

689622

Quantal

libx11

689623

Quantal

libxml2

689624

Quantal

libxslt

689625

Quantal

make-dfsg

689626

Quantal

module-init-tools

689627

Quantal

nano

689628

Quantal

ncurses

Quantal

patch

Quantal

pcre3

Quantal

popt

Quantal

sed

Quantal

Packages that are built

  • pcre3
  • attr
  • acl
  • zlib
  • hostname
  • libsepol
  • gpm
  • hostname
  • ifupdown
  • insserv
  • module-init-tools
  • python-defaults
  • libselinux (STAGE1 to avoid swig and rub2deb)
  • ncurses
  • bash
  • libpciaccess
  • libpng
  • slang (needs chrpath 0.14)
  • base-files
  • mawk
  • makedev
  • make
  • xz-utils
  • dpkg
  • netbase
  • base-files
  • time
  • manpages
  • libxau
  • libxdmcp
  • libpthread-stubs
  • diffutils
  • tar
  • cpio
  • diffutils
  • findutils
  • gmp
  • gdbm
  • sed
  • tzdata
  • base-passwd
  • coreutils

Packages that failed to build

debconf and base-passwd:
cd po4a && po4a po4a.cfg
Can't locate Encode.pm in @INC
cd man && po4a --rm-translations po4a/po4a.cfg
Can't locate Encode.pm in @INC
(Multiarch cross-perl issue?)

debianutils:
dpkg-shlibdeps debian/tmp/bin/run-parts \
    debian/tmp/bin/tempfile \
    debian/tmp/usr/bin/ischroot
Can't locate Storable.pm in @INC 
(Multiarch cross-perl issue?)

libusb-0.1.12:
aarch64-linux-gnu-strip: Unable to recognise the format of the input file `debian/libusb-0.1-udeb/lib/libusb-0.1.so.4.4.4'

libgcrypt11:
Compiler ICE: https://bugs.launchpad.net/linaro-aarch64/+bug/1078967

procps:
configure: error: ncurses support missing/incomplete (for partial build use --without-ncurses)
(but it is installed)



klibc: 
aarch64-linux-gnu-gcc: error: unrecognized command line option '-m64'


sysvinit:
aarch64-linux-gnu-gcc   init.o init_utmp.o  -lsepol -lselinux  -o init
/usr/lib/gcc/aarch64-linux-gnu/4.7/../../../../aarch64-linux-gnu/bin/ld: cannot find -lsepol
/usr/lib/gcc/aarch64-linux-gnu/4.7/../../../../aarch64-linux-gnu/bin/ld: cannot find -lselinux
(but they are installed)


util-linux:
libtool: link: aarch64-linux-gnu-gcc -std=gnu99 -fsigned-char -I../libblkid/src -g -O2 --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z -Wl,relro -Wl,-rpath-link=/usr/lib/aarch64-linux-gnu:/lib/aarch64-linux-gnu:/usr/lib -o .libs/fdisk fdisk-fdisk.o fdisk-fdiskbsdlabel.o fdisk-fdisksgilabel.o fdisk-fdisksunlabel.o fdisk-fdiskaixlabel.o fdisk-fdiskmaclabel.o fdisk-partname.o fdisk-i386_sys_types.o fdisk-gpt.o fdisk-blkdev.o fdisk-wholedisk.o fdisk-mbsalign.o fdisk-linux_version.o fdisk-canonicalize.o  -L/usr/lib/aarch64-linux-gnu -L/lib/aarch64-linux-gnu -L/usr/lib ../libblkid/src/.libs/libblkid.so
/usr/lib/gcc/aarch64-linux-gnu/4.7/../../../../aarch64-linux-gnu/bin/ld: warning: libuuid.so.1, needed by ../libblkid/src/.libs/libblkid.so, not found (try using -rpath or -rpath-link)
../libblkid/src/.libs/libblkid.so: undefined reference to `uuid_unparse@UUID_1.0'

libuuid is in ../libuuid/src/.libs/libuuid.so but that's not listed. Adding  ../libuuid/src/libuuid.la on the end of command line fixes it. how does native build find it? rpath? (whcih we've overridden?)

Build-dependencies that need fixes

Mostly things need multiarchifying.

  • Python-2.7
  • perl
  • chrpath (0.14, earlier versions are not 32/64/endian aware)

Using dose to see what is multiarch-buildable

Dose 3.1 can analyse build-dependencies for a given package, and understands about cross and multi-arch. It can tell you what is currently buildable given the current state of relevant source and package files. Use 3.1.2 for the --checkonly option and the --defaultedMAforeign option

Install it (it's in the bootstrap tools repo)

sudo apt-get install dose-builddebcheck
  • -f shows packages you can't build
  • -s shows packages you can build
  • -e prints an explanation of what the problem is
  • --checkonly checks just one package rather than everything
  • --defaultedMAforeign uses the Ubuntu apt algorithm (not in wheezy) of assuming all arch:all Build-Deps can be considered Multi-Arch:Foreign

To check

dose-builddebcheck -f --checkonly <package> \
--defaultedMAforeign --deb-native-arch=amd64 --deb-foreign-archs=arm64 --deb-host-arch=arm64 \
/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_quantal_main_binary-amd64_Packages \
/var/lib/apt/lists/people.debian.org_%7ewookey_bootstrap_ubunturepo_dists_quantal-bootstrap_main_binary-amd64_Packages \
/var/lib/apt/lists/people.debian.org_%7ewookey_bootstrap_ubunturepo_dists_quantal-bootstrap_main_binary-arm64_Packages \
/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_quantal_universe_source_Sources | grep source

Current list of things needed for debootstrap which have missing build-deps:

  • db : default-jdk:arm64 staging this to build without java would be good
  • dbus : libdbus-glib-1-dev:arm64
  • e2fsprogs : libblkid-dev:arm64
  • gnupg : libldap2-dev:arm64