'arm64' is the Debian port name for the 64-bit ARMv8 architecture, referred to as 'aarch64' in upstream toolchains (GNU triplet aarch64-linux-gnu)

Hardware (that actually works) started to becoming available in October 2013, but access was restricted. Debian has very kindly been donated a machine which is now (March 2014) running as a buildd in Debian-ports. The port was started (by ARM and Linaro working with the community in commendable fashion) long before hardware was available so that there would be something to run when it arrives.

Hardware, emulators and models

Arm64 hardware is starting to become available in Q42013, but is not commercically available yet.

Arm64 qemu emulation (user-space only) became initially available in October 2013. It was added to qemu upstream in v2.0. This is much faster than the model for building software, but can't be used for kernel-space things like bootloader/kernel work, or anything that uses multithreading. See Qemu Rootfs below for details.

There is a (non-free, free beer) 'Foundation Model' simulator which can be used to run arm64 code, which is available here

Much of the work packagers need to worry about is updating autoconf, multiarch and cross-build fixes, with a few actual arm64 changes here and there, and doesn't actually need a model or emulator.

This is the first non-x86 self-bootstrapped Debian port: the first 150 packages cross-built using build-profiles to untangle cyclic build-dependencies.

Status

(last updated: March 2014)

Debian unstable is being bootstrapped in debian-ports: http://buildd.debian-ports.org/status/architecture.php?a=arm64&suite=sid

Current blockers are reported at: http://people.debian.org/~zumbi/arm64/index.unstable.html

We hope the port will be in a fit state to release as an official architecture in Jessie, but right now there is still a lot to do.

Detailed status is below.

The initial port was done entirely as a cross-build using Ubuntu raring packages. It was done in Ubuntu because multiarch and the cross-toolchains were more advanced there than in Debian at the time, and Debian was frozen for Wheezy. The resulting image and updates were then used to natively build most of Saucy once real hardware that worked was available (to canonical). Then that image was used (in Qemu) to natively build a Debian buildd image.

Ubuntu Saucy and Trusty are 3/4 built (30,000 binaries) for arm64. The missing stuff is largely unported languages (haskell, mono, ruby, go, etc).

The initial Debian bootstrap (350 source, 2100 binaries) was built in a [[http://people.debian.org/~wookey/bootstrap.html|personal repo ). Debian arm64 is now debootstrappable from there.

This bootstrap was done natively starting with an Ubuntu Saucy base to supply a base chroot and non-library build-deps, lsb+dpkg-vendor was set to 'Debian', each package was built, uploaded and added to a Debianise script which replaced all available packages in the Saucy clean tarball chroot until it contained no Ubuntu packages and was effectively Debian unstable. Then a new clean debian chroot was deboostrapped and packages built/rebuilt in there using build profiles to break cycles.

There are crosstoolchains built for Raring/Saucy/Trusty. Debian cross-toolchains are being worked on.

Binutils, kernel, gcc and glibc port patches were sent upstream over the summer of 2012, with enough stuff to build a cross-toolchain available by October 2012. In Oct/Nov/Dec a quantal-based archive of base packages was built. In Jan 2013 the effort was switched to raring to take advantage of cross-build fixes, multiarch improvements, libc and arm64 updates going in there.

Current cross-buildability (in Ubuntu) for arm64 is tracked on this status page. Native buildability is tracked at http://qa.ubuntuwire.org/ftbfs/arm64.html.

Pre-built Rootfses

There are a selection of rootfses here in the form of tarballs and disk images, with and without qemu-static installed.

plain tarball rootfs

Ubuntu saucy multiarch tarball suitable for use as a chroot on arm64 hardware. Identical in content to the foundation model disk image below. Adding qemu-static would make it useful under qemu. Configured for armhf/arm64 multiarch use.

qemu rootfs

Debian: unstable debootstrap+build-essential suitable for use with arm64 qemu (88MB)

Ubuntu: Ubuntu saucy tarball chroot (68MB), suitable for use with arm64 qemu (qemu-arm64 (static) is installed inside it).

See Arm64Qemu for details of building QEMU for arm64 and setting it up. As of 2014-02-17 arm64 qemu is now in qemu git master (at 1.7.50). This is packaged and built, but it doesn't actually work for me (under investigation).

There is a script to enable sbuild builds using this chroot, overlayed with all currently available Debian packages.

Using The Foundation model

See https://wiki.debian.org/Arm64Port/raringbootstrap#Raring_rootfs_for_Foundation_model

Here is a Ubuntu saucy disk image (92MB compressed, 2G uncompressed). Suitable for use with the foundation model or real hardware. This image is configured for armhf/arm64 multiarch usage, but it's a one-liner to turn that off.

And here's a suitable kernel (3.13, linaro foundation release 2014-01), modified for use with Linux instead of OE.

Debootstrap arm64

Plain debootstrap will now produce an arm64 rootfs from wookey's bootstrap repo. libselinux is missing from the debian-ports unstable suite until openjdk is built (it's in unreleased but debootstrap is too dim to find it there), so debootstrap from there needs libselinux copied over manually for now.

Run natively if you have hardware, qemu or a model): sudo debootstrap --no-check-gpg unstable debian-arm64-chroot http://people.debian.org/~wookey/bootstrap/debianrepo2/

Then change your sources.list to http://ftp.debian-ports.org

deb http://ftp.debian-ports.org/debian/ unstable main
deb http://ftp.debian-ports.org/debian/ unreleased main

and upgrade to get to a current set of kosher, and unstable-tracking packages

It's useful to include this as well until there is nothing left in there that's not also in debian-ports (should be true by end-april 2014)

deb http://people.debian.org/~wookey/bootstrap/debianrepo2 debianstrap main

Porting packages for arm64 - Maintainer info

If your package does not build for arm64 it is usualy trivial to fix, but sometimes a bit harder and sometimes a big deal. Here is info on what to change and where to go for help, updates and info.

autoconf updating

Most packages just need updated autoconf config info which includes the new arch. It is now best practice for (autoconf-using) Debian packages to re-autoconf on every build in order to pick up this info automaticaly, or at least use the autotools-dev package to ensure latest config.sub and config.guess files. Good info on making these updates is on https://wiki.debian.org/qa.debian.org/FTBFS#A2014-01-21_using_dh-autoreconf_during_the_build

Packages that use CDBS (and autoconf) will automatically get updated config.sub and config.guess if autotools-dev is installed (it will not be automatically on buildds unles your package build-depends it (see Debianbug:744915) so until a better solution is found you'll need to add it to your build-deps).

Nomenclature and defines

If your package does architecture-specific things explicitly then you will need to understand what names to use in tests.

The gnu name for the architecture (as given to configure) is aarch64-linux-gnu.

The debian name for the architecture is arm64

GCC defines __aarch64__ for the architecture.

Be careful of things which check for arm* in debian architecture tests, as it is usually wrong to do the same thing for arm64 as for arm. In general, if you are not sure you should do the same thing as amd64 as that matches quite closely (64 bit, little endian, 64-bit pointers and floats, 128-bit doubles).

Check the link below for 'upstream package porting' to see if your package has had porting attention from Linaro.

There is also a big-endian version of the architecture/ABI: aarch64_be-linux-gnu but we're not supporting that in Debian (so there is no corresponding Debian architecture name) and hopefully will never have to. Nevertheless you might want to check for this by way of completeness in upstream code.

Useful porting docs

Bug tracking

Please tag all arm64/aarch64-specific bugs 'arm64' (user=debian-arm@lists.debian.org, tag=arm64 - ) in the BTS (see bugs.debian.org/usertags for instructions), or in launchpad.

Here is an example (assuming you have a patch file, and a body template file)

reportbug $package  -V $version -A $patchfile --src --subject "Add arm64 support" --tag patch --pseudo-header 'User: debian-arm@lists.debian.org' --pseudo-header 'Usertag: arm64' --no-tags-menu --severity normal --body-file $template

Here is the current list of arm64 Debian bugs

Here is the current list of arm64 Ubuntu bugs

Upstream-relevant bugs should be filed in the linaro-aarch64 bug tracker to avoid distro duplication of effort. In this case, file bugs in the Debian BTS as well, then link to them in the Linaro tracker.

Repository

Raring, and unstable armv8 bootstrap repos have been set up here. These contain current state of play for modified tools, sources, cross-tools, and packages built for arm64. Notable packages include eglibc2.16, gcc-4.7, linux-source, arm64-cross-toolchain-base, gcc-4.7-aarch64-linux-gnu, dpkg (with build profile support), multiarch versions of perl and python2.7. A quantal chroot was used for initial work but is now (Dec 2012) frozen/abandonned.

You should just be able to add that repo and apt-get install crossbuild-essential-arm64 to get a working cross environment. If build and host arch packages have got out of sync this won't work, which happens often in raring and unstable.

Cross Toolchain

Before anything useful can happen dpkg needs to have arm64 support. This was done in v1.16.4

Toolchain Bootstrap

First job is to bootstrap a cross-toolchain so other stuff can be built.

This is relatively easy to do from upstream sources either directly or using a framework like ?OpenEmbedded, but to get a Debian-packaged toolchain easily usable in chroots, sbuild etc requires merging the new port with the Debian packaging, which is frankly a PITA.

When starting from scratch multiarch doesn't help you because you need eglibc:arm64 to make aarch64-linux-gnu-gcc and you need aarch64-linux-gnu-gcc to make eglibc:arm64. So you have to do a 3-stage bootstrap.

Ubuntu has (thanks to Linaro) a package to automate the building of fully-bootstrapped compilers for armhf/el so that was munged to make arm64-cross-toolchain-base, and the delighful task of getting everything working was started.

The process is:

Easy peasy.

This work was initially done in Ubuntu due to better multiarch support, availability of *-cross-toolchain-base and newer eglibc.

Toolchain Bootstrap Issues

Quantal bootstrap

The initial arm64 packaged toolchain was done in Ubuntu Quantal, as the first release where multiarch crossbuilding basically worked. The details of this are documented on toolchain/BootstrapIssues#Quantal.

Raring bootstrap

Easier than quantal as all the base porting has already been done. However not quite as easy as one might like. arm64-cross-toolchain-base was updated with some fixes/updates from the armhf/armel-c-t-b packages. Binutils and linux-libc-dev headers were straightforward.

But gcc stage1 build fell over first with bugs in gcc-4.7.2-12ubuntu2 (patches that didn't apply, missing arm64 symbols-files). Then with missing bits/predefs.h header when building libgcc. This turned out to be 1091823.

Then the build completed but there was a libgconv.a packaging error for some reason. Updating to gcc-4.7-2-14ubuntu2 seemed to fix that.

Debian bootstrap

The Debian toolchain bootstrap is being attempted fully multiarch, to avoid the above issues with needing a separate libc:arm64 and equivs packages to make everything work. It's also an opportunity to integrate DEB_BUILD_PROFILE boostrap features to make it all nicely automatable, without toolchains being a special case in the archive. This was initially (December 2012) too troublesome, mostly due to new versions of GCC coming out every few days with this stuff being changed. As of gcc-4.7.2-18 onwards ThibG's patches for the toolchain using the multiarch system libraries are integrated so I had another go.

This process is documented on MultiarchCrossToolchains.

Current status is that experiemntal is being used as the stuff in unstable is too old.

binutils and gcc stage1 are built.

The kernel header patches are being updated for 3.8

Building packages

Given a cross-build chroot, in general you should be able to do

apt-get build-dep -aarm64 <package>
apt-get source <package>
cd <package>-<ver>
CONFIG_SITE=/etc/dpkg-cross/cross-config.arm64  DEB_BUILD_OPTIONS=nocheck dpkg-buildpackage --preserve-env -aarm64

But there are of course some caveats.

Details on setting up an arm64 cross-chroot are given here: https://wiki.linaro.org/Platform/DevPlatform/CrossCompile/arm64bootstrap

There is no libssp (stack protection) support in aarch64 toolchain yet (Nov 2013, https://cards.linaro.org/browse/TCWG-23) so the -fstack-protector will cause an error:

/usr/lib/gcc/aarch64-linux-gnu/4.7/../../../../aarch64-linux-gnu/bin/ld: cannot find -lssp_nonshared
/usr/lib/gcc/aarch64-linux-gnu/4.7/../../../../aarch64-linux-gnu/bin/ld: cannot find -lssp

For any package that uses dpkg-buildflags this is easy to fix by setting DEB_BUILD_MAINT_OPTIONS=hardening=-stackprotector You can set this for the whole build chroot if using one (probably best for now), or for a user, or for a build by doing:

DEB_BUILD_MAINT_OPTIONS=hardening=-stackprotector dpkg-buildpackage -aarm64

Better to fix this permanently in /etc/dpkg/buildflags:

STRIP CFLAGS -fstack-protector

Package porting

Many packages need no more than config.guess and config.sub updating to build for arm64. autotools got aarch64 support upstream in June 2012. So autotools-dev 20120608.1 has the files needed. However many (most) packages don't autoreconfig on build, or otherwise update those files so they have to be patched. A lot of boring patches were filed with the tag arm64, such as 689613

In many cases dh_autoreconf can be used to fix this without making horrible, unreadable, autofoo patches. Sometimes it's better to use autotools-dev which only updates config.{sub,guess} without re-generating all the configury because this is less invasive. dh-autoreconf is preferred as it covers more cases (for example the ppc64el port need re-libtooling, so dh-autoreconf works, but the autotools-dev dh_updateconfig is not sufficient. Some packages won't build a second time after dh_autoreconf has poked them around. This QA page goes into more details about updating your packages

For the initial botstrap everything had to be cross-built as there was no hardware (and no OS) so cross-building patches were also needed in many packages, and multiarch patches in many build-deps. Due to the Wheezy freeze and thus the more progressed state of multiarchification and more cross-build fixes in Ubuntu, this work was initialy done in a private Quantal-based repo, and later in a raring-tracking repo, but all bugs are either directly filed, or also pushed upstream to Debian. There are still (April 2014) a lot of cross-build fixes that are in Ubuntu but not yet uploaded to Debian.

Quite a few packages need actual changes. Here is a list so far:

Packages that needed arm64-related changes

package

Debian Bug/Fixed version

Ubuntu Bug/Fixed version

dpkg

672408

autotools

20120210.1

20120210.1

binutils

binutils-2.22.90.20120924

coreutils

698330

gcc

gcc-4.7-4.7.2-3

gcc-4.7-4.7.2-3ubuntu1arm64 1133104

eglibc

690873 2.16-0arm64.1

2.16-0ubuntu3 2.17-0ubuntu4 1120810

fftw3

734675

1267107

linux

695241

1063895

util-linux

689607

perl

https://github.com/codehelp/perl-cross-debian/tree/master/aarch64-linux-gnu

dpkg-cross

693730 2.6.8

2.6.7arm64

(cross)-build-essential

693220

11.6ubuntu4

gmp

693467

1079831

kde-pkg-tools

744173

libcaca

http://people.debian.org/~wookey/bootstrap/patches/arm64/debian/libcaca_0.99.beta18-1-docdisable-arm64.patch

libffi

698344

3.0.11-2ubuntu1

libgc

732349

klibc

1081162

libbsd

1109050

libx11

1129389

qt4-x11

735488

mesa

732437

openssl

732348

1102107

xorg

731766

xutils-dev

734944

Packages that needed crossbuild fixes

package

Debian Bug

Ubuntu Bug

cups

734670

diffutils

1:3.2-3

1:3.2-1ubuntu1

tzdata

gdbm

604648

1.8.3-11ubuntu1

iptables

1.4.16

1081592

libsemanage

1103271

db

1105368

linux

1105251

sqlite3

packages that needed multiarch-related changes

package

Debian Bug

Ubuntu Bug

perl

633884

python-2.7

683755

chrpath

0.14

0.14

less

693318

1081611

check

693221

1101069

gettext

683751

ed

693824

dctrl-tools

693474

1129373

indent

693823

fdupes

693822

1117304

bc

681784

1098408

equivs

697820

1097993

linux-atm

1098417

tcl8.5

698674

1122120

autogen

1118246

netpbm-free

700007

2:10.0-15ubuntu2

libxcb

1129376

pkgconfig

726598

Packages that needed changes for eglibc 2.16

package

Debian Bug

Ubuntu Bug

diffutils

693346

1079770

tar

693352

1079750

gettext

693361

1079768

cpio

693440

1079748

coreutils

8.20

1081220

or eglibc 2.17

gpm

1.20.4-6ubuntu1

Packages that needed config.{guess,sub} updates

package

Debian Bug

Ubuntu Bug

acl

689610

Quantal

check

733038

chrpath

700117

0.14-1ubuntu1

cloog-ppl

coreutils

689611

Quantal

cpio

689612

Quantal

db

689613

Quantal,

dialog

689615

Quantal

diffstat

700118

1.55-3ubuntu1

diffutils

689617 1:3.2-7

1:3.2-7

dropbear

689618

expat

689619

Quantal

findutils

689620

Quantal

gawk

714795

gnupg

Quantal

json-c

1102043

kbd

700119

1.15.3-9ubuntu4

lcms2

717839

libasyncns

725778

libelf

693996 0.8.13-4~1

1082134 0.8.13-3ubuntu1

libgpg-error

689621

Quantal

libnfnetlink

693825

1081600

libnih

689622

Quantal

libsamplerate

734673

libsigc++

727299

libsndfile

732346

libx11

689623

Quantal

libxml2

689624

Quantal

libxslt

689625

Quantal

make-dfsg

689626

Quantal

mlocate

700094

0.25-0ubuntu2

module-init-tools

689627

Quantal

mpfr4

700065

3.1.0-5ubuntu1

nano

689628

Quantal

ncurses

Quantal

ocl-icd

732821

patch

Quantal

pam

pcre3

Quantal

policykit-1

734082

poppler

734014

popt

Quantal

sed

Quantal

sqlite

727510

zzlib

717837

x11-kbd-utils

717841

x11-server-utils

735489

Packages that just needed fixes

guile-1.8

711029

w3m

0.5.3-14

zsh

734765

Packages that need build profiles

The set of patches for this (in unstable/raring) is here: http://people.debian.org/~wookey/bootstrap/patches/

Packages waiting on build-deps

Perl

Perl is a bit of a special case. It needs both cross-build support and multiarchifying. Status is being tracked on Multiarch/Perl

Cross-build supprt currently involves creating config files on a host-arch machine, which currently means a model, or creating the files by inspection and comparison with others. A set of arm64 configs has been created from a model run and comparison with armhf and amd64 configs for Debian and arm64 config for openembedded. These are now checked-in to perl-cross-debian. And perl cross-builds successfully with these configs and that version of perl-cross-debian plus its corresponding perl patches.

Meanwhile There is a multiarch branch of perl here: 'ntyni/multiarch-5.14' branch of git://git.debian.org/perl/perl.git which is discussed in this thread: http://lists.debian.org/debian-perl/2012/09/msg00000.html

This builds and works OK, but does not currently cross-build. Merging these two pieces of work to get a cross-buildable, multiarched, arm64, perl is currently underway.

Packages that don't cross build

dialog

Wrong-arch strip run.

fuse

Conflicting 64-bit definitions: 1087757

icu

configure: error: Error! Cross compiling but no --with-cross-build option specified - please supply the path to an executable ICU's build root

libatomic-ops

Needs aarch64 assembly

Packages that are built for jessie

See the debian-ports status page: http://buildd.debian-ports.org/status/architecture.php?a=arm64&suite=sid

The initial bootstrap now has its own page, largely for historical interest: https://wiki.debian.org/Arm64Port/PackagesBuilt

Packages with issues

libcaca fails in

something wrong with pdflatex? Hacky workaround patch

transfig failed to build: make[3]: Entering directory `/home/wookey/debian/transfig-3.2.5.e/fig2dev' make[3]: Warning: File `/usr/include/X11/Xlib.h' has modification time 2.3e+08 s in the future rm -f fig2dev.o gcc -c DefaultGcc2AArch64Opt -I.. -Dlinux -Daarch64 -D_POSIX_C_SOURCE=199309L -D_POSIX_SOURCE -D_XOPEN_SOURCE -D_BSD_SOURCE -D_SVID_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DFUNCPROTO=15 -DNARROWPROTO -DNFSS -DUSE_INLINE -DI18N -DUSE_PNG -DUSE_XPM -I/usr/include/X11 -I/usr/include/X11 fig2dev.c gcc: error: DefaultGcc2AArch64Opt: No such file or directory This was same imake (xuitls-dev) issue that broke nas: 734944

w3m failed to build: first imlib2 bug 734425 has to be fixed But then the configury fails to add -lX11 to the link line so it fails to link. Both fixed in 0.5.3-14

fftw3 failed to build neon code (ICE) https://bugs.launchpad.net/linaro-aarch64/+bug/1267113 Simply not enabling neon for arm64 allows a build

guile-1.8 failed building tex tutorial. Upstream issue. Fixed in 711029

mksh segfaults in the tests (does not honour 'nocheck')

Packages that needed fixes

#util-linux apt-get install automake autopoint libtool # (29Mb of stuff) DEB_BUILD_OPTIONS="parallel=80" dpkg-buildpackage -sa > ../buildlog.util-linux-2.20.1 failedwith:

The fix is in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=689607

Having to add every new arch one by one to this package is silly. Listing the exceptions would be a lot more useful. Are there any?

#coreutils apt-get install dh-buildinfo groff gperf bison DEB_BUILD_OPTIONS="parallel=80" dpkg-buildpackage -sa fails to build:

see https://bugzilla.redhat.com/show_bug.cgi?id=917735 and http://gmplib.org:8000/gmp/rev/187b7b1646ee which says this is the same issue as fixed in gmp This is the bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=698330 Applying that fix (updated) works.

Using dose to see what is multiarch-buildable

Dose 3.1 can analyse build-dependencies for a given package, and understands about cross and multi-arch. It can tell you what is currently buildable given the current state of relevant source and package files. Use 3.1.2 for the --checkonly option and the --defaultedMAforeign option

Install it (it's in the bootstrap tools repo)

sudo apt-get install dose-builddebcheck

To check

dose-builddebcheck -f --checkonly <package> \
--defaultedMAforeign --deb-native-arch=amd64 --deb-foreign-archs=arm64 --deb-host-arch=arm64 \
/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_quantal_main_binary-amd64_Packages \
/var/lib/apt/lists/people.debian.org_%7ewookey_bootstrap_ubunturepo_dists_quantal-bootstrap_main_binary-amd64_Packages \
/var/lib/apt/lists/people.debian.org_%7ewookey_bootstrap_ubunturepo_dists_quantal-bootstrap_main_binary-arm64_Packages \
/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_quantal_universe_source_Sources | grep source