'arm64' is the Debian port name for the 64-bit ARMv8 architecture, referred to as 'aarch64' in upstream toolchains (GNU triplet aarch64-linux-gnu)

Hardware (that actually works) started to becoming available in October 2013, but access was restricted. Debian has very kindly been donated two 8-core machines, installed in March 2014 running two buildds in Debian-ports. One has been split with xen so we also have a porterbox. The port was started in 2010 (by ARM and Linaro working with the community in commendable fashion) long before hardware was available so that there would be something to run when it arrives.

Hardware, emulators and models

Arm64 hardware was available in the form of the iPhone 5 in 2013. Hardware useful for GNU/Linux started to become available from Q42013, with the first commercially available boards announced June 2013.

Arm64 qemu user-space emulation became initially available in October 2013. It was added to qemu upstream in v2.0, and is available in Debian 8 (Jessie) onwards. This is much faster than the model for building software, but can't be used for kernel-space things like bootloader/kernel work, or anything that uses multithreading. See Qemu Rootfs below for details.

Arm64 qemu system emulation was added to qemu upstream but is not yet available in Debian.

There is a (non-free, free beer) 'Foundation Model' simulator which can be used to run arm64 code, which is available here

Much of the work packagers need to worry about is updating autoconf, multiarch and cross-build fixes, with a few actual arm64 changes here and there, and doesn't actually need a model or emulator.

This is the first non-x86 self-bootstrapped Debian port: the first 150 packages cross-built using build-profiles to untangle cyclic build-dependencies.

Status

(last updated: July 2014)

Debian unstable is being bootstrapped in debian-ports: http://buildd.debian-ports.org/status/architecture.php?a=arm64&suite=sid

80% of Debian was built by mid-july: over 8500 source (arch-specific) packages.

About 630 packages have FTBFS. Please check the packages in the 'Build Attempted' section there to see what needs fixing. Many of them are trivial fixes. (See bug-tracking below for info on bug-filing and resources on fixing).

Whilst that 'build-attempted' list is over 499 packages it doesn't display the list, so these ordered lists of blockers (not yet built/not in debian proper) and outofdate packages are very useful to see what to work on (ordered by how many other packages they block the builds of):

We hope the port will be in a fit state to release as an official architecture in Jessie, but there is still a lot to do.

ARM has promised hardware for porterbox and DSA-run buildds

Status details and history

Binutils, kernel, gcc and glibc port patches were sent upstream over the summer of 2012, with enough stuff to build a cross-toolchain available by October 2012. The initial port (2012) was done entirely as a cross-build using Ubuntu packages, initially quantal, then raring (from Jan 2013) to take advantage of cross-build fixes, multiarch improvements, libc and arm64 updates going in there. This port was done in Ubuntu because multiarch and the cross-toolchains were more advanced there than in Debian at the time, and Debian was frozen for Wheezy.The resulting image and updates were then used to natively build most of Saucy once real hardware that worked was available (to canonical). Trusty has 'most' stuff built. The missing stuff is largely unported languages (haskell, mono, ruby, go, etc) and large unported packages like libreoffice and firefox.

The initial Debian bootstrap (from late 2013) (350 source, 2100 binaries) was built in a personal repo ). This bootstrap was done natively starting with an Ubuntu Saucy base to supply a base chroot and non-library build-deps, lsb+dpkg-vendor was set to 'Debian', each package was built, uploaded and added to a Debianise script which replaced all available packages in the Saucy clean tarball chroot until it contained no Ubuntu packages and was effectively Debian unstable. Then a new clean debian chroot was debootstrapped and packages built/rebuilt in there using build profiles to break cycles.

Once hardware was available this Debian bootstrap image/repo was used to set up a real buildd for debian-ports and keep the port uptodate against unstable. Build profiles were again used to cleanly rebootstrap.

There are crosstoolchains built for Raring/Saucy/Trusty. Debian cross-toolchains are being worked on.

Current cross-buildability (in Ubuntu) for arm64 is tracked on this status page. Native buildability is tracked at http://qa.ubuntuwire.org/ftbfs/arm64.html.

Pre-built Rootfses

There are a selection of rootfses here in the form of tarballs and disk images, with and without qemu-static installed.

tarball rootfs

Debian

unstable minimal debootstrap (64 MB)

unstable buildd debootstrap (117 MB) Includes build-essential (binutils, make, gcc), sudo, wget, nano editor, openssh-server so you can ssh in if networking is working.

Ubuntu

Ubuntu saucy multiarch tarball suitable for use as a chroot on arm64 hardware. Identical in content to the foundation model disk image below. Adding qemu-static would make it useful under qemu. Configured for armhf/arm64 multiarch use.

qemu

To use arm64 qemu, just install it from unstable/testing:

apt-get install qemu-user-static

This package installs fine on stable as well, and is available in backports.

(Note: Older qemu released referenced on this page used a different name for the static binary and binfmt-misc config (now qemu-aarch64-static, was qemu-arm64) and thus new config/old images (or vice versa) will not interoperate).

Using The Foundation model

See https://wiki.debian.org/Arm64Port/raringbootstrap#Raring_rootfs_for_Foundation_model

Here is a Ubuntu saucy disk image (92MB compressed, 2G uncompressed). Suitable for use with the foundation model or real hardware. This image is configured for armhf/arm64 multiarch usage, but it's a one-liner to turn that off.

And here's a suitable kernel (3.13, linaro foundation release 2014-01), modified for use with Linux instead of OE.

Debootstrap arm64

Plain debootstrap will now produce an arm64 rootfs from the Debian Ports archive. Use --variant=buildd for a trouble-free rootfs creation.

On an x86 machine, install qemu-user-static 2.0 or later (the package from unstable works fine on stable/testing), and:

sudo apt-get install debian-ports-archive-keyring
sudo apt-key add /usr/share/keyrings/debian-ports-archive-keyring.gpg
sudo qemu-debootstrap --arch=arm64 --keyring /usr/share/keyrings/debian-ports-archive-keyring.gpg --variant=buildd unstable debian-arm64 http://ftp.debian-ports.org/debian

Will do the whole process (creating an unstable chroot in the 'debian-arm64' dir).

If you have hardware or want to do it on qemu or a model), use these commands for a native debootstap:

sudo apt-get install debian-ports-archive-keyring
sudo apt-key add /usr/share/keyrings/debian-ports-archive-keyring.gpg
sudo debootstrap  --keyring /usr/share/keyrings/debian-ports-archive-keyring.gpg unstable debian-arm64 http://ftp.debian-ports.org/debian

Inside the chroot change your sources.list to http://ftp.debian-ports.org unstable _and_ unreleased.

deb [arch=arm64] http://ftp.debian-ports.org/debian/ unstable main
deb-src http://ftp.debian-ports.org/debian/ unstable main
deb [arch=arm64] http://ftp.debian-ports.org/debian/ unreleased main

to have the latest packages available. (packages locally packaged for arm64 go into 'unreleased' until the fixed version is uploaded to debian-proper.

The old bootstrap repo at http://people.debian.org/~wookey/bootstrap/debianrepo2 is now obsolete and should not be used.

Porting packages for arm64 - Maintainer info

If your package does not build for arm64 it is usualy trivial to fix, but sometimes a bit harder and sometimes a big deal. Here is info on what to change and where to go for help, updates and info.

autoconf updating

Most packages just need updated autoconf config info which includes the new arch. It is now best practice for (autoconf-using) Debian packages to re-autoconf on every build in order to pick up this info automaticaly, or at least use the autotools-dev package to ensure latest config.sub and config.guess files. Good info on making these updates is on https://wiki.debian.org/qa.debian.org/FTBFS#A2014-01-21_using_dh-autoreconf_during_the_build

Packages that use CDBS (and autoconf) will automatically get updated config.sub and config.guess if autotools-dev is installed (it will not be installed automatically on buildds unles your package build-depends it (see Debianbug:744915) so until a better solution is found you'll need to add it to your build-deps).

Nomenclature and defines

If your package does architecture-specific things explicitly then you will need to understand what names to use in tests.

The gnu name for the architecture (as given to configure) is aarch64-linux-gnu.

The debian name for the architecture is arm64

GCC defines __aarch64__ for the architecture.

Be careful of things which check for arm* in debian architecture tests, as it is usually wrong to do the same thing for arm64 as for arm. In general, if you are not sure you should do the same thing as amd64 as that matches quite closely (64 bit, little endian, 32-bit ints and floats, 64-bit pointers, longs and doubles).

Check the link below for 'upstream package porting' to see if your package has had porting attention from Linaro.

There is also a big-endian version of the architecture/ABI: aarch64_be-linux-gnu but we're not supporting that in Debian (so there is no corresponding Debian architecture name) and hopefully will never have to. Nevertheless you might want to check for this by way of completeness in upstream code.

Useful porting docs

Bug tracking

Please tag all arm64/aarch64-specific bugs 'arm64' (user=debian-arm@lists.debian.org, tag=arm64 - ) in the BTS (see bugs.debian.org/usertags for instructions), or in launchpad.

Here is an example (assuming you have a patch file, and a body template file)

reportbug $package  -V $version -A $patchfile --src --subject "Add arm64 support" --tag patch --pseudo-header 'User: debian-arm@lists.debian.org' --pseudo-header 'Usertag: arm64' --no-tags-menu --severity normal --body-file $template

Here is the current list of arm64 Debian bugs

Other distros often have useful bug-fixes already:

Upstream-relevant bugs should be filed in the linaro-aarch64 bug tracker to avoid distro duplication of effort. In this case, file bugs in the Debian BTS as well, then link to them in the Linaro tracker.

Repository

Packages are all in debian-ports.

The initial debian (using ubuntu arm64 packages) and ubuntu bootstraps were stored here but are now obsolete. A quantal chroot was used for initial work but is now (Dec 2012) frozen/abandonned.

You should just be able to add that repo and apt-get install crossbuild-essential-arm64 to get a working cross environment. If build and host arch packages have got out of sync this won't work, which happens often in raring and unstable.

Cross Toolchain

Before anything useful can happen dpkg needs to have arm64 support. This was done in v1.16.4.

binutils-aarch64-linux-gnu is now (July 2014) in debian unstable (from the cross-binutils package)

Toolchain Bootstrap

First job is to bootstrap a cross-toolchain so other stuff can be built.

This is relatively easy to do from upstream sources either directly or using a framework like ?OpenEmbedded, but to get a Debian-packaged toolchain easily usable in chroots, sbuild etc requires merging the new port with the Debian packaging, which is, frankly, a PITA.

When starting from scratch multiarch doesn't help you because you need eglibc:arm64 to make aarch64-linux-gnu-gcc and you need aarch64-linux-gnu-gcc to make eglibc:arm64. So you have to do a 3-stage bootstrap.

Ubuntu has (thanks to Linaro) a package to automate the building of fully-bootstrapped compilers for armhf/el so that was munged to make arm64-cross-toolchain-base, and the delighful task of getting everything working was started.

The process is:

Easy peasy.

This work was initially done in Ubuntu due to better multiarch support, availability of *-cross-toolchain-base and newer eglibc.

Toolchain Bootstrap Issues

Quantal bootstrap

The initial arm64 packaged toolchain was done in Ubuntu Quantal, as the first release where multiarch crossbuilding basically worked. The details of this are documented on toolchain/BootstrapIssues#Quantal.

Raring bootstrap

Easier than quantal as all the base porting has already been done. However not quite as easy as one might like. arm64-cross-toolchain-base was updated with some fixes/updates from the armhf/armel-c-t-b packages. Binutils and linux-libc-dev headers were straightforward.

But gcc stage1 build fell over first with bugs in gcc-4.7.2-12ubuntu2 (patches that didn't apply, missing arm64 symbols-files). Then with missing bits/predefs.h header when building libgcc. This turned out to be 1091823.

Then the build completed but there was a libgconv.a packaging error for some reason. Updating to gcc-4.7-2-14ubuntu2 seemed to fix that.

Debian bootstrap

The Debian toolchain bootstrap is being attempted fully multiarch, to avoid the above issues with needing a separate libc:arm64 and equivs packages to make everything work. It's also an opportunity to integrate DEB_BUILD_PROFILE boostrap features to make it all nicely automatable, without toolchains being a special case in the archive. This was initially (December 2012) too troublesome, mostly due to new versions of GCC coming out every few days with this stuff being changed. As of gcc-4.7.2-18 onwards ThibG's patches for the toolchain using the multiarch system libraries are integrated so I had another go.

This process is documented on MultiarchCrossToolchains.

Current status is that experiemntal is being used as the stuff in unstable is too old.

binutils and gcc stage1 are built.

The kernel header patches are being updated for 3.8

Building packages

Given a cross-build chroot, in general you should be able to do

apt-get build-dep -aarm64 <package>
apt-get source <package>
cd <package>-<ver>
CONFIG_SITE=/etc/dpkg-cross/cross-config.arm64  DEB_BUILD_OPTIONS=nocheck dpkg-buildpackage --preserve-env -aarm64

But there are of course some caveats.

Details on setting up an arm64 cross-chroot are given here: https://wiki.linaro.org/Platform/DevPlatform/CrossCompile/arm64bootstrap

There is no libssp (stack protection) support in aarch64 toolchain yet (Nov 2013, https://cards.linaro.org/browse/TCWG-23) so the -fstack-protector will cause an error:

/usr/lib/gcc/aarch64-linux-gnu/4.7/../../../../aarch64-linux-gnu/bin/ld: cannot find -lssp_nonshared
/usr/lib/gcc/aarch64-linux-gnu/4.7/../../../../aarch64-linux-gnu/bin/ld: cannot find -lssp

For any package that uses dpkg-buildflags this is easy to fix by setting DEB_BUILD_MAINT_OPTIONS=hardening=-stackprotector You can set this for the whole build chroot if using one (probably best for now), or for a user, or for a build by doing:

DEB_BUILD_MAINT_OPTIONS=hardening=-stackprotector dpkg-buildpackage -aarm64

Better to fix this permanently in /etc/dpkg/buildflags:

STRIP CFLAGS -fstack-protector

Package porting

Many packages need no more than config.guess and config.sub updating to build for arm64. autotools got aarch64 support upstream in June 2012. So autotools-dev 20120608.1 has the files needed. However many (most) packages don't autoreconfig on build, or otherwise update those files so they have to be patched. A lot of boring patches were filed with the tag arm64, such as 689613

In many cases dh_autoreconf can be used to fix this without making horrible, unreadable, autofoo patches. Sometimes it's better to use autotools-dev which only updates config.{sub,guess} without re-generating all the configury because this is less invasive, or the package simple doesn't autoreconf properly. dh-autoreconf is preferred as it covers all cases (for example the ppc64el port needs re-libtooling, so dh-autoreconf works, but the autotools-dev dh_updateconfig is not sufficient. Some packages won't build a second time after dh_autoreconf has poked them around. This QA page goes into more details about updating your packages

For the initial botstrap everything had to be cross-built as there was no hardware (and no OS) so cross-building patches were also needed in many packages, and multiarch patches in many build-deps. Due to the Wheezy freeze and thus the more progressed state of multiarchification and more cross-build fixes in Ubuntu, this work was initialy done in a private Quantal-based repo, and later in a raring-tracking repo, but all bugs are either directly filed, or also pushed upstream to Debian. There are still (April 2014) a lot of cross-build fixes that are in Ubuntu but not yet uploaded to Debian.

Quite a few packages need actual changes. Here is a list for the early stages (see the BTS for complete info):

Packages that needed arm64-related changes

package

Debian Bug/Fixed version

Ubuntu Bug/Fixed version

dpkg

672408

autotools

20120210.1

20120210.1

binutils

binutils-2.22.90.20120924

coreutils

698330

gcc

gcc-4.8

gcc-4.7-4.7.2-3ubuntu1arm64 1133104

eglibc

690873 2.16-0arm64.1 2.17

2.16-0ubuntu3 2.17-0ubuntu4 1120810

fftw3

734675

1267107

linux

695241

1063895

util-linux

689607

perl

https://github.com/codehelp/perl-cross-debian/tree/master/aarch64-linux-gnu

dpkg-cross

693730 2.6.8

2.6.7arm64

(cross)-build-essential

693220

11.6ubuntu4

gmp

693467

1079831

kde-pkg-tools

744173

libcaca

http://people.debian.org/~wookey/bootstrap/patches/arm64/debian/libcaca_0.99.beta18-1-docdisable-arm64.patch

libffi

698344

3.0.11-2ubuntu1

libgc

732349

klibc

1081162

libbsd

1109050

libx11

1129389

qt4-x11

735488

mesa

732437

openssl

732348

1102107

tbb

748508

xorg

731766

xutils-dev

734944 748010

Packages that needed crossbuild fixes

package

Debian Bug

Ubuntu Bug

cups

734670

diffutils

1:3.2-3

1:3.2-1ubuntu1

tzdata

gdbm

604648

1.8.3-11ubuntu1

iptables

1.4.16

1081592

libsemanage

1103271

db

1105368

linux

1105251

sqlite3

packages that needed multiarch-related changes

package

Debian Bug

Ubuntu Bug

perl

633884

python-2.7

683755

chrpath

0.14

0.14

less

693318

1081611

check

693221

1101069

gettext

683751

ed

693824

dctrl-tools

693474

1129373

indent

693823

fdupes

693822

1117304

bc

681784

1098408

equivs

697820

1097993

linux-atm

1098417

tcl8.5

698674

1122120

autogen

1118246

netpbm-free

700007

2:10.0-15ubuntu2

libxcb

1129376

pkgconfig

726598

Packages that needed changes for eglibc 2.16

package

Debian Bug

Ubuntu Bug

diffutils

693346

1079770

tar

693352

1079750

gettext

693361

1079768

cpio

693440

1079748

coreutils

8.20

1081220

or eglibc 2.17

gpm

1.20.4-6ubuntu1

Packages that needed config.{guess,sub} updates

package

Debian Bug

Ubuntu Bug

acl

689610

Quantal

check

733038

chrpath

700117

0.14-1ubuntu1

cloog-ppl

coreutils

689611

Quantal

corosync

743243

cpio

689612

Quantal

db

689613

Quantal,

dialog

689615

Quantal

diffstat

700118

1.55-3ubuntu1

diffutils

689617 1:3.2-7

1:3.2-7

dropbear

689618

expat

689619

Quantal

findutils

689620

Quantal

gawk

714795

gnupg

Quantal

graphviz

749061

json-c

1102043

kbd

700119

1.15.3-9ubuntu4

lcms

543464

lcms2

717839

libasyncns

725778

libelf

693996 0.8.13-4~1

1082134 0.8.13-3ubuntu1

libgpg-error

689621

Quantal

libnfnetlink

693825

1081600

libnih

689622

Quantal

libsamplerate

734673

libsigc++

727299

libsndfile

732346

libx11

689623

Quantal

libxml2

689624

Quantal

libxslt

689625

Quantal

make-dfsg

689626

Quantal

mlocate

700094

0.25-0ubuntu2

module-init-tools

689627

Quantal

mpfr4

700065

3.1.0-5ubuntu1

nano

689628

Quantal

ncurses

Quantal

ocl-icd

732821

patch

Quantal

pam

pcre3

Quantal

policykit-1

734082

poppler

734014

popt

Quantal

sed

Quantal

sqlite

727510

zzlib

717837

x11-kbd-utils

717841

x11-server-utils

735489

Packages that just needed fixes

guile-1.8

711029

w3m

0.5.3-14

zsh

734765

Packages that need build profiles

The set of patches for this (in unstable/raring) is here: http://people.debian.org/~wookey/bootstrap/patches/

Packages waiting on build-deps

Nothing of significance now (July 2014)

Perl

Perl is a bit of a special case. It needs both cross-build support and multiarchifying. Status is being tracked on Multiarch/Perl

Cross-build supprt currently involves creating config files on a host-arch machine, which currently means a model, or creating the files by inspection and comparison with others. A set of arm64 configs has been created from a model run and comparison with armhf and amd64 configs for Debian and arm64 config for openembedded. These are now checked-in to perl-cross-debian. And perl cross-builds successfully with these configs and that version of perl-cross-debian plus its corresponding perl patches.

Meanwhile There is a multiarch branch of perl here: 'ntyni/multiarch-5.14' branch of git://git.debian.org/perl/perl.git which is discussed in this thread: http://lists.debian.org/debian-perl/2012/09/msg00000.html

This builds and works OK, but does not currently cross-build. Merging these two pieces of work to get a cross-buildable, multiarched, arm64, perl is currently underway.

Packages that don't cross build

dialog

Wrong-arch strip run.

fuse

Conflicting 64-bit definitions: 1087757

icu

configure: error: Error! Cross compiling but no --with-cross-build option specified - please supply the path to an executable ICU's build root

libatomic-ops

Needs aarch64 assembly

Packages that are built for jessie

See the debian-ports status page: http://buildd.debian-ports.org/status/architecture.php?a=arm64&suite=sid

The initial bootstrap now has its own page, largely for historical interest: https://wiki.debian.org/Arm64Port/PackagesBuilt

Packages with issues

fftw3 failed to build neon code (ICE) https://bugs.launchpad.net/linaro-aarch64/+bug/1267113 Simply not enabling neon for arm64 allows a build

emacs23 FTBFS (hangs compiling lisp libraries): #752031

qt4-x11 only builds with -fpermissive, which is not acceptable to maintainers: #735488

xutils-dev only builds correctly with cpp-4.7 which doesn't exist for arm64: #748010

postgres9.3 fails in the documentation build - apprently indicating a problem in the arm64 ghostscript package.

mksh segfaults in the tests (does not honour 'nocheck')

flex fais to buld docs due to pdftex error. Something really wrong?:

ading

/usr/bin/texi2dvi: pdfetex exited with bad status, quitting. Makefile:434: recipe for target 'flex.pdf' failed

Packages that needed fixes

#util-linux apt-get install automake autopoint libtool # (29Mb of stuff) DEB_BUILD_OPTIONS="parallel=80" dpkg-buildpackage -sa > ../buildlog.util-linux-2.20.1 failedwith:

The fix is in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=689607

Having to add every new arch one by one to this package is silly. Listing the exceptions would be a lot more useful. Are there any?

#coreutils apt-get install dh-buildinfo groff gperf bison DEB_BUILD_OPTIONS="parallel=80" dpkg-buildpackage -sa fails to build:

see https://bugzilla.redhat.com/show_bug.cgi?id=917735 and http://gmplib.org:8000/gmp/rev/187b7b1646ee which says this is the same issue as fixed in gmp This is the bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=698330 Applying that fix (updated) works.

Using dose to see what is multiarch-buildable

Dose 3.1 can analyse build-dependencies for a given package, and understands about cross and multi-arch. It can tell you what is currently buildable given the current state of relevant source and package files. Use 3.1.2 for the --checkonly option and the --defaultedMAforeign option

Install it (it's in the bootstrap tools repo)

sudo apt-get install dose-builddebcheck

To check

dose-builddebcheck -f --checkonly <package> \
--defaultedMAforeign --deb-native-arch=amd64 --deb-foreign-archs=arm64 --deb-host-arch=arm64 \
/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_quantal_main_binary-amd64_Packages \
/var/lib/apt/lists/people.debian.org_%7ewookey_bootstrap_ubunturepo_dists_quantal-bootstrap_main_binary-amd64_Packages \
/var/lib/apt/lists/people.debian.org_%7ewookey_bootstrap_ubunturepo_dists_quantal-bootstrap_main_binary-arm64_Packages \
/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_quantal_universe_source_Sources | grep source