based on Matt Taggart's earlier write-up at http://lackof.org/taggart/hacking/multiarch/

Introduction

The Debian Multiarch proposal represents a radical rethinking of the filesystem heirarchy with respect to library and header paths. As such, it is a very disruptive change; software has long assumed that on Unix systems the library and include paths are sister directories at the same level of the heirarchy, and multiarch violates this assumption thoroughly. Even if all the various upstream build systems avoided hard-coding this assumption and were perfectly content to trust the system path (which today they are not), there would still be the matter of getting these directories on the system path to begin with - which means patching (or wrapping) all the various compilers in use.

If we are going to ask compiler upstreams to accomodate this seemingly gratuitous difference, and ask other distributions to embrace multiarch as a new standard, it behooves us to make a case for why such a change is needed at all.

Problem

Where supporting libraries of different ABIs on a single filesystem is concerned, there are several related problems to consider:

32/64 Architectures

Architectures with support for 32bit and 64bit versions of the same instruction set.

Examples

Current practices

The FHS and LSB have standardized the x86_64 architecture to use /lib64 as the path for 64-bit x86 libraries, with /lib reserved for 32-bit x86 libraries on such systems. This is in spite of the fact that for performance reasons, x86_64 is the preferred ABI to use on hardware that supports it.

Red Hat and SuSE have adopted this standard. Debian and Ubuntu have declined to adopt this provision of the FHS, because the inconsistency introduced by special-casing of x86_64 would require deep changes to the packaging tools for incremental benefit.

Comments

We have both the case of a mostly 32bit system with some 64bit libs/binaries and a mostly 64 bit system with some 32bit libs/binaries. Ideally they would be treated the same. So what goes in /lib? It may need to exist for legacy reasons, should it be a symlink to /lib32 or /lib64?

Mixed instruction sets

Architectures with support for running other instruction sets (either via hardware or software emulation)

Examples

Current practices

Comments

Since this could potentially be any target combination, it's wasteful to reinvent a new schema for each emulator or target. These emulation environments also have all the same problems that the introduction of /lib64 for x86_64 was intended to address, getting none of the benefits of the special-case "biarch" solution for 32/64bit environments since the pairs aren't the special-cased ones.

Mixed OS environment

Running binaries from one OS on another via a compatibility layer

Examples

Current practices

Comments

As most proprietary Unices have long since been overtaken by Linux in the market, demand for this capability has probably already peaked. Nevertheless, there will be uses for OS compatibility layers for some time to come.

Cross-compilation environments

Building binaries for an architecture other than the current one.

Current practices

Cross-build environments commonly mirror the FHS structure (bin, include, lib) in a parallel, architecture-specific tree (e.g., /usr/arm-linux-gnueabi).

Comments

Of all the use cases that multiarch would try to address, cross-building is the one where existing practice comes closest to providing a complete solution: it approximates the FHS, and it addresses all architectures. Even so, it's not symmetrical; binaries built natively look for their auxiliary files in /usr/lib, binaries targeting a cross-build environment would look in /usr/arch/lib, still making it impossible to reuse a single binary in both native and non-native contexts. Adding top-level arch/lib directories to replace /lib for system libraries would almost certainly also be unwelcome.

Mixed endian

Architectures that support running mixed endian binaries.

Examples

Comments

Not currently a large problem but documented here for completeness.

Mixed ABIs and instruction set extensions

Architectures with more than one ABI.

Examples

Comments

It is dubious that we should try to address ABI-compatible, optimized subarchs with the same solution as the other problems listed here. GNU libc's hwcaps implementation already does an adequate job of autodetecting appropriate optimized libraries for use; it is probably better to combine a hwcaps-style implementation for ABI-compatible subarchs with a multiarch solution for incompatible ABIs. See Multiarch/Tuples for an exploration of how compatible subarchs should be defined.

Proposed Solution

Requirements

In order to support binaries for multiple targets on the same system, we need to be able to:

Proposed General solution

Key: The term prefix is intended to be replaced with the FHS compliant install prefixes such as /, /usr, /usr/local, /opt/foo, etc. The term target is meant to represent the canonical GNU tuple (triplet) for the architecture / os combination.

Implementation Options

Runtimes implementing this proposal can put the libraries/headers/program interpreters wherever they like as long as they can be accessed as described in the proposal. This allows for a range of possible implementations and the ability for a gradual transition. Here are some potential options.

Benefits

FAQ