Differences between revisions 39 and 40
Revision 39 as of 2011-10-12 15:19:30
Size: 8250
Editor: ?BernhardKaindl
Revision 40 as of 2012-11-10 08:52:29
Size: 8331
Editor: OsamuAoki
Comment: Add references pointing to Russ Allbery's site on this issue
Deletions are marked like this. Additions are marked like this.
Line 121: Line 121:
 * [[http://www.eyrie.org/~eagle/notes/rpath.html|Shared Library Search Paths]]

RPATH issue


There is quite a lot of information on the RPATH issue in Debian, however there is no document which consistently explains what the problem actually is, why the use of RPATH is discouraged in Debian and what are the preferred ways to correct upstream builds that use this feature. This page was created by JurijSmakov with the intention of collecting all the relevant information on the subject.

The Problem

Dynamic linking

The job of the dynamic linker and loader (ld.so) is to resolve the executable's dependencies on shared libraries and to load the required ones at run-time. It relies on the executable's NEEDED headers, to find the shared libraries with a matching SONAME (which includes the library name and API version).

The dynamic linker will look for a matching library in the following locations, in this order, which can be changed (see the footnotes below):

  1. the DT_RPATH dynamic section attribute of the library causing the lookup
  2. the DT_RPATH dynamic section attribute of the executable
  3. the LD_LIBRARY_PATH environment variable, unless the executable is setuid/setgid.
  4. the DT_RUNPATH dynamic section attribute of the executable
  5. /etc/ld.so.cache

  6. base library directories (/lib and /usr/lib)

Footnotes (the small print...):

  • Regarding steps 1 and 2: The DT_RPATH attribute is ignored if the DT_RUNPATH attribute is found. Then, LD_LIBRARY_PATH is searched first!
  • Regarding step 3: LD_LIBRARY_PATH can be overridden by calling the dynamic linker with the option --library-path (e.g. /lib/ld-linux.so.2 --library-path $HOME/mylibs myprogram

  • Regarding steps 5 and 6: If the executable was linked with -z nodeflib linker option, /lib and /usr/lib are skipped at step 5 and 6.
  • Regarding all steps: If the dynamic linker is called using --inhibit-rpath LIST, the objects in LIST are ignored.

  • Before library searching takes place, the libraries in LD_PRELOAD are loaded

Why RPATH is an issue

A problem arises when binary A defines a NEEDED dependency on libraries B.so.1 and C.so.2, while library B.so.1 depends on library C.so.1. This means parts of the code will use one version of a library and other parts another. The many versions of a library scenario is needed to deal with gradual migrations, but maintainers shuffle libraries around when dealing with such a situation and packages with RPATH could end up finding the wrong version of a dependent library, one with incompatible dependencies.

Since RPATH is set at build-time, it can only be overridden with a rebuild or by setting the LD_LIBRARY_PATH variable and this turns packages using it into management problems:

  • LD_LIBRARY_PATH can't be used because it has its own problems: it'll be inherited by all processes generated by the parent and is therefore also discouraged for distribution-wide use for its possible side-effects.

  • Needing a rebuild means that all dependencies of the affected library would need to be rebuilt during any move, which is an unacceptable scenario for a distribution.

This situation would be better dealt with by the dynamic linker, since maintainers would have a central place to inform all dependent packages where to find libraries during transitional periods.

libtool's role

Historically, GNU libtool made extensive use of the RPATH feature, including the hardcoded library path information by default into all executables and shared libraries, which meant that this issue used to appear on many libtool based packages whose libtool used RPATH as default and didn't specifically implement workarounds for it.

Since libtool 1.5.2 (released 2004-01-25), on Linux libtool no longer sets RPATH for any directories in the dynamic linker search path, so this should no longer be an issue unless upstream used a really old version of libtool when creating their distribution tarball.

Debian's Stance

While there's no policy dictating the accepted use of RPATH, it's been a general consensus that RPATH use is discouraged, given the interactions between the above reasons and Debian's way of dealing with libraries and package dependencies.

Currently, the only generally accepted use of this feature in Debian is to add non-standard library path (like /usr/lib/<package>) to libraries that are only intended to be used by the executables or other libraries within the same source package.

Possible solutions

Upstream changes

Ideally, upstream should implement a build-system that supports turning RPATH usage off during configuration.

If the build-system is based on pre-1.5.2 libtool, suggest upstream upgrade to a newer version.

Patching the build-system

Alternatively, the build-system can be patched to avoid RPATH at build-time.

Following example (taken from configure.ac in wmaker) illustrates it:


 # by Marcelo Magallon <mmagallo@efis.ucr.ac.cr>
 # Turn around -rpath problem with libtool 1.0c
 # This define should be improbable enough to not conflict with anything
 case ${host} in
     AC_MSG_RESULT([Fixing libtool for -rpath problems.])
     sed < libtool > libtool-2 \
     's/^hardcode_libdir_flag_spec.*$'/'hardcode_libdir_flag_spec=" -D__LIBTOOL_IS_A_FOOL__ "/'
     mv libtool-2 libtool
     chmod 755 libtool

Another possibility (similar by the spirit) is presented by Matthew Garrett:

 AC_MSG_RESULT(patching libtool to fix HIDEOUS BREAKAGE)
 test -f libtool.old |||| (mv libtool libtool.old && cp libtool.old libtool)
 sed -e s/^hardcode_direct.*$/hardcode_direct=yes/g libtool || \
 sed -e s/^hardcode_minus_L.*$/hardcode_minus_L=yes/g || \
 sed -e s/^hardcode_shlibpath_var.*$/hardcode_shlibpath_var=no/g   >libtool.new
 mv libtool.new libtool

Or more generally, it's also possible to patch the configure generated libtool script, the autoconf generated configure script or the raw acloca.m4 script via debian/rules with:

 sed -i -r 's/(hardcode_into_libs)=.*$/\1=no/' $(CURDIR)/<file to patch>

NOTE: if by need or choice the aclocal.m4, configure.in or configure.ac files were patched, it's necessary to re-run autoconf in order to generate an up-to-date configure script.

Using chrpath

The package chrpath can be used during debian/rules' install target to strip the RPATH header from the generated binary.

rpath issue on amd64

Fedora (and possibly some other distros) patched their libtool 1.5 packages so that on 64 bit Linux platforms it calculates the list of directories ld.so searches as /lib64 and /usr/lib64 plus anything in ld.so.conf (rather than /lib and /usr/lib plus anything in ld.so.conf which is what vanilla upstream libtool does).

Debian use /lib and /usr/lib on all architectures, so if upstream built their tarballs on a Fedora 10 or earlier box, then an rpath would end up set in Debian packages for 64 bit architectures. The libtool packages in Fedora 11 and later (which are of libtool 2.2.6) have a revised patch which adds /lib64 and /usr/lib64 while keeping /lib and /usr/lib, so this problem is now resolved unless upstream uses a no longer supported version of Fedora (or hasn't made a release in a while).