Differences between revisions 8 and 9
Revision 8 as of 2004-10-17 13:39:38
Size: 8772
Editor: anonymous
Comment:
Revision 9 as of 2004-10-17 13:40:07
Size: 8770
Editor: anonymous
Comment:
Deletions are marked like this. Additions are marked like this.
Line 6: Line 6:
consistently explains what the problem actually is, and why the use of RPATH is discouraged in Debian. Neither have I heard of a document, which would list preferred (or even acceptable) ways to correct upstream builds, which use this feature. My intention is to collect all the relevant information here, and I would really appreciate if you would contribute to it and, especially, correct it (just hit the Edit button below). consistently explains what the problem actually is, and why the use of RPATH is discouraged in Debian. Neither have I heard of a document which would list preferred (or even acceptable) ways to correct upstream builds which use this feature. My intention is to collect all the relevant information here, and I would really appreciate if you would contribute to it and, especially, correct it (just hit the Edit button below).

RPATH issue


Background

There is quite a lot of information on RPATH issue in Debian. However, to the best of my knowledge, there is no document which consistently explains what the problem actually is, and why the use of RPATH is discouraged in Debian. Neither have I heard of a document which would list preferred (or even acceptable) ways to correct upstream builds which use this feature. My intention is to collect all the relevant information here, and I would really appreciate if you would contribute to it and, especially, correct it (just hit the Edit button below).

SONAME

To identify the version of the shared library, needed by the executable, and allow the concurrent presence of a few binary-incompatible versions of the same library (for backward compatibility, for example) each of them has an encoded SONAME. The SONAME of the library may be found by using a command

[wiki:?SONAME =objdump -p <library_name> || grep]

When the executables and shared libraries are built, the list of !["SONAMEs"] of the shared libraries they depend on is encoded into them. This list may be viewed by using a command

[wiki:?NEEDED =objdump -p <executable_or_library_name> || grep]

More information on how it works in Debian may be found in the [http://www.netfort.gr.jp/~dancer/column/libpkg-guide/libpkg-guide.html Debian Library Packaging Guide].

Dynamical linking

The job of the dynamic linker and loader (ld.so) is to resolve the executable's dependencies on shared libraries and to load the required ones. It relies on SONAME information, encoded in the executable's NEEDED fields, to find the shared libraries with a matching SONAME and load them. According to ld.so manual page, it will look for a matching library using three different sources: the LD_LIBRARY_PATH environment variable, /etc/ld.so.cache, and trusted library directories /usr/lib and /lib.


Problem

There exists yet another mechanism to specify the location of the NEEDED libraries. While building the executables and shared libraries the linker may be given -rpath or -R switch, followed by the path to the NEEDED libraries. This information is then hardcoded into the binary and is examined by ld.so in the beginning of the execution. Currently, the only valid use of this feature in Debian is to add non-standard library path (like /usr/lib/<package>) to libraries, which are only intended to be used by the executables (or other libraries) within the package.

Historically, GNU libtool makes extensive use of RPATH feature, including the hardcoded library path information by default into all executables and shared libraries. This created problems during the libc5 to libc6 transition. In the January 1999 debian-devel thread, discussing the problem, J. H. M. Dassen [http://lists.debian.org/debian-devel/1999/debian-devel-199901/msg02254.html wrote]:

 I've seen one too many instances of "<foo> crashes on Debian" that turned
 out to be "<foo> is a libc5 binary with an RPATH of /usr/X11R6/lib" which on
 any reasonably up to date Debian system contains libc6 X libraries.
 The preferred way for libtool to make binaries, which know where the shared
 libraries they depend on are installed, is through the use of rpath. Basically,
 by default, libtool produces binaries, which contain hardcoded path to all
 the needed shared libraries. This caused extensive problems in Debian during
 the libc5 to libc6 transition.

What happened (as I understand it) is that the executable may have depended, for example, on the X libraries, linked against libc5. As the transition to libc6 was accomplished, the libc5 X libraries were replaced by those, linked against libc6. The former ones were moved to some other directory, but since it was essentially the same library (only linked against a different version of libc), the SONAME of the library remained the same (and the Debian could not just bump the SONAME on its own, since that would cause a discrepancy with upstream version numbering). The task of the dynamic linker in this situation is somewhat more complicated than usual: it has to choose the correct version of the library, relying on the information other than SONAME. Jason Gunthorpe [http://lists.debian.org/debian-devel/1999/debian-devel-199901/msg02276.html wrote]:

 Not exactly, the x libraries are fully compatible and were built with the
 same source - the trouble is that one is linked with libc5 and the other
 with libc6. In normal cases the dymanic linker would figure this out one
 way or another with rpath this functionality is disabled as it overrides
 the library versioning scheme.

 The linux dynamic linker will resolve things in some magical way based on
 the library dependencies and the program dependencies to locate the proper
 library in many cases - rpath does cripples, not enhances, this function.

The problem is [http://lists.debian.org/debian-devel/1999/debian-devel-199901/msg02285.html summarized] nicely by the libtool developer Alexander Oliva:

 It seems to me that the main problem has to do with library
 versioning.  Even though the existing library versioning mechanisms
 are great to describe the evolution of the API offerred by a library,
 they by no means describe the dependencies of a library, so we end up
 with libraries that have the same version numbers but that are not
 compatible at all.

 What we all are desperately looking for is a mechanism to allow the
 dynamic loader to pick the right version of a shared library out of a
 set of multiple builds of the same version of a library, the main
 difference between them being their dependencies.

He, however, did not think that the proper solution to the problem would be modification of default libtool behaviour (which is to always include -rpath while linking). This point of view contradicted the general consensus on debian-devel and the requirement to not encode the RPATH into binaries and libraries (except for some special cases mentioned above) has been accepted. Therefore, workarounds are required while building the Debian packages, if upstream is using the standard libtool machinery.


Possible solutions

I have examined a number of packages, which have to deal with RPATH issue. Note that the discussion is limited to the case when libtool is used, since otherwise disabling RPATH is more or less trivial. Most popular way seems to be to just edit libtool after it is generated. Following example (taken from configure.ac in wmaker) illustrates it:

 AC_PROG_LIBTOOL

 # by Marcelo Magallon <mmagallo@efis.ucr.ac.cr>
 # Turn around -rpath problem with libtool 1.0c
 # This define should be improbable enough to not conflict with anything
 case ${host} in
   *-pc-linux-gnu)
     AC_MSG_RESULT([Fixing libtool for -rpath problems.])
     sed < libtool > libtool-2 \
     's/^hardcode_libdir_flag_spec.*$''hardcode_libdir_flag_spec=" -D__LIBTOOL_IS_A_FOOL__ "'''
     mv libtool-2 libtool
     chmod 755 libtool
   ;;
 esac

Another possibility (similar by the spirit) is [http://lists.debian.org/debian-devel/2003/debian-devel-200306/msg00569.html presented] by Matthew Garrett:

 AC_MSG_RESULT(patching libtool to fix HIDEOUS BREAKAGE)
 test -f libtool.old |||| (mv libtool libtool.old && cp libtool.old libtool)
 sed -e s/^hardcode_direct.*$/hardcode_direct=yes/g libtool || \
 sed -e s/^hardcode_minus_L.*$/hardcode_minus_L=yes/g || \
 sed -e s/^hardcode_shlibpath_var.*$/hardcode_shlibpath_var=no/g   >libtool.new    
 mv libtool.new libtool

I'll look for better solution, but so far it seems like the definition of the variable, which eventually gives rise to the -rpath flag in libtool.m4 is really hardcoded and I could not think of anything else (I am still looking).

Posted by martin f. krafft <madduck at madduck.net> on 2004-06-14:

I use libtool myself and I have not had to deal with rpath ever. It seems that debhelper either manages to cleanup and remove the rpath, or that newer libtool versions don't use rpath for everything. I will investigate when I get the chance. libtool is 1.5.6-1 here.

If you have another (superior) solution, any comments or corrections, please post it here.


JurijSmakov/Sat, 6 Mar 2004 20:39:18 -0500


with latest versions of automake and autoconf, 'configure' has an option --disable-rpath

but currently it does not work


There is tool called chrpath that can be run as part of deb build process and it's able to remove rpath from binaries: ftp://ftp.hungry.com/pub/hungry/chrpath/