Differences between revisions 20 and 21
Revision 20 as of 2011-05-17 22:46:59
Size: 11037
Editor: ?ckk
Comment: Update for python2.6+python2.7 with python2.6 as default; bump S-V
Revision 21 as of 2013-11-02 20:06:06
Size: 11037
Editor: BenFinney
Comment: examples assume Python 2.7 is default
Deletions are marked like this. Additions are marked like this.
Line 91: Line 91:
 1. that `python2.6` is the default Python version and that  1. that `python2.7` is the default Python version and that

Building Python Debug Extensions with dh

Using dh (the debhelper command sequencer), the process of creating both regular and -dbg packages for Python extensions can be automated to a larger extent. Because there is no canonical way to do this (yet) and even the basic objectives are sometimes misunderstood, this is often implemented wrong.

This page attempts to clarify the objectives, and to document one possible implementation. Of the various known approaches, the one taken by Jakub Wilk in python-djvu was chosen, as it is comparatively simple yet satisfies all of the objectives listed below. Portions of the Ubuntu Debug Build PyDbgBuilds documentation have been adapted as well.

This guide assumes that the developer is already familiar with packaging Python extensions in general, and with dh and python-support in particular.

This guide also assumes that you are developing for Debian's unstable distribution, and therefore may utilize features yet present only there. If you are developing for an older distribution or intend to provide backports for your package, #Alternatives lists alternative instructions for some of the steps described below. In particular, this guide assumes the following package versions:

  • python (>= 2.6)

  • debhelper (>= 8.1.0)

  • lintian (>= 2.5.0)

Goals

  1. a binary package containing the stripped extension, built for the regular interpreter
    • We will call this package python-foo.

  2. a binary package containing an extension built for the debug interpreter,
  3. as well as the stripped debugging symbols for use with gdb

    • We will call this package python-foo-dbg.

  4. Full unit tests (where available) after the build process for both the regular an the debug extensions.

debian/control

  • You must Build-Depend on debhelper (>= 8.1.0~) and python-all-dbg. This will trigger support for building debug extensions in dh. You must also Build-Depend on python-support (>= 0.90).

  • python-foo-dbg must Depend on python-foo (= ${binary:Version}). python-foo-dbg must have Priority: extra, Section: debug.

  • python-foo-dbg should Recommend the debug interpreter python-dbg. A Depends is not strictly necessary because by installing python-foo-dbg, someone might merely be interested in the stripped debugging symbols for use with gdb and not care for the debug extension.

  • If python-foo Depends on other Python extensions, then python-foo-dbg requires the -dbg packages of those extensions for use with the debug interpreter. Therefore, if python-foo were to Depend on python-bar, then python-foo-dbg should at least Recommend: python-bar-dbg. A Depends is not strictly necessary for the same reasons as listed in the previous point. A Build-Depends, however, is necessary if you intend to run build-time unit tests.

Here is an example for how debian/control would look like including all of the above:

Source: python-foo
Section: python
Priority: optional
Maintainer: Foo Bar <foo.bar@example.com>
Build-Depends:
    python-all-dev,
    python-all-dbg,
    debhelper (>= 8.1.0~),
    python-support (>= 0.90),
    python-bar,
    python-bar-dbg
Standards-Version: 3.9.2

Package: python-foo
Architecture: any
Depends:
    ${python:Depends},
    ${misc:Depends},
    ${shlibs:Depends},
    python-bar
Description: Python support for FOO
 This is the long package description.

Package: python-foo-dbg
Section: debug
Priority: extra
Architecture: any
Depends:
    ${python:Depends},
    ${misc:Depends},
    ${shlibs:Depends},
    python-foo (= ${binary:Version})
Recommends:
    python-dbg,
    python-bar-dbg
Description: Python support for FOO (debug extension)
 This is the long package description for the debug package.

Build

With python-all-dbg in Build-Depends, dh_auto_build in debian/rules will automatically run the build process twice for every supported Python version: first with the debug interpreter, then with the regular interpreter.

Assuming that

  1. python2.6 and python2.7 are present,

  2. that python-foo supports both,

  3. that python2.7 is the default Python version and that

  4. dh_auto_build detects a distutils-based setup,

dh_auto_build in debian/rules will perform (additional options passed by dh_auto_build have been omitted here for brevity):

  • python2.7-dbg setup.py build
    python-dbg setup.py build
    python2.7 setup.py build
    python setup.py build

Unit Tests

Unit tests for all of the built extensions are performed by overriding dh_auto_test in debian/rules and running them manually. For each run, an appropriate PYTHONPATH must be set. Assuming that the unit tests are run with the script mytests.py within the top directory, one way to accomplish this could be:

# Get the supported Python versions
PYVERS = $(shell pyversions -r -v)

override_dh_auto_test:
ifeq (,$(findstring nocheck,$(DEB_BUILD_OPTIONS)))
        set -e -x;\
        for py in $(PYVERS); do \
                PYTHONPATH=$(CURDIR)/build/lib.*-$$py  python$$py mytests.py ;\
                PYTHONPATH=$(CURDIR)/build/lib.*-$$py-pydebug python$$py-dbg mytests.py ;\
        done
endif

Install

Just as for #Build above, dh_auto_install in debian/rules will automate most of the installation process for us (additional options passed by dh_auto_install have been omitted here for brevity):

  • python2.7-dbg setup.py install
    python-dbg setup.py install
    python2.7 setup.py install
    python setup.py install

Because our source package is now building multiple binary packages, dh_auto_install will install everything to debian/tmp by default. We therefore need to move those files to their intended destinations. Specifically, everything should go into python-foo except for *_d.so: those go into python-foo-dbg. Two possible ways to accomplish this are:

Option 1: *.install files

  • Using *.install files, dh_install in debian/rules can move them to the correct destinations for us:

    debian/python-foo.install

    usr/lib/python*/*-packages/*/*[!_][!_].so
    usr/lib/python*/*-packages/*/*.py
    ... etc ...

    debian/python-foo-dbg.install

    usr/lib/python*/*-packages/*/*_d.so

Option 2: override_dh_install

If your source package doesn't build any binary packages other than python-foo and python-foo-dbg, a simpler approach can be used. It does not require *.install files; overriding dh_install in debian/rules is sufficient:

  • override_dh_install:
            # Install everything excluding the *_d.so debug extensions to python-foo
            dh_install -X"*_d.so" "debian/tmp/*" -p python-foo
            
            # Install the debug extensions to python-foo-dbg
            dh_install "debian/tmp/usr/lib/python*/*-packages/foo/*_d.so" -p python-foo-dbg
            
            # Continue with regular dh_install
            dh_install



The last step is to include the stripped debugging symbols in the -dbg package, which dh_strip will automatically do for us with the --dbg-package option.

You currently still need to manually rename a directory within the installation path, namely pyshared to pymodules. Lintian will warn about that, see 576014

override_dh_strip:
ifeq (,$(filter nostrip,$(DEB_BUILD_OPTIONS)))
        # Stripped symbols go into python-foo-dbg
        dh_strip --dbg-package=python-foo-dbg
        
        # Rename pyshared to pymodules
        cd debian/python-foo-dbg/usr/lib/debug/usr/lib && mv pyshared pymodules
endif

Results

  • python-foo contains a stripped regular extension
    • → Goal A achieved
  • python-foo-dbg will contain an extension built for the debug interpreter,
    • → Goal B achieved
  • as well as the stripped debugging symbols for use with gdb.

    • → Goal C achieved
  • Unit tests (if available) have been performed for both the regular extensions and the debug extensions:
    • → Goal D achieved

See Also

Alternatives

As previously mentioned, the above assumes that you are developing for unstable, and has corresponding package dependencies for debhelper et al. If this is not the case, you may need to modify some of the steps according to the rules below.

debhelper (<< 8.1.0)

At build time, if your package contains any scripts, the shebang will be changed to #!/usr/bin/python-dbg, which you'll have to change back manually (see 589759). The following workaround supplied by Joey Hess should work in those cases:

  • override_dh_auto_build:
            dh_auto_build -- --force

python (<< 2.6)

Building debug extensions for Python versions 2.5 and earlier or 2. requires a bit of hackery because the pathname for the debug build directory changed in 2.6 (from lib_d.* to lib.*-pydebug).

Assuming that the unit tests are run with the script mytests.py within the top directory:

# Get the supported Python versions
PYVERS = $(shell pyversions -r -v)

# Callable functions to determine the correct PYTHONPATH
pythonpath = $$(ls -d $(CURDIR)/build/lib.*-$(1))
pythonpath_dbg = $$(ls -d $(CURDIR)/build/lib_d.*-$(1) 2>/dev/null || ls -d $(CURDIR)/build/lib.*$(1)-pydebug)

override_dh_auto_test:
ifeq (,$(findstring nocheck,$(DEB_BUILD_OPTIONS)))
        set -e -x;\
        for py in $(PYVERS); do \
                PYTHONPATH=$(call pythonpath,$$py) python$$py mytests.py ;\
                PYTHONPATH=$(call pythonpath_dbg,$$py) python$$py-dbg mytests.py ;\
        done
endif


CategoryDebianDevelopment