Wiki

• FrontPage
• RecentChanges
• FindPage
• HelpContents
• Python/LibraryStyleGuide

Style Guide for Packaging Python Libraries

NOTE: Also see Python/AppStyleGuide for alternative single package debian/rules.

The purpose of this document is to provide a quick style guide for packaging new Python libraries. It is not intended to supplant the Debian Python Policy and in fact, if you have not read that document, stop now and go read it. Instead, this page captures in example form, best practices for simple Python libraries that you might find on the Python Package Index. A few assumptions about the Python package you are working on are being made:

• The package is setuptools based. It should have a working setup.py (or setup.cfg) file and should generally be installable without modification into a virtualenv.

• Pure-Python or simple extension modules only. If you can do python3 setup.py install you'll be fine. For more complex building instructions, you will have to adapt these guidelines.

• The package supports at least Python 3.
• The package has a test suite.
• The package has some Sphinx-based documentation.

The design of this document is to capture best-practices that you can cargo cult into your own packages. It will forever be a work in progress.

Overview

There are many ways to build Python libraries for Debian, and each package presents its own corner cases and challenges. In modern Debian (e.g. unstable as of November 2013) and derivatives, the easiest way to build Python library packages is with the pybuild debhelper build system. This link has details on pybuild, including more examples for your cargo culting pleasure.

If you can't use pybuild, say because you want to be able to backport your package to distribution versions before pybuild was introduced, you may be interested in the historical version of this document, which contains all the gory details you had to implement manually, but which pybuild now takes care of for you.

The examples below are taken from the enum34 package, but modified for generality.

Note: It is highly recommended you explicitly set the dh build system to pybuild. By default, if there is a setup.py, dh will set the build system to python_distutils which only supports Python 2 and you'll get errors during package build as it tries to invoke some nonexistent Python 2 tools.

debian/control

Build-Depends

One very important thing to get right is the Build-Depends line in the source package stanza. setuptools/distribute-based packages have the nasty habit of downloading dependencies from PyPI if they are needed at python3 setup.py build time. If the package is available from the system (as would be the case when Build-Depends is up-to-date), then distribute will not try to download the package, otherwise it will try to download it. This is a huge no-no, and pybuild internally sets the http_proxy and https_proxy environment variables (to 127.0.0.1:9) to prevent this from happening.

dh_python3 will correctly fill in the installation dependencies (via ${python3:Depends}), but it cannot fill in the build dependencies. Take extra care in getting this right, and double check your build logs for illegal access to pypi.org.

You'll want to have at least the following build dependencies:

• debhelper (>= 11~)

• dh-python
• python3-all
• python3-setuptools

If the package has documentation in reStructuredText format, which often uses Sphinx as well, add these dependencies:

• python3-docutils
• python3-sphinx

Python versions

You may want this line in the source package stanza:

X-Python3-Version: >= 3.X

X is the minimum version of python3 the package requires. The stable version of Debian (as of 2024-12-11) includes python3.11, so if the minimum version is less than 3.11, there is no need to include X-Python3-Version. This is to support proper dependency generation for backports.

Documentation package

If you're building upstream documentation, it's best to put those files into a separate python-foo-doc package, like so:

Package: python-foo-doc
Architecture: all
Section: doc
Depends: ${sphinxdoc:Depends}, ${misc:Depends}
Description: Python frobnicator (common documentation)
 This package frobnicates a Python-based doohicky so that you no longer
 need to kludge a dingus to make the doodads work.
 .
 This is the common documentation package.

debian/rules

The contents of this file depends on whatever build system you choose, however we highly recommend the pybuild build system for its simplicity (at least for setup-based packages, i.e. the majority of those on PyPI).

Here is a debian/rules file for you to start with. First, I'll show you the whole thing, then I'll explain it line by line.

#! /usr/bin/make -f

#export DH_VERBOSE = 1
export PYBUILD_NAME = foo

%:
        dh $@ --with python3 --buildsystem=pybuild

The file starts with the standard #! line. I like adding the (commented out when uploading) DH_VERBOSE line because it can make build problems easier to debug.

The next line is:

export PYBUILD_NAME = foo

pybuild supports a number of variables which can be used to control such things as the destination directory for build artifacts. The defaults generally do the right thing, but you do need to at least tell pybuild the name of your package. Here, we're telling it that the name is foo. This should match the module name, so for example, in enum34, you'd see:

export PYBUILD_NAME = enum34

even though the binary package that get produced is python3-enum34.

The next line is the standard debhelper-based catch-all rule which is used to get the whole build started:

%:
        dh $@ --with python3 --buildsystem=pybuild

What's important to note here is that the dh_python3 helper is being invoked, and also that the build system that dh will use is pybuild. There's a lot of magic packed into this line, and the pybuild manpage goes into more detail, but essentially what this does is:

• Builds the Python 3 package for all supported Python 3 versions.
• Automatically detects the command to run the test suite.
• Runs the Python 3 test suite.

Once all that's done, it properly installs all the files into binary packages.

And that's it! You usually won't need a debian/python3-foo.install file even if you have multiple binary packages, because again, pybuild does the magic for you. You might need these if there are some non-standard installation tricks you need to implement.

debian/*.links

You may or may not need this file. It can be useful for Sphinx-based documentation, by providing the original reST files used to build the HTML documentation. Here is an example debian/python-foo-doc.links file.

usr/share/doc/python-foo-doc/html/_sources usr/share/doc/python-foo-doc/rst

Sphinx documentation

The Sphinx documentation section has been moved to its own page, to better accommodate new contributors who were confused by the Python specificity of its directions, and by how it was difficult to discover this information when learning how to build Sphinxdocs for a non-Python package. The new page is found at SphinxDocumentation.

debian/watch

Make sure that you have a working debian/watch file, for referencing upstream distribution tarballs. Usually, uscan is sufficient to download tarballs from the declarations in debian/watch (or, if you're using svn-buildpackage, this can be done automatically with svn-buildpackage --svn-download-orig), but if the source needs to be repackaged, you should then provide a get-orig-source (policy) target in debian/rules which does the job.

Since probably for most of our packages, the original tarball comes from the Python Package Index (a.k.a. PyPI or the Cheeseshop), here is a sample debian/watch file that you might be able to adapt for your own package:

version=3
opts=uversionmangle=s/(rc|a|b|c)/~$1/ \
https://pypi.debian.net/mypackage/mypackage-(.+)\.(?:zip|tgz|tbz|txz|(?:tar\.(?:gz|bz2|xz)))

Notice the use of pypi.debian.net instead of pypi.org. We're now running a redirector for PyPI to adjust for changes in the format of package listings on the latter site. In fact, it's trivial to get a valid debian/watch file for your package. Just do this from the parent of your debian directory:

$ curl -o debian/watch https://pypi.debian.net/mypackage/watch

Overriding

If you need to override the dh_python3 defaults, do it with an override method in your debian/rules file. Something like this will work:

override_dh_python3:
        dh_python3 --shebang=/usr/bin/python3

If you need to override the default testing regime (e.g. to use pytest), you can do it one of two ways. Either:

export PYBUILD_TEST_PYTEST = 1

or you can override the dh_auto_test rule:

override_dh_auto_test:
        dh_auto_test -- --test-pytest

Building python -dbg packages

As of 2022, you should not manually build separate -dbg packages for Python libraries. No special configuration is needed, and AutomaticDebugPackages will handle everything when you use dh. Please read https://lists.debian.org/debian-python/2021/09/msg00004.html for more information.

If you need details on how this was done before, please check current page's history.

autopkgtest

It's good idea to supply CI (DEP-8) tests for package. These tests check to make sure your built module can be imported, and that they are working in case any runtime dependencies change. autodep8 have support for Python packaging. To enable it, just add to debian/control:

Testsuite: autopkgtest-pkg-python

This will check if a Python module, named the same as the package, can be imported from the system. You could write more complex tests yourself.

You should check these tests locally by installing autopkgtest and autodep8 package. Then run autopkgtest command (see the manpage for details).

Gotchas

Executables and library packages

Let's say you have a Python package which results in libraries and an executable. What is the best practices for naming and organizing your binary packages?

Clearly you want to at least have:

• python3-foo -- the Python 3 library

but where should the /usr/bin/foo script go? You could put it in python3-foo, but that might not be ideal: users may not expect to find executables in packages that look like library package names, and it would make it difficult to support things like ?PyPy3 packages in future.

Here are some recommendations. We do not have a standard (though maybe we should):

• foo - this is nice because it parallels the /usr/bin/foo executable. In all likelihood, if the /usr/bin script name doesn't collide, then the foo binary package won't either.

• python3-foo - if the script is Python 3.x compatible.

MANIFEST.in issues and missing files

If the upstream package requires some non-standard files to be installed, it must include a MANIFEST.in file. Sometimes these files are omitted from the original tarball because they're mostly useful for creating the sdist. Case in point: a package contained a bunch of doctests in .rst files but python3 setup.py sdist does not include most .rst files by default. Thus, when pybuild went to build the package locally, the .rst files were omitted and the test suite failed because it could not find the doctest.

The solution for this is to either ensure that upstream includes the MANIFEST.in file, or that you add one before pybuild runs.

TODO

• describe egg related information, what should go in the binary package and what shouldn't.

• when to build-depend against (lib|)python3(-all|)(-dev|)

  • build-depending against lib* should never ever be needed.

  • the -dev variants are for when you need to compile C extension: if your project is pure python you don't want this

  • Using the -all variants will compile/test against all current Python(3) versions, rather than only the default one (i.e. binary .so modules will be compiled multiple times).

  • A bare dependency on python3-all-dev breaks cross building. Do use python3-all-dev:native, libpython3-all-dev instead. In a similar vein, rather than using python3-dev, it should be python3-dev:native, libpython3-dev.

  • The non -all variants are to be used when your project can't and is not building against all supported versions.

See also

• Python/Pybuild

• Python/GitPackaging

CategoryPackaging