Translation(s): English
Links
Introduction
Pybuild is a newer (as in 2013) and more modular build system for Python packages.
Features:
Build Python2, Python3, ?PyPy with a one-liner dh command
- Support nose/pytest and tox testing
Note: New Python2 packages should not be uploaded to unstable. See Python2 Removal
debian/control
dh-python is a mandatory build dependency.
The pybuild dh integration script (not pybuild itself) uses the Build-Depends to figure out which interpreters it should build for. If the package provides Python 2.X modules it should build-depend on python-all (or python-all-dev in case of Python extensions), if it builds extensions with debug symbols: python-all-dbg; if it provides Python 3 script - python3 needs to be added to Build-Depends, etc. If you want to provide only python3 packages you can build-depend on python3-all (python3-all-dev, python3-all-dbg).
One very important thing to get right is the Build-Depends line in the source package stanza. setuptools/distribute-based packages have the nasty habit of downloading dependencies from PyPI if they are needed at python setup.py build time. If the package is available from the system (as would be the case when Build-Depends is up-to-date), then distribute will not try to download the package, otherwise it will try to download it. This is a huge no-no, and pybuild internally sets the http_proxy and https_proxy environment variables (to http[s]://127.0.0.1:9/) to prevent this from happening.
dh_python2 and dh_python3 will correctly fill in the installation dependencies (via ${python:Depends and ${python3:Depends respectively), but it cannot fill in the build dependencies. Take extra care in getting this right, and double check your build logs for illegal access to pypi.python.org.
debian/rules
Here is a debian/rules file for you to start with. First, I'll show you the whole thing, then I'll explain it line by line.
#export DH_VERBOSE=1
export PYBUILD_NAME=foo
%:
dh $@ --with python2,python3 --buildsystem=pybuildThe file starts with the standard #! line. I like adding the (commented out when uploading) DH_VERBOSE line because it can make build problems easier to debug.
The next line is:
export PYBUILD_NAME=foo
pybuild supports a number of variables which can be used to control such things as the destination directory for build artifacts for both the Python 2 and Python 3 builds. The defaults generally do the right thing, but you do need to at least tell pybuild the name of your package. Here, we're telling it that the name is foo. This should match the module name, so for example, in enum34, you'd see:
export PYBUILD_NAME=enum34
even though the binary packages that get produced are python-enum34 and python3-enum34.
The next line is the standard debhelper-based catch-all rule which is used to get the whole build started:
%:
dh $@ --with python2,python3 --buildsystem=pybuildWhat's important to note here is that both of the dh_python2 and dh_python3 helpers are being invoked, and also that the build system that dh will use is pybuild. There's a lot of magic packed into this line, and the pybuild manpage goes into more detail, but essentially what this does is:
- Builds the Python 2 package for all supported Python 2 versions.
- Builds the Python 3 package for all supported Python 3 versions.
- Automatically detects the command to run the test suite.
- Runs the Python 2 test suite.
- Runs the Python 3 test suite.
Once all that's done, it properly installs all the files into binary packages.
You usually won't need debian/python-foo.install or debian/python3-foo.install files even if you have multiple binary packages, because again, pybuild does the magic for you (if PYBUILD_NAME is correctly set). You might need these if there are some non-standard installation tricks you need to implement.
CUSTOMIZATION
Pybuild can be configured via command line or CDBS-like - by exporting environment variables in debian/rules that look like this:
PYBUILD_OPTION PYBUILD_OPTION_$interpreter PYBUILD_OPTION_$interpreter$version
For example:
- If you don't want to use debian/*.install files you can set DESTDIR for each interpreter (note that you can use "python2" for all python2.X interpreters and "python3" for all python3.X ones, this doesn't include -dbg interpreters, though):
export PYBUILD_DESTDIR_python2=debian/python-foo/ export PYBUILD_DESTDIR_python2-dbg=debian/python-foo-dbg/ export PYBUILD_DESTDIR_python3=debian/python3-foo/ export PYBUILD_DESTDIR_python3-dbg=debian/python3-foo-dbg/ export PYBUILD_DESTDIR_python3.1-dbg=debian/spam-pkg-with-python3.1-only/ export PYBUILD_DESTDIR_pypy=debian/pypy-foo/
- If you want to add some custom arguments to Python 3's install command:
export PYBUILD_INSTALL_ARGS_python3=--install-scripts=/usr/share/foo/
- You can disable (almost) any action via --disable / DISABLE env. var:
export PYBUILD_DISABLE=configure export PYBUILD_DISABLE_python3.1=test
- If you want to disable Build-Depends detection:
export PYBUILD_INTERPRETERS=python{version} python{version}-dbg
export PYBUILD_VERSIONS=2.7 3.3- If auto detection didn't detect the right build system:
export PYBUILD_SYSTEM=distutils
- If you want to add a custom command before or after each iteration:
export PYBUILD_BEFORE_BUILD=echo {version} >> '{dir}/enabled'
export PYBUILD_AFTER_INSTALL=rm -rf '{destdir}/{install_dir}/foo/tests'- If your build system is not yet supported, you can use custom build system:
export PYBUILD_SYSTEM=custom
export PYBUILD_CLEAN_ARGS=rm -rf {build_dir}/python{version}/
export PYBUILD_CONFIGURE_ARGS=./configure --python={version}
export PYBUILD_BUILD_ARGS=make build --dir={build_dir} --python={version}
export PYBUILD_INSTALL_ARGS=make --destdir={destdir} --python={version} --install-dir={install_dir}
export PYBUILD_TEST_ARGS_python2=cd {build_dir}; nosetests
export PYBUILD_TEST_ARGS_python3=cd {build_dir}; python{version} -m discover- If your package contains tests that are not installed/converted by distutils yet need 2to3 conversion:
export PYBUILD_BEFORE_TEST=cp -r {dir}/tests {build_dir}
export PYBUILD_BEFORE_TEST_python3=cp -r {dir}/tests {build_dir}; 2to3 -w {build_dir}/tests
export PYBUILD_AFTER_TEST=rm -rf {build_dir}/tests
To disable a specific test (pytest), note the '-k $exp'
export PYBUILD_TEST_ARGS=-k 'not test_that_needs_internet'
To drop more than one test add additional tests for exclusion by using and not test_foo to the variable.
export PYBUILD_TEST_ARGS=-k 'not test_1 and not test_2'
The '-k $exp' command skips running the test, but the tests are still included in the collection. This will cause failures if the tests have includes that cannot be satisfied. In that case, the --ignore command can be used instead, which skips adding the test to the collection.
export PYBUILD_TEST_ARGS=--ignore path/to/test.py
the --ignore command may be called multiple times if needed.
export PYBUILD_TEST_ARGS=--ignore path/to/test1.py --ignore path/to/test2.py
To package a program instead of a module:
export PYBUILD_INSTALL_ARGS=--install-lib=/usr/share/packagename/ --install-scripts=/usr/share/packagename/
Resources
Examples of packages using Pybuild on codesearch: search
introduction to pybuild and Python packaging talk at DebConf14: slides, video