Background

Two of the most common ways to test software are unit testing and integration testing - i.e. checking whether (a component of) a system works correctly within itself, and checking whether (a component of) a system interacts properly with its surroundings.

Projects often provide a command like make test that runs unit tests, and sometimes also runs integration tests between units. But packagers also need to check the whole package integrates properly with the rest of the system. For example, does a program crash if its configuration file is missing? Does a library fail to load because a build-dependency should have been a runtime dependency?

To run make test during the build process, see dh_auto_test. Or to create whole-package integration tests, continue reading this page.

Add tests to your package

First, add one of the following to your debian/control file:

# For Python packages:
Testsuite: autopkgtest-pkg-python
# For Perl packages:
Testsuite: autopkgtest-pkg-perl
# Other types of package:
Testsuite: autopkgtest-pkg-...
# Default value:
Testsuite: autopkgtest

For a complete list of autopkgtest-pkg- suites, see EXAMPLES OF PRODUCED TEST CONTROL FILES in autodep8's man page.

Second, define the tests you want to run. If you used an autopkgtest-pkg- suite, you can probably use the tests produced by autodep8. If you used the default autopkgtest suite, or need to extend the produced tests, create a new debian/tests/control file with a collection of blocks that look like this:

Tests: test1.sh, test2.py, test3.pl, ...
Depends: @builddeps@, pkg1, pkg2 [amd64] | pkg3 (>= 3), ...
Restrictions: needs-root, breaks-testbed, ...
...

For special dependency values, see dependency field syntax. For a list of restrictions, see defined restrictions. For the complete list of fields, see the specification.

Finally, create any files required by your autopkgtest-pkg- suite, plus files in debian/tests named after values in the Tests field. For example, debian/tests/test1.sh might look like:

# exit with a non-zero value to indicate an error:
if ! my-program --version > /dev/null 2>&1
then exit 1
fi

# or print to STDERR to indicate an error:
grep '(rude words)' /usr/share/my-resource/polite-words.txt >&2

# to indicate success, exit successfully without printing to STDERR:
exit 0

Run tests

autopkgtests checks that your package integrates with the wider system, so you need to pick a virtualization server to provide a wider system for it to run on.

Run tests in your development system

If you have installed all of a package's dependencies, you can run its tests on your main system with the null virtualization server:

# Test a source package:
autopkgtest /path/to/my_package.changes -- null
# Test the current directory:
autopkgtest --no-built-binaries -- null

Other common arguments to autopkgtest include --shell-fail (start a shell on test failure) and --apt-upgrade (update and upgrade your VM before running tests). For more information about command-line arguments and ways to specify a package, see the autopkgtest man page.

Run tests with unshare

unshare runs commands in a new namespace, without needing root access. Use this if you want to run tests without creating an environment in advance or giving your test-runner elevated privileges.

The following example creates a new environment running the i386 version of unstable. Some architectures may require QEMU user-mode emulation.

autopkgtest /path/to/my/package.changes \
    -- unshare --arch i386 --release unstable

Run tests with podman

podman is an alternative to docker that does not require root access. Use this if you want to run tests without elevated privileges, and want to reuse an existing container.

You will need an appropriate podman image. For details, see Build images and especially Build images from Debian environments.

autopkgtest /path/to/my/package.changes --apt-upgrade \
    -- podman --init <your-podman-image>

Run tests with QEMU

QEMU lets you run commands in a virtual machine. This may be necessary if your test needs to e.g. modify kernel variables, but consider user-mode emulation instead if you just need to run tests on a different architecture.

First, make a disk image with mmdebstrap-autopkgtest-build-qemu or autopkgtest-build-qemu. Adapt this to your requirements:

# Pick whichever of these best suits your workflow:
mmdebstrap-autopkgtest-build-qemu --efi --architecture=i386 unstable /path/to/my_disk.img
sudo autopkgtest-build-qemu unstable /path/to/my_disk.img http://deb.debian.org/debian i386

Then test it in the normal way:

# Only pass `--efi` if you used `mmdebstrap-autopkgtest-build-qemu`:
autopkgtest /path/to/my/package.changes --apt-upgrade \
    -- qemu --efi /path/to/my_disk.img

autopkgtest