Translation(s): none
Contents
- What is sbuild
- Setup
- Building packages
- Speeding up build process
-
Advanced Tips
- Cross-compiling packages
- External Commands
- Enabling experimental
- Build for experimental
- Enabling incoming.debian.org
- Using aliases
- Adding extra packages
- Remote build servers
- Validate package cleanup
- Using podman for autopkgtests
- Using qemu for autopkgtests
- Using /var/cache/apt/archives/ as package cache
What is sbuild
What is sbuild for:
sbuild is used on the official buildd network to build binary and source packages for all supported architectures.
- sbuild can also be used by individuals to test that their packages build in a minimal Debian installation.
- sbuild can help ensure that you haven't missed any build dependencies.
sbuild can test packages by running lintian, piuparts, autopkgtest or other commands.
sbuild can optionally produce a .changes file suitable for making a source-only upload using classic dput.
This last feature allows you to skip a separate dpkg-buildpackage -S execution after doing a final test binary build.
This last feature is irrelevant if you use modern dgit push-source for source-only uploads.
sbuild is fundamentally a tool for making binary packages. Its isolation and testing features are not directly relevant for making source-only uploads (but do help you test those packages before uploading).
An alternative to sbuild is pbuilder combined with cowbuilder
This main part of this page is intended as a short guide to sbuild. It documents how to set up sbuild and build packages with it. The later parts document optional enhancements to the simple setup described in the first section.
Setup
There are many ways to use sbuild. This section documents a modern configuration, using mmdebstrap and unshare, which allows you to build and test packages without being root. In this configuration packages are built and tested in a clean and ephemeral chroot that is extracted from a tarball. It is recommend to use the sbuild version from bookworm-backports.
1. Install necessary packages:
1 sudo apt install sbuild mmdebstrap uidmap
2. Create a directory for the chroot tarball
1 mkdir -p ~/.cache/sbuild
3. Create/Update the tarball
1 mmdebstrap --variant=buildd unstable ~/.cache/sbuild/unstable-amd64.tar.zst
You can use mmdebstrap arguments such as --include=gnupg,debhelper or --aptopt='Acquire::http { Proxy "http://127.0.0.1:3142"; }' to customise the chroot. The chroot is extracted into a temporary directory whenever it is used, ensuring builds and tests happen in a clean environment.
You can use choose the compression algorithm for the tarball by specifying the extension (.tar.xz, .tar.gz or plain .tar, etc). As of May 2024, ZST seems to provide the best size/time ratio. It certainly is the fastest on a Dell Precision 3800M, 16GB RAM, on an SSD drive (a computer from early 2015):
Format |
Tarball size |
Time |
.xz |
~100MB |
179,60s user 7,09s system 75% cpu 4:07,49 total |
.gz |
~150MB |
38,51s user 6,13s system 83% cpu 53,423 total |
.zst |
~139MB |
22,68s user 6,28s system 74% cpu 38,868 total |
4. Configure ~/.sbuildrc
The command below will create a new ~/.sbuildrc with suitable options. It will overwrite any existing file, so make sure you do not already have an ~/.sbuildrc or create a backup copy.
1 cat << "EOF" > ~/.sbuildrc
2 $chroot_mode = 'unshare';
3 $external_commands = { "build-failed-commands" => [ [ '%SBUILD_SHELL' ] ] };
4
5 # run autopkgtest after every build: use --no-run-autopkgtest to override
6 $run_autopkgtest = 1;
7 $autopkgtest_root_args = [''];
8 $autopkgtest_opts = [ '--apt-upgrade', '--', 'unshare', '--release', '%r', '--arch', '%a' ];
9
10 # run piuparts after every build: use --no-run-piuparts to override
11 $run_piuparts = 1;
12 $piuparts_root_args = ['PATH=/usr/sbin:/usr/bin:/sbin:/bin', 'unshare', '--pid', '--fork', '--mount-proc', '--map-root-user', '--map-auto'];
13 $piuparts_opts = ["--basetgz=$HOME/.cache/sbuild/%r-%a.tar.zst", '--fake-essential-packages=systemd-sysv', '--distribution=%r'];
14 EOF
There are many options for sbuild: see the man pages for sbuild(1) and sbuild.conf(5) for details.
Updating chroot manually (with unshare)
To keep the chroot updated it is recommended to simply delete it and recreate it by re-running the mmdebstrap command (see above): this is fast and makes sure the build environment remains clean.
If you really want to update an old chroot you can call:
1 sbuild-update --chroot-mode=unshare --update --upgrade --dist-upgrade --autoremove unstable
Building packages
Standalone
With properly configured ~/.sbuildrc as above, you can build a package by running the following from the source directory of the package. This will build a source package (.dsc) and then build all the binary packages in it.
1 sbuild
If you have a .dsc (generated by dpkg-buildpackage, git-buildpackage, etc) you can use that:
1 sbuild package_*.dsc
If you have deb-src lines configured for apt you can build any package without downloading the source code first:
1 sbuild -d unstable hello
Integration with git-buildpackage (gbp-buildpackage)
To build a package from a git repository managed with git-buildpackage run
1 gbp buildpackage --git-builder=sbuild
There is more information about packaging with git at PackagingWithGit.
Speeding up build process
Use apt-cacher-ng
1 sudo apt install apt-cacher-ng
Add this to mmdebstrap when creating the chroot tarball:
--aptopt='Acquire::http { Proxy "http://127.0.0.1:3142"; }'
Building on tmpfs [pre-trixie]
With trixie this is no longer needed as /tmp is a tmpfs already by default.
1 echo "\$unshare_tmpdir_template = '/dev/shm/tmp.sbuild.XXXXXXXXXX';" >> ~/.sbuildrc
This requires `/dev/shm' to be large enough to unpack the chroot and run the build process.
Using ccache with sbuild
ccache is a compiler wrapper that will cache compilation results (object files) from gcc and g++. If you repeatedly compile the same source code (or parts of it), ccache will shorten recompilation times by avoiding recompilation of files that have not changed between runs.
This is usually a significant saving during package development because only a few files are changed between runs.
It is also effective with packages that are frequently updated, because often only a few files actually change during updates to a software.
Add this to mmdebstrap when creating the chroot tarball:
1 --include=ccache --customize-hook='chroot "$1" update-ccache-symlinks'
(The update-ccache-symlinks is needed due to #632779)
Add this to your .sbuildrc:
1 cat << "EOF" >> ~/.sbuildrc
2 $build_environment = { "CCACHE_DIR" => "/build/ccache" };
3 $path = "/usr/lib/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games";
4 $build_path = "/build/package/";
5 $dsc_dir = "package";
6 $unshare_bind_mounts = [ { directory => "$HOME/.cache/ccache", mountpoint => "/build/ccache" } ];
7 $autopkgtest_opts = [ '--apt-upgrade', '--env=CCACHE_DIR=/build/ccache', '--env=PATH=/usr/lib/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games', '--', 'unshare', '--release', '%r', '--arch', '%a', '--prefix=/dev/shm/tmp.autopkgtest.', '--bind', '$HOME/.cache/ccache', '/build/ccache' ];
8 EOF
The sbuild user in the chroot needs to have access to $HOME/.cache/ccache which is on the host. You can either do chmod a+X "$HOME" "$HOME/.cache" and chmod -R a+rwX "$HOME/.cache/ccache" or use another location (note that files in /tmp are at risk of being wiped on reboot or by systemd's periodic cleanup).
Advanced Tips
Cross-compiling packages
sbuild also supports cross-compiling a package to build it for a different processor architecture. For example, you can build arm64 or ppc64el packages on an amd64 machine.
To build a package for arm64 in an amd64 chroot (as created above), you can use:
1 sbuild --host=arm64
NB: There's no need to cross-compile packages marked as arch:all as they are independent of architecture.
External Commands
sbuild supports running external commands at various stages of the build process. For example you can run a script inside the chroot after it has been setup.
To run /usr/local/bin/myscript, add it to the chroot tarball and include the following in ~/.sbuildrc:
sbuild supports some percent-escaped keywords here. For example, %SBUILD_CHANGES is changed to the path of the '.changes' file for a successfully built package.
Here is an example of adding a post build command to run /usr/local/bin/postbuildscript with %SBUILD_CHANGES as an argument.
Here is how to drop into a shell inside the ephemeral chroot if the build fails
See the 'EXTERNAL COMMANDS' section of the sbuild man page for more information on external commands.
Enabling experimental
To allow the build to use packages from the Debian experimental repository you can use the --extra-repository argument. You may also want to use the aspcud resolver:
1 sbuild --extra-repository='deb https://deb.debian.org/debian experimental main' --build-dep-resolver=aspcud mypkg.dsc
To exactly mimic the resolver used on the official buildds you need to add aspcud preferences that minimise the number of packages from experimental. See the DSA puppet repository at https://salsa.debian.org/dsa-team/mirror/dsa-puppet/-/blob/production/modules/buildd/templates/sbuild.conf.erb
1 sbuild --extra-repository='deb https://deb.debian.org/debian experimental main' --build-dep-resolver=aspcud --aspcud-criteria '-count(solution,APT-Release:=/a=experimental/),-removed,-changed,-new' mypkg.dsc
You can also use the aptitude resolver (used by *-backports), which for most packages should have the same effect.
1 sbuild --extra-repository='deb http://deb.debian.org/debian experimental main' --build-dep-resolver=aptitude mypkg.dsc
Another possibility is to have a dedicated ~/.sbuildrc.experimental configuration file and run sbuild via
1 SBUILD_CONFIG=~/.sbuildrc.experimental sbuild mypkg.dsc
with ~/.sbuildrc.experimental containing for example:
If you need to test a build against a versioned Build-Depends from experimental, you can use --add-depends:
1 sbuild --extra-repository='deb http://deb.debian.org/debian experimental main' --build-dep-resolver=aspcud --add-depends='foo-dev (>= 1.2.3-4)' mypkg.dsc
Build for experimental
If you want to build for unstable but upload to experimental, use, for example,
1 sbuild -d experimental -c unstable-amd64-sbuild mypkg.dsc
Alternatively, if you know that you always want to build your packages for experimental in a unstable chroot, just add experimental as an alias for your unstable chroot:
Enabling incoming.debian.org
If you want to build before the next dinstall you can add the incoming queue as an extra repositiory:
1 sbuild --extra-repository='deb http://incoming.debian.org/debian-buildd/ buildd-unstable main' mypkg.dsc
Using aliases
sbuild chooses the chroot based on the distribution in debian/changelog. You can change this with the -d option or you can use symlinks to set up aliases so that source package with UNRELEASED or experimental in debian/changelog are built in an unstable chroot (or so that -d sid can be used to build in unstable, etc):
Adding extra packages
It is often necessary to add extra binary packages as build dependencies for the package you are building. For example, you might want to provide a build dependency that is waiting in the NEW queue and so isn't available from the mirrors. To do this, use the --extra-package=/path/to/foo.deb option to sbuild (the path must be an absolute path on the host, and ~ is not automatically expanded to $HOME).
You might find that the output from the resolver is not helpful in determining which extra package you need to make available. In this case, it can be useful to pass --build-dep-resolver=aptitude which tends to provide more useful output (though you should remove it once you've figured out the problem).
Remote build servers
One advantage of cowbuilder over sbuild is that it supports offloading builds to a remote server with cowpoke. One way to do this with sbuild is to create a source package locally and transfer it to the remote machine for building. The latter can be done with dcmd. Example:
Validate package cleanup
Packages will fail to build twice in a row if the clean target of debian/rules does not restore the source directory to its initial state. Adding the following to your ~/.sbuildrc will help you to detect modifications:
$external_commands = {
"starting-build-commands" => [
'bash -c \'find %SBUILD_PKGBUILD_DIR -print0 |
sort -z |
while read -d $\'\\\'\'\0\'\\\'\' file; do
echo -n "$(stat -c "%n %F %%s" "${file}") "
if [ -f "${file}" ]; then
sha256sum "${file}" |
cut -d " " -f 1
else
echo
fi
done > /tmp/file-list.pre-build\''
],
"chroot-cleanup-commands" => [
'cd %SBUILD_PKGBUILD_DIR && ./debian/rules clean',
'bash -c \'find %SBUILD_PKGBUILD_DIR -print0 |
sort -z |
while read -d $\'\\\'\'\0\'\\\'\' file; do
echo -n "$(stat -c "%n %F %%s" "${file}") "
if [ -f "${file}" ]; then
sha256sum "${file}" |
cut -d " " -f 1
else
echo
fi
done > /tmp/file-list.post-build\'',
'diff /tmp/file-list.pre-build /tmp/file-list.post-build'
]
};
Using podman for autopkgtests
1 sudo apt install podman passt
2 mmdebstrap --include=libpam-systemd,systemd-resolved --customize-hook='sed "s/^deb /deb-src /" "$1/etc/apt/sources.list" > "$1/etc/apt/sources.list.d/src.list"' unstable | podman import - debian-unstable
3 cat << "EOF" >> ~/.sbuildrc
4 $autopkgtest_root_args = 'PATH=/usr/sbin:/usr/bin:/sbin:/bin';
5 $autopkgtest_opts = ['--shell-fail', '--apt-upgrade', '--', 'podman', '--init', 'debian-%r'];
6 EOF
Using qemu for autopkgtests
1 sudo apt install qemu-system
2 mmdebstrap-autopkgtest-build-qemu --boot=efi unstable $HOME/.cache/sbuild/unstable-amd64.img
3 cat << "EOF" >> ~/.sbuildrc
4 $autopkgtest_opts = ['--shell-fail', '--apt-upgrade', '--', 'qemu', '--cpus', '4', '--ram-size', '8096', '--efi', "$HOME/.cache/sbuild/%r-%a.img"];
5 EOF
Using /var/cache/apt/archives/ as package cache
1 sudo apt -o Dir::State::status=lock build-dep -d <pkg>
2 sudo apt -o Dir::State::status=lock install -d lintian
3 cd $(mktemp -d)
4 ln -s /var/cache/apt/archives/
5 apt-ftparchive packages . > Packages
6 apt-ftparchive release . > Release
7 python3 -m http.server 5678 --bind 127.0.0.1
8 sbuild --extra-repository="deb [trusted=yes] http://127.0.0.1:5678 ./" --chroot-setup-commands "rm /etc/apt/sources.list"