Translation(s): none
Contents
- What is sbuild
- Setup
- Building packages
- Speeding up build process
-
Advanced Tips
- Cross-compiling packages
- External Commands
- Enabling experimental
- Build for experimental
- Enabling incoming.debian.org
- Using aliases
- Adding extra packages
- Remote build servers
- Validate package cleanup
- Using podman for autopkgtests
- Using qemu for autopkgtests
- Using /var/cache/apt/archives/ as package cache
What is sbuild
What is sbuild for:
sbuild is used on the official buildd network to build binary and source packages for all supported architectures.
- sbuild can also be used by individuals to test that their packages build in a minimal Debian installation.
- sbuild can help ensure that you haven't missed any build dependencies.
sbuild can test packages by running lintian, piuparts, autopkgtest or other commands.
sbuild can optionally produce a .changes file suitable for making a source-only upload using classic dput.
This last feature allows you to skip a separate dpkg-buildpackage -S execution after doing a final test binary build.
This last feature is irrelevant if you use modern dgit push-source for source-only uploads.
sbuild is fundamentally a tool for making and testing binary packages (.deb) from a debian source package (.dsc). Its isolation features help you test your packages before uploading. To maintain and upload a package you will also want to use a package builder like git-buildpackage).
An alternative to sbuild is pbuilder combined with cowbuilder.
This main part of this page is intended as a short guide to sbuild. It documents how to set up sbuild and build packages with it. The later parts document optional enhancements to the simple setup described in the first section.
Setup
There are many ways to use sbuild. This section documents a modern configuration, using mmdebstrap and unshare, which allows you to build and test packages without being root. In this configuration packages are built and tested in a clean and ephemeral chroot that is extracted from a tarball. It is recommend to use the sbuild version from bookworm-backports.
1. Install necessary packages:
sudo apt install sbuild mmdebstrap uidmap
2. Create a directory for the chroot tarball
mkdir -p ~/.cache/sbuild
3. Create/Update the tarball
mmdebstrap --include=ca-certificates --skip=output/dev --variant=buildd unstable ~/.cache/sbuild/unstable-amd64.tar.zst https://deb.debian.org/debian
Change deb.debian.org to any mirror if you need. You can use any mmdebstrap arguments such as --include=gnupg,debhelper to customise the chroot. The chroot is extracted into a temporary directory whenever it is used, ensuring builds and tests happen in a clean environment.
If you are using apt-cacher-ng, specify http:// instead of https:// and enable the apt-cacher-ng proxy using --aptopt='Acquire::http { Proxy "http://127.0.0.1:3142"; }'. You can use any tarball compression extension you prefer.
mmdebstrap --include=ca-certificates --skip=output/dev --variant=buildd unstable ~/.cache/sbuild/unstable-amd64.tar.xz http://deb.debian.org/debian --aptopt='Acquire::http { Proxy "http://127.0.0.1:3142"; }'You can use choose the compression algorithm for the tarball by specifying the extension (.tar.xz, .tar.gz or plain .tar, etc). As of May 2024, ZST seems to provide the best size/time ratio. It certainly is the fastest on a Dell Precision 3800M, 16GB RAM, on an SSD drive (a computer from early 2015):
Format |
Tarball size |
Time |
.xz |
~100MB |
179,60s user 7,09s system 75% cpu 4:07,49 total |
.gz |
~150MB |
38,51s user 6,13s system 83% cpu 53,423 total |
.zst |
~139MB |
22,68s user 6,28s system 74% cpu 38,868 total |
4. Configure ~/.config/sbuild/config.pl
The command below will create a new ~/.config/sbuild/config.pl with suitable options. It will overwrite any existing file, so make sure you do not already have an ~/.config/sbuild/config.pl or create a backup copy.
Use ~/.sbuildrc with older sbuild
Before sbuild version 0.87.1, you must use ~/.sbuildrc instead of ~/.config/sbuild/config.pl. See #1087378 and 0.87.1 changelog.
1 cat << "EOF" > ~/.config/sbuild/config.pl
2 # Set the chroot mode to be unshare.
3 $chroot_mode = 'unshare';
4
5 $external_commands = { "build-failed-commands" => [ [ '%SBUILD_SHELL' ] ] };
6
7 # Uncomment below to specify the distribution; this is the same as passing `-d unstable` to sbuild.
8 # Specifying the distribution is currently required for piuparts when the changelog targets UNRELEASED. See #1088928.
9 #$distribution = 'unstable';
10
11 # Build Architecture: all packages; this is the same as passing `-A` to sbuild.
12 $build_arch_all = 1;
13
14 # Build the source package in addition to the other requested build artifacts; this is the same as passing `-s` to sbuild.
15 $build_source = 1;
16
17 # Produce a .changes file suitable for a source-only upload; this is the same as passing `--source-only-changes` to sbuild.
18 $source_only_changes = 1;
19
20 ## Run lintian after every build (in the same chroot as the build); use --no-run-lintian to override.
21 $run_lintian = 1;
22 # Display info tags.
23 $lintian_opts = ['--display-info', '--verbose', '--fail-on', 'error,warning', '--info'];
24 # Display info and pedantic tags, as well as overrides.
25 #$lintian_opts = ['--display-info', '--verbose', '--fail-on', 'error,warning', '--info', '--pedantic', '--show-overrides'];
26
27 ## Run autopkgtest after every build (in a new, clean, chroot); use --no-run-autopkgtest to override.
28 $run_autopkgtest = 1;
29 # Specify autopkgtest options. The commented example below is the default since trixie.
30 #$autopkgtest_opts = ['--apt-upgrade', '--', 'unshare', '--release', '%r', '--arch', '%a' ];
31
32 ## Run piuparts after every build (in a new, temporary, chroot); use --no-run-piuparts to override.
33 # this does not work in bookworm
34 $run_piuparts = 1;
35 # Build a temporary chroot.
36 $piuparts_opts = ['--no-eatmydata', '--distribution=%r', '--warn-on-leftovers-after-purge', '--fake-essential-packages=systemd-sysv'];
37 # Build a temporary chroot that uses apt-cacher-ng as a proxy to save bandwidth and time.
38 #$piuparts_opts = ['--no-eatmydata', '--distribution=%r', '--warn-on-leftovers-after-purge', '--bootstrapcmd=mmdebstrap --skip=check/empty --variant=minbase --aptopt="Acquire::http { Proxy \"http://127.0.0.1:3142\"; }"'];
39
40 EOF
Make sure to setup aliases if building for UNRELEASED or experimental unless you want to manually specify the distribution.
When using this example ~/.config/sbuild/config.pl, Piuparts generates a temporary, minimal chroot with every run, because doing so gives more complete results than using the one built with --variant=buildd. There are many options for sbuild: see the man pages for sbuild(1) and sbuild.conf(5) for details.
"Updating" chroot manually (with unshare)
To keep the chroot updated it is recommended to simply delete it and recreate it by re-running the mmdebstrap command (see above). This is fast and makes sure the build environment remains clean.
sbuild-update does not work with unshare.
Building packages
Standalone
With properly configured ~/.config/sbuild/config.pl as above, you can build a package by running the following from the source directory of the package. This will build a source package (.dsc) and then build all the binary packages in it.
sbuild
If you have a .dsc (generated by dpkg-buildpackage, git-buildpackage, etc) you can use that:
sbuild package_*.dsc
If you have deb-src lines configured for apt you can build any package without downloading the source code first:
sbuild -d unstable hello
Integration with git-buildpackage (gbp-buildpackage)
To build a package from a git repository managed with git-buildpackage run
gbp buildpackage --git-builder=sbuild
or put the following in ~/.gbp.conf:
[buildpackage] builder = sbuild
This creates a source package from the git repository (on the host machine), and then runs sbuild to build and test the binary packages (in a chroot) There is more information about packaging with git at PackagingWithGit.
Speeding up build process
Use apt-cacher-ng
sudo apt install apt-cacher-ng
Add this to the mmdebstrap options when creating the chroot tarball:
--aptopt='Acquire::http { Proxy "http://127.0.0.1:3142"; }'
Hash sum mismatch errors
apt-cacher-ng clients may (regularly) found errors about hash sum mismatches that yield to non-valid key signatures. One way to solve this is to clean the expired cache files, as documented at the apt-cacher-ng wiki page.
Building on tmpfs [pre-trixie]
Since trixie this is no longer needed because /tmp is now a tmpfs by default.
echo "\$unshare_tmpdir_template = '/dev/shm/tmp.sbuild.XXXXXXXXXX';" >> ~/.config/sbuild/config.pl
This requires /dev/shm to be large enough to unpack the chroot and run the build process: to temporarily enlarge it you can do (as root)
mount -o remount,size=2G /dev/shm
Using ccache with sbuild
ccache is a compiler wrapper that will cache compilation results (object files) from gcc and g++. If you repeatedly compile the same source code (or parts of it), ccache will shorten recompilation times by avoiding recompilation of files that have not changed between runs.
This is usually a significant saving during package development because only a few files are changed between runs.
It is also effective with packages that are frequently updated, because often only a few files actually change during updates to a software.
Add this to the mmdebstrap options when creating the chroot tarball:
--include=ccache --customize-hook='chroot "$1" update-ccache-symlinks'
(The update-ccache-symlinks is needed due to 632779)
Add this to your sbuild configuration:
1 cat << "EOF" >> ~/.config/sbuild/config.pl
2 $build_environment = { "CCACHE_DIR" => "/build/ccache" };
3 $path = "/usr/lib/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games";
4 $build_path = "/build/package/";
5 $dsc_dir = "package";
6 $unshare_bind_mounts = [ { directory => "$HOME/.cache/ccache", mountpoint => "/build/ccache" } ];
7 $autopkgtest_opts = [ '--apt-upgrade', '--env=CCACHE_DIR=/build/ccache', '--env=PATH=/usr/lib/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games', '--', 'unshare', '--release', '%r', '--arch', '%a', '--prefix=/dev/shm/tmp.autopkgtest.', '--bind', "$HOME/.cache/ccache", '/build/ccache' ];
8 EOF
The sbuild user in the chroot needs to have access to $HOME/.cache/ccache which is on the host. You can either do chmod a+X "$HOME" "$HOME/.cache" and chmod -R a+rwX "$HOME/.cache/ccache" on the host, or use another cache location (note that files in /tmp are at risk of being wiped on reboot or by systemd's periodic cleanup).
use eatmydata
It is unclear how much this helps if the chroot is extracted to a tmpfs, but running eatmydata may provide a speed boost by preventing data being written to the disk as often -- this is, obviously, with a risk of data corruption. It only affects the package build, not the rest of your system. To use it:
1. Include eatmydata in the chroot: add --include=eatmydata to the mmdebootstrap options when creating the chroot tarball.
2. Use eatmydata in the build: add the following to the sbuild configuration: this chroot is amd64: you can adapt the path on other architectures.
1 # use it in the chroot
2 $chroot-setup-commands => [ "export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libeatmydata.so" ];
3
4 # use it in the build
5 $build_environment ={
6 "LD_PRELOAD" => "/usr/lib/x86_64-linux-gnu/libeatmydata.so"
7 };
8
9 # add LD_PRELOAD to variables retained
10 $environment_filter=[Dpkg::BuildInfo::get_build_env_allowed(), '^LD_PRELOAD$']];
3. (You may want to add --no-eatmydata to $piuparts_opts since piuparts also enables eatmydata by default).
4. You can then call sbuild as usual: you may want to also install eatmydata on the host and call sbuild as
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libeatmydata.so sbuild
which may help speed up the unpacking of the chroot tarball.
Use schroot instead of unshare
sbuild can use chroots created by schroot, these use overlayfs which can be faster than unshare. However, this requires greater use of root than unshare: with schroot you can run the build, lintian and autopkgtests as a user (if root has granted access), but piuparts and creating a new chroot requires root (or permission to elevate privileges with sudo or similar).
Advanced Tips
Cross-compiling packages
sbuild also supports cross-compiling a package to build it for a different processor architecture. For example, you can build arm64 or ppc64el packages on an amd64 machine.
To build a package for arm64 in an amd64 chroot (as created above), you can use:
sbuild --host=arm64
NB: There's no need to cross-compile packages marked as arch:all as they are independent of architecture.
External Commands
sbuild supports running external commands at various stages of the build process. For example you can run a script inside the chroot after it has been setup.
To run /usr/local/bin/myscript, add it to the chroot tarball and include the following in your sbuild configuration:
sbuild supports some percent-escaped keywords here. For example, %SBUILD_CHANGES is changed to the path of the '.changes' file for a successfully built package.
Here is an example of adding a post build command to run /usr/local/bin/postbuildscript with %SBUILD_CHANGES as an argument.
Here is how to drop into a shell inside the ephemeral chroot if the build fails
See the 'EXTERNAL COMMANDS' section of the sbuild man page for more information on external commands.
Enabling experimental
To allow the build to use packages from the Debian experimental repository you can use the --extra-repository argument. You may also want to use the aspcud resolver:
sbuild --extra-repository='deb https://deb.debian.org/debian experimental main' --build-dep-resolver=aspcud mypkg.dsc
To exactly mimic the resolver used on the official buildds you need to add aspcud preferences that minimise the number of packages from experimental. See the DSA puppet repository at https://salsa.debian.org/dsa-team/mirror/dsa-puppet/-/blob/production/modules/buildd/templates/sbuild.conf.erb
sbuild --extra-repository='deb https://deb.debian.org/debian experimental main' --build-dep-resolver=aspcud --aspcud-criteria '-count(solution,APT-Release:=/a=experimental/),-removed,-changed,-new' mypkg.dsc
You can also use the aptitude resolver (used by *-backports), which for most packages should have the same effect.
sbuild --extra-repository='deb http://deb.debian.org/debian experimental main' --build-dep-resolver=aptitude mypkg.dsc
Another possibility is to have a dedicated ~/.config/sbuild/config.experimental.pl configuration file and run sbuild via
SBUILD_CONFIG=~/.config/sbuild/config.experimental.pl sbuild mypkg.dsc
with ~/.config/sbuild/config.experimental.pl containing for example:
$extra_repositories = [ 'deb http://deb.debian.org/debian experimental main' ]; $build_dep_resolver = 'aptitude';
If you need to test a build against a versioned Build-Depends from experimental, you can use --add-depends:
sbuild --extra-repository='deb http://deb.debian.org/debian experimental main' --build-dep-resolver=aspcud --add-depends='foo-dev (>= 1.2.3-4)' mypkg.dsc
Build for experimental
If you want to build for unstable but upload to experimental, use, for example,
sbuild -d experimental -c unstable-amd64-sbuild mypkg.dsc
Alternatively, if you know that you always want to build your packages for experimental in a unstable chroot, just add experimental as an alias for your unstable chroot:
Enabling incoming.debian.org
If you want to build before the next dinstall you can add the incoming queue as an extra repositiory:
sbuild --extra-repository='deb http://incoming.debian.org/debian-buildd/ buildd-unstable main' mypkg.dsc
Using aliases
sbuild chooses the chroot based on the distribution in debian/changelog. You can change this with the -d option or you can use symlinks to set up aliases so that source package with UNRELEASED or experimental in debian/changelog are built in an unstable chroot (or so that -d sid can be used to build in unstable, etc):
Adding extra packages
It is often necessary to add extra binary packages as build dependencies for the package you are building. For example, you might want to provide a build dependency that is waiting in the NEW queue and so isn't available from the mirrors ; or you're precisely trying to assess if your new package breaks anything before uploading. To do this, use the --extra-package=/path/to/foo.deb option to sbuild ; and if there are several such packages, --extra-package=/path/to/ will also scan this directory. The path must be an absolute path on the host, and ~ is not automatically expanded to $HOME!
You might find that the output from the resolver is not helpful in determining which extra package you need to make available. In this case, it can be useful to pass --build-dep-resolver=aptitude which tends to provide more useful output (though you should remove it once you've figured out the problem).
Remote build servers
One advantage of cowbuilder over sbuild is that it supports offloading builds to a remote server with cowpoke. One way to do this with sbuild is to create a source package locally and transfer it to the remote machine for building. The latter can be done with dcmd. Example:
Validate package cleanup
Packages will fail to build twice in a row if the clean target of debian/rules does not restore the source directory to its initial state. Adding the following to your sbuild configuration will help you to detect modifications:
$external_commands = {
"starting-build-commands" => [
'bash -c \'find %SBUILD_PKGBUILD_DIR -print0 |
sort -z |
while read -d $\'\\\'\'\0\'\\\'\' file; do
echo -n "$(stat -c "%n %F %%s" "${file}") "
if [ -f "${file}" ]; then
sha256sum "${file}" |
cut -d " " -f 1
else
echo
fi
done > /tmp/file-list.pre-build\''
],
"chroot-cleanup-commands" => [
'cd %SBUILD_PKGBUILD_DIR && ./debian/rules clean',
'bash -c \'find %SBUILD_PKGBUILD_DIR -print0 |
sort -z |
while read -d $\'\\\'\'\0\'\\\'\' file; do
echo -n "$(stat -c "%n %F %%s" "${file}") "
if [ -f "${file}" ]; then
sha256sum "${file}" |
cut -d " " -f 1
else
echo
fi
done > /tmp/file-list.post-build\'',
'diff /tmp/file-list.pre-build /tmp/file-list.post-build'
]
};
Using podman for autopkgtests
1 sudo apt install podman passt
2 mmdebstrap --include=libpam-systemd,systemd-resolved --customize-hook='sed "s/^deb /deb-src /" "$1/etc/apt/sources.list" > "$1/etc/apt/sources.list.d/src.list"' unstable | podman import - debian-unstable
3 cat << "EOF" >> ~/.config/sbuild/config.pl
4 $autopkgtest_root_args = 'PATH=/usr/sbin:/usr/bin:/sbin:/bin';
5 $autopkgtest_opts = ['--shell-fail', '--apt-upgrade', '--', 'podman', '--init', 'debian-%r'];
6 EOF
Using qemu for autopkgtests
1 sudo apt install qemu-system
2 mmdebstrap-autopkgtest-build-qemu --boot=efi unstable $HOME/.cache/sbuild/unstable-amd64.img
3 cat << "EOF" >> ~/.config/sbuild/config.pl
4 $autopkgtest_opts = ['--shell-fail', '--apt-upgrade', '--', 'qemu', '--cpus', '4', '--ram-size', '8096', '--efi', "$HOME/.cache/sbuild/%r-%a.img"];
5 EOF
Using /var/cache/apt/archives/ as package cache
1 sudo apt -o Dir::State::status=lock build-dep -d <pkg>
2 sudo apt -o Dir::State::status=lock install -d lintian
3 cd $(mktemp -d)
4 ln -s /var/cache/apt/archives/
5 apt-ftparchive packages . > Packages
6 apt-ftparchive release . > Release
7 python3 -m http.server 5678 --bind 127.0.0.1
8 sbuild --extra-repository="deb [trusted=yes] http://127.0.0.1:5678 ./" --chroot-setup-commands "rm /etc/apt/sources.list"