Differences between revisions 1 and 220 (spanning 219 versions)
Revision 1 as of 2011-06-13 19:47:56
Size: 2244
Editor: ?ansgar
Comment:
Revision 220 as of 2022-05-26 19:40:16
Size: 32461
Editor: ?JochenSprickerhof
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
== sbuild ==

[[http://packages.debian.org/unstable/sbuild|sbuild]] is used on the official buildd network to build binary packages for all supported architectures.
It can also be used by individuals to prepare packages.
Alternatives to sbuild are [[cowbuilder]] and [[http://packages.debian.org/unstable/pbuilder|pbuilder]].
#language en
~-[[DebianWiki/EditorGuide#translation|Translation(s)]]: none-~
----

<<TableOfContents(2)>>

= What is sbuild =

What is DebianPts:sbuild:
 * DebianPts:sbuild is a convenience wrapper script of DebianPts:schroot to build '''binary''' package easily under specified chroot.
 * DebianPts:sbuild is used on the official [[buildd]] network to build binary and source packages for all supported architectures.
   * This helps good and timely code updates along Debian changes.
 * DebianPts:sbuild can also be used by individuals to test that their package builds in a minimal installation of Debian Unstable
   * DebianPts:sbuild now can help ensure that you haven't missed any build dependencies. (This feature was the advantage for pbuilder/cowbuilder over buildd)
   * DebianPts:sbuild can help ensure the goodness of package by integrating DebianPts:lintian, DebianPts:piuparts, DebianPts:autopkgtest
   * DebianPts:sbuild can optionally produce a .changes file suitable for making a [[SourceOnlyUpload|source-only upload]] using classic dput.
     * This last feature allows you to skip a separate `dpkg-buildpackage -S` execution after doing a final test '''binary''' build.
     * This last feature is irrelevant if you use modern `dgit push-source` for source-only uploads.

{{{{#!wiki note
DebianPts:sbuild is fundamentally a tool for making binary builds. None of its isolation and testing features are actually relevant for making source-only uploads.
}}}}

The alternative to sbuild are
 * DebianPts:pbuilder combined with [[cowbuilder]] (older code base)
 * DebianPts:propellor platform. (its [[http://hackage.haskell.org/package/propellor/docs/Propellor-Property-Sbuild.html|Propellor.Property.Sbuild]] module can perform most of this setup for you.)

This main part of this page is intended as a short guide to sbuild.
It documents how to set up sbuild and build packages with it. The
later parts of the page document optional enhancements to the simple
setup described in the first section.
Line 9: Line 35:
As root run
{{{
user=user
dist=unstable
# default to host architecture. you can also use i386 on amd64 or vice versa
arch=$(dpkg --print-architecture)
mirror=http://cdn.debian.net/debian

apt-get install sbuild
sbuild-adduser $user

# optional, but recommended if not using a local mirror:
echo /var/cache/apt/archives /var/cache/apt/archives none rw,bind 0 0 >>/etc/schroot/sbuild/fstab

mkdir -p /srv/chroot
sbuild-createchroot --keyring=/etc/apt/trusted.gpg $dist /srv/chroot/$dist-$arch-sbuild $mirror

# optional, but recommended:
cd /; schroot -c $dist-$arch-sbuild apt-get install eatmydata
# or, when using btrfs:
# cd /; schroot -c source:$dist-$arch-sbuild apt-get install eatmydata
echo command-prefix=eatmydata >>/etc/schroot/chroot.d/$dist-$arch-sbuild*
}}}

== Setup, with using btrfs snapshots ==

Using snapshots for the chroot environment allows to skip removing installed build dependencies and cleaning the build directory.
One filesystem supporting this is btrfs, but it is also possible to use LVM snapshots or aufs/unionfs.

To use btrfs snapshots with the above setup, run

{{{
btrfs subvolume create /srv/chroot/$dist-$arch-sbuild
}}}

before `sbuild-createchroot`. After the remaining steps run

{{{
mkdir -p /srv/chroot/snapshots
cat >>/etc/schroot/chroot.d/$dist-$arch-sbuild* <<EOT
type=btrfs-snapshot
btrfs-source-subvolume=/srv/chroot/$dist-$arch-sbuild
btrfs-snapshot-directory=/srv/chroot/snapshots
EOT
}}}

= Usage =

To build a package just run
{{{
sbuild -A -d unstable
}}}
in the source directory. You can also specify a different architecture with `--arch=$arch` or pass a .dsc file.

The chroot environment should be updated from time to time. Just run
{{{
sbuild-update -ugd unstable
}}}
Again, use `--arch=$arch` for a chroot environment for a architecture that differs from the host architecture.
We assume you are running at least Debian 11 bullseye (testing) using a `type=directory` chroot.

== Automatic setup using sbuild-debian-developer-setup ==

sbuild provides the package sbuild-debian-developer-setup which helps you setup an sbuild environment easily, if you're interested, make sure to take a look at the manpage at https://manpages.debian.org/unstable/sbuild/sbuild-debian-developer-setup.1

You may still want to further customize setting manually as the following to enjoy additional optional features.

== Using unshare with mmdebstrap (no root needed) ==

{{{#!highlight sh
sudo apt install sbuild mmdebstrap
mkdir -p ~/.cache/sbuild
mmdebstrap --variant=buildd unstable ~/.cache/sbuild/unstable-amd64.tar.xz
cat << "EOF" > ~/.sbuildrc
$chroot_mode = 'unshare';
$distribution = 'unstable';

$run_autopkgtest = 1;
$autopkgtest_root_args = '';
$autopkgtest_opts = [ '--apt-upgrade', '--', 'unshare', '--release', '%r', '--arch', '%a' ];
EOF
}}}

== Manual setup of sbuild ==

This {{{~/.sbuildrc}}} sets up sbuild in a way compatible with most contemporary Debian practices.

Tip: For creating a .sbuildrc file based on an example:

`cp /usr/share/doc/sbuild/examples/example.sbuildrc $HOME/.sbuildrc`

{{{#!highlight bash
sudo apt-get install sbuild schroot debootstrap apt-cacher-ng devscripts piuparts
sudo tee ~/.sbuildrc << EOF
##############################################################################
# PACKAGE BUILD RELATED (additionally produce _source.changes)
##############################################################################
# -d
$distribution = 'unstable';
# -A
$build_arch_all = 1;
# -s
$build_source = 1;
# --source-only-changes (applicable for dput. irrelevant for dgit push-source).
$source_only_changes = 1;
# -v
$verbose = 1;
# parallel build
$ENV{'DEB_BUILD_OPTIONS'} = 'parallel=5';
##############################################################################
# POST-BUILD RELATED (turn off functionality by setting variables to 0)
##############################################################################
$run_lintian = 1;
$lintian_opts = ['-i', '-I'];
$run_piuparts = 1;
$piuparts_opts = ['--schroot', 'unstable-amd64-sbuild', '--no-eatmydata'];
$run_autopkgtest = 1;
$autopkgtest_root_args = '';
$autopkgtest_opts = [ '--', 'schroot', '%r-%a-sbuild' ];

##############################################################################
# PERL MAGIC
##############################################################################
1;
EOF
sudo sbuild-adduser $LOGNAME
sudo ln -sf ~/.sbuildrc /root/.sbuildrc
}}}

... *logout* and *re-login* or use `newgrp sbuild` in your current shell

{{{#!highlight bash
sudo sbuild-createchroot --include=eatmydata,ccache unstable /srv/chroot/unstable-amd64-sbuild http://127.0.0.1:3142/ftp.us.debian.org/debian
}}}

This invokes `debootstrap` with `--variant=buildd` option to install required, `apt` ,`fakeroot` and `build-essential` packages in addition to specified `eatmydata`, and `ccache` packages. This also configures `/etc/hosts`, `/usr/sbin/policy-rc.d`, and `/etc/apt/sources.list` in the newly created chroot environment. The schroot configuration `/etc/schroot/chroot.d/sid-amd64-sbuild-$suffix` for this newly created chroot is created in the host system as:

{{{
[sid-amd64-sbuild]
description=Debian sid/amd64 autobuilder
groups=root,sbuild
root-groups=root,sbuild
profile=sbuild
type=directory
directory=/srv/chroot/sbuild-createchroot
union-type=overlay
}}}
    {{{{#!wiki note
The schroot created using {{{sbuild-createchroot}}} has a few more packages available by default than the ones used by [[https://ci.debian.net/|debci]] (for example {{{build-essential}}}) so if you are adding or removing dependencies, it is recommended to test the autopkgtest using {{{autopkgtest-build-lxc}}} and {{{autopkgtest your_package_dsc.dsc -- lxc -s -e autopkgtest-unstable}}} at least one to make sure your package won't fail when running on debci due to missing dependencies.
}}}}

Now for a brief explanation on what these commands do.

 1. Install sbuild and other useful packages onto the system.

 1. Create the sbuild template configuration to your home folder. (This example is setting up sbuild for personal use, instead of as part of a build server.)

   1. This setting allows you to avoid long commandline options for typical workflow needs and run all post-build tests.
   1. Adjust {{{parallel=5}}} to match your system resources.
   1. Setting the {{{$run_lintian}}} variable to 0 instead will disable running of lintian.
   1. Setting the {{{$run_piuparts}}} variable to 0 instead will disable running piuparts.
     1. '--no-eatmydata' for piuparts is needed when you configure schroot with "command-prefix=eatmydata" in `/etc/schroot/chroot.d/unstable-amd64-sbuild-*`.
   1. Setting the {{{$run_autopkgtest}}} variable to 0 instead will disable running autopkgtest.
   1. See [[https://manpages.debian.org/unstable/sbuild/sbuild.conf.5.en.html|the sbuild.conf manpage]] for more options.

 1. Update the active user group set to include sbuild.

   1. This will add your username so that it may use the {{{sbuild}}} command. Additional users may be added by running {{{sudo sbuild-adduser USER1 USER2 ...}}}. {{{sbuild-adduser}}} will prompt you to copy the template sbuild configuration in {{{/usr/share/doc/sbuild/examples/example.sbuildrc}}} to each user's {{{~/.sbuildrc}}}, to be used as their user sbuild configuration. You can customize sbuild settings here, but you usually won't need to customize anything. This should be done once per user.

 1. Use {{{sbuild-createchroot}}} to create a chroot used by sbuild meant for building packages targeting Debian unstable main and configured to be compatible with apt-cacher-ng. (See AptCacherNg )

   {{{#!wiki important
Use of classic mirror server URLs such as `http://ftp.us.debian.org` instead of its modern `http://deb.debian.org` is intentional choice to avoid DebianBug:986356 for apt-cacher-ng.
}}}

    {{{{#!wiki note
The chroot is saved in `/srv/chroot/unstable-amd64-sbuild`. It installs the packages ccache and eatmydata in the chroot in case you want to use some of the enhancements detailed below. The apt repository used is the mirror service `http://ftp.us.debian.org/debian` through apt-cacher-ng which will choose a suitable local mirror automatically. This can be changed to use a URL for a different mirror of the Debian archive. You can run this command once per distribution you want, and pass #
{{{--arch=i386}}} to create a chroot for a different architecture (the default is your host architecture).
}}}}

    {{{{#!wiki tip
Each post build package test feature can be turned off by changing the corresponding variable to 0 .
    }}}}


== Updating chroot manually ==

The chroot should be up-to-date before building packages. Use the DebianMan:sbuild-update to perform updates.

{{{#!highlight bash
sudo sbuild-update -udcar u
}}}

The arguments '-udcar' will tell sbuild-update to run an apt-get update, dist-upgrade, clean, autoclean, and autoremove in the chroot. "u" stands for unstable

All sbuild chroots built with {{{sbuild-createchroot}}} are created by schroot and will have a suffix of '-sbuild'. Thus to find the names of all sbuild chroots, run the following.

{{{#!highlight bash
schroot -l | grep sbuild
}}}


You can also pass {{{--apt-update}}} {{{--apt-distupgrade}}} to the individual sbuild invocation to update the temporary copy of the build chroot, but this won't cause any changes to happen in the persistent copy of the chroot (in the .tar.gz file). So if you are building more than once, you should run sbuild-update instead of relying on this.

== Building packages ==

With properly configured {{{~/.sbuildrc}}} as above, you can simply run the following to build a package from the source directory of a debianized package.

{{{#!highlight bash
sbuild
}}}

Alternatively, you may pass in the '.dsc' file of a package generated by {{{dpkg-buildpackage}}}, {{{git-buildpackage}}}, and so forth so that it may be built with sbuild. For example, to build sbuild from its '.dsc' file, do the following.

{{{#!highlight bash
sbuild sbuild_*.dsc
}}}

== Integration with gbp (gbp-buildpackage) ==

Sbuild can be integrated into gbp's workflow with little effort with properly configured {{{~/.sbuildrc}}} as above with {{{~/.gbp.conf}}} which includes:

{{{
[DEFAULT]
# the default build command:
builder = sbuild
}}}

(You can alternatively specify `--git-builder=sbuild` option to `gbp` command.)

Then, you can build package for normal source only upload and for source and binary upload just with:
{{{#!highlight bash
gbp buildpackage
}}}


More information on git packaging at [[PackagingWithGit]]

== Speeding up build process ==

Consider configuring sbuild/schroot with tmpfs or eatmydata. Otherwise, running dpkg especially by adt-run will slow the build a lot. (If modern fast NVMe SSD is used as storage device, you may not see much speed differences as you see with HDD.)

=== sbuild with eatmydata ===

Edit the corresponding {{{/etc/schroot/chroot.d/$dist-$arch-sbuild-$suffix}}} to append the line to use [[DebianPts:libeatmydata|eatmydata]]:
{{{
command-prefix=eatmydata
}}}

Note that piuparts invokes eatmydata by default and nested eatmydata
invocations don't work. To deal with this pass --no-eatmydata to
piuparts in your ~/.sbuildrc, or set up a separate schroot profile for
piuparts as described above.

=== sbuild with tmpfs (schroot) ===

You could configure {{{/etc/fstab}}} to mount all short term contents on tmpfs. (Added bonus: less access to precious SSD.)

{{{
# For speeding up sbuild/schroot and prevent SSD wear-out
none /var/lib/schroot/session tmpfs uid=root,gid=root,mode=0755 0 0
none /var/lib/schroot/union/overlay tmpfs uid=root,gid=root,mode=0755 0 0
none /var/lib/sbuild/build tmpfs uid=sbuild,gid=sbuild,mode=2770 0 0
}}}

=== sbuild with tmpfs (unshare) ===

{{{#!highlight bash
echo "$unshare_tmpdir_template = '/dev/shm/tmp.sbuild.XXXXXXXXXX';" >> ~/.sbuildrc
}}}

== Using "ccache" with sbuild ==

`ccache` is a compiler wrapper that will cache compilation results (produced object files) from `gcc` and `g++`; if you repeatedly compile the same source code (or parts of it), `ccache` will greatly shorten compilation times by avoiding recompilation of files that it has cached earlier.

This is especially useful during package development, when you might have
to rebuild a package with a long compilation phase several times. It is
also effective with packages that are frequently updated, because often
only a few files actually change during updates to a software.

In order to prepare your `sbuild` environment for `ccache`, first perform
the following setup in the host environment (i.e., outside the chroot), as
user `root`:
{{{#!highlight bash
dir=/var/cache/ccache-sbuild
install --group=sbuild --mode=2775 -d $dir
env CCACHE_DIR=$dir ccache --max-size 4G
cat >>/etc/schroot/sbuild/fstab <<END
$dir $dir none rw,bind 0 0
END
}}}

This assumes that you trust all members of the `sbuild` group.

It is perfectly fine to share the cache among chroots, even for different
architectures. `ccache` honours the compiler name, size and timestamp as
well as the command line when calculating hash values. At least one of
these will differ between builds of the same file for different
architectures.

Next place the following script into `$dir/sbuild-setup`:
{{{#!highlight bash
cat >$dir/sbuild-setup <<END
#!/bin/sh
export CCACHE_DIR=$dir
export CCACHE_UMASK=002
export CCACHE_COMPRESS=1
unset CCACHE_HARDLINK
export PATH="/usr/lib/ccache:\$PATH"
exec "\$@"
END
}}}
and make it executable:
{{{#!highlight bash
chmod a+rx $dir/sbuild-setup
}}}

Then for each chroot (`$dist-$arch-sbuild`) where you want to enable
`ccache`

 1. Install `ccache` inside the chroot by running
 {{{#!highlight bash
 sudo sbuild-shell $dist-$arch
 apt-get install ccache
 exit
 }}}

 2. and edit the corresponding configuration file in `/etc/schroot/chroot.d/` by appending the line
 {{{
 command-prefix=/var/cache/ccache-sbuild/sbuild-setup
 }}}
 (Multiple `command-prefix` can be joined with commas, in case you already have one configured; see `eatmydata` below.)


=== Using eatmydata and ccache with sbuild ===

If you want to combine eatmydata with the ccache instructions from above then use:
{{{
command-prefix=/var/cache/ccache-sbuild/sbuild-setup,eatmydata
}}}


= Basic tips =

== Delete a chroot ==

{{{#!highlight bash
sudo rm -r /srv/chroot/unstable-amd64-sbuild/
sudo rm /etc/schroot/chroot.d/unstable-amd64-sbuild-* /etc/sbuild/chroot/unstable-amd64-sbuild
}}}

Also see `sbuild-destroychroot(1)`.

Note: You may be unable to completely delete a chroot sometimes with a failing message, "Device or resource busy". This may mean that chroot has open or active session(s) which you should close or end first. See below for how to end sessions.

== Cleaning up schroot session ==

Sometimes you can end up with dangling chroot sessions potentially
taking up valuable system resources (find them with `schroot -l
--all`). This command is useful:

{{{#!highlight bash
sudo schroot --end-session --all-sessions
}}}

== Customizations of sbuild chroots ==

Sometimes it is desirable to further customize your sbuild chroot environment (type=directory) permanently. Typical customizations done are installing more packages inside the chroot, modifying /etc/apt/sources.list, and installing custom scripts to be run inside the chroot.

You start a root shell prompt inside the unstable chroot.

{{{#!highlight bash
sudo sbuild-shell u
}}}

Here, "u" is short for "unstable". You can specify longer form such as unstable-amd64 here, too.

This starts a schroot session for `source:unstable-$arch-sbuild`. Any modifications done inside the chroot will be saved upon exiting. Inside this chroot, you may run {{{apt-get}}} commands to install or remove packages as desired. For example, to install ccache, do the following in the chroot session.

{{{#!highlight bash
apt-get install ccache
exit
}}}


Modifications to /etc/apt/sources.list inside the unstable chroot can be done from outside the chroot with root privileges using your favorite editor as:

{{{#!highlight bash
sudo -e /srv/chroot/unstable-amd64-sbuild/etc/apt/sources.list
}}}

Proceed to edit the sources.list file as you see fit, then save.

To add scripts inside the chroot, simply place the scripts in the directory {{{/srv/chroot/unstable-amd64-sbuild/usr/local/bin/}}}

Be sure to make the scripts executable.

{{{#!highlight bash
sudo chmod a+x /srv/chroot/unstable-amd64-sbuild/usr/local/bin/*
}}}

Other modifications may be done to the chroot, either by running commands available within the chroot session, or by running commands with root privileges via the secondary terminal session. Once you are done making modifications to the chroot, simply exit the session. Inside the chroot session, do the following.

{{{#!highlight bash
exit
}}}

After exiting, your modifications will be saved and made available for every new chroot session created afterwards.

{{{{#!wiki note
Think negative impacts of modifications to the chroot environment used by the sbuild command before making modifications to it. If you simply want to have a chroot environment with many pre-loaded packages for debugging packages, it is better to set up a custom chroot environment using [[Schroot]] directly and use it.
}}}}

= Advanced Tips =

== Cross compiling ==

sbuild also supports cross-compiling a package to build for a different processor architecture e.g arm64 or ppc64el etc( if a package has an `arch:all package` option, then there's no point trying to cross build, since that's already arch independent): to build a package which is e.g. only buildable on mips, in the amd64 chroot from the above example, you can use:

{{{#!highlight bash
sbuild --host=mips
}}}

To build packages available from the apt repositories used in the
sbuild chroot, just pass in a package name (older versions of sbuild
required {{{$package_$version}}}). For example to build the latest
sbuild:

{{{#!highlight bash
sbuild -d unstable sbuild
}}}

Everything needed to build the latest sbuild version will be
downloaded from the repositories and will be saved in your current
directory after building of the packages is finished.

== External Commands ==

sbuild supports running external commands at various stages of the build process. This is useful for cases such as running a script inside the chroot after it has been setup. As an example, to run a script in {{{/usr/local/bin/myscript}}}, edit the {{{$external_commands}}} in the sbuild configuration file {{{~/.sbuildrc}}} as follows.

{{{#!highlight perl
$external_commands = {
                        'post-build-commands' => [],
                        'chroot-setup-commands' => ['/usr/local/bin/myscript'],
                        'chroot-cleanup-commands' => [],
                        'pre-build-commands' => []
                      };
}}}

sbuild can also translate certain percent escaped keywords for external commands during certain portions of a build. For example, in post build commands, {{{%SBUILD_CHANGES}}} is changed to the path of the '.changes' file for a successfully built package.

Here is an example of adding a post build command to run {{{/usr/local/bin/postbuildscript}}} with {{{%SBUILD_CHANGES}}} as an argument.
{{{#!highlight perl
$external_commands = {
                        'post-build-commands' => ['/usr/local/bin/postbuildscript', '%SBUILD_CHANGES'],
                        'chroot-setup-commands' => ['/usr/local/bin/myscript'],
                        'chroot-cleanup-commands' => [],
                        'pre-build-commands' => []
                      };
}}}

See the 'EXTERNAL COMMANDS' section of the sbuild man page for more information on external commands.

== Enabling experimental ==

The Debian experimental repository can be added dynamically on top of an existing unstable chroot during each sbuild run that requires experimental. You can use either the aspcud resolver (in use on the experimental buildds) or aptitude (used by *-backports).

{{{#!highlight bash
sbuild --extra-repository='deb http://deb.debian.org/debian experimental main' --build-dep-resolver=aspcud mypkg.dsc
}}}

or

{{{#!highlight bash
sbuild --extra-repository='deb http://deb.debian.org/debian experimental main' --build-dep-resolver=aptitude mypkg.dsc
}}}

Another possibility is to have a dedicated {{{~/.sbuildrc.experimental}}} configuration file and run sbuild via

{{{#!highlight bash
SBUILD_CONFIG=~/.sbuildrc.experimental sbuild mypkg.dsc
}}}

with {{{~/.sbuildrc.experimental}}} containing for example:

{{{#!highlight bash
$extra_repositories = [ 'deb http://deb.debian.org/debian experimental main' ];
$build_dep_resolver = 'aptitude';
}}}

If you need to test a build against a versioned Build-Depends from experimental, you can add `--add-depends`:

{{{#!highlight bash
sbuild --extra-repository='deb http://deb.debian.org/debian experimental main' --build-dep-resolver=aspcud --add-depends='foo-dev (>= 1.2.3-4)' mypkg.dsc
}}}


== Build for experimental ==

If you want to build for unstable but upload to experimental. Use `schroot -l --all-source-chroots` to get the name of the chroot (unstable-amd64-sbuild in this case).

{{{#!highlight bash
sbuild -d experimental -c unstable-amd64-sbuild mypkg.dsc
}}}

Alternatively, if you know that you always want to build your packages for experimental in a sid chroot, just add `experimental` as an alias of your sid schroot to your sid schroot configuration:

{{{
aliases=experimental
}}}

== Enabling incoming.debian.org ==

Another useful repository to add as `--extra-repository` option is `deb http://incoming.debian.org/debian-buildd/ buildd-unstable main` in case you want to build or do a new upload before [[https://ftp-master.debian.org/dinstall.html|the next dinstall]]

{{{#!highlight bash
sbuild --extra-repository='deb http://incoming.debian.org/debian-buildd/ buildd-unstable main' mypkg.dsc
}}}

== Disabling network access for dpkg-buildpackage ==

=== Using iptables ===

<!> This doesn't seem to work with recent sbuild (2016/05)

Source packages must be buildable without accessing any remote machines. On the other hand, the build process needs network access for the installation of the build dependencies. Conveniently, the build dependencies are installed by the "root" user while `dpkg-buildpackage` is run under fakeroot by the user running sbuild. So the following will deny any network access during the build process through blocking traffic originating from any process owned by the sbuild group while the root user will still have network access:

{{{#!highlight bash
sudo iptables -I OUTPUT -m owner --gid-owner sbuild ! -d 127.0.0.0/8 -j DROP
sudo -u sbuild sbuild mypkg.dsc
}}}

A better fix would be if `schroot` allowed to unshare the network namespace for the `dpkg-buildpackage` invocation. See bugs DebianBug:802850 and DebianBug:802849

=== Using linux network namespaces ===

As an alternative, you can also use linux network namespace to isolate sbuild. In its most basic form, it will create a namespace with only the loopback interface (in a down state). This means no network access outside the namespace. Your dependencies must already be installed in the chroot. The main drawback of this method is that it requires root privileges.

Run the following to build your package without network access:

{{{#!highlight bash
sudo ip netns add no-net
sudo ip netns exec no-net sbuild --no-apt-update --no-apt-upgrade --no-apt-distupgrade mypkg.dsc
}}}

A more elaborate setup could include a pair of peer interfaces, shared between the main and no-net namespace, without default routes, fetching packages from a local proxy.

== Using aliases ==

In Debian, an unstable chroot is used for building in a number of situations like building an unpacked source package with `UNRELEASED` in `debian/changelog` or building packages for experimental. Furthermore, distributions have alternative names like sid/unstable or experimental/rc-buggy. Sbuild selects the chroot to use either from `debian/changelog` (so it would be nice if UNRELEASED would trigger a build in an unstable chroot) or through the `-d` option which also sets the `Distribution` value in the resulting `.changes` file (so it would be handy if saying `-d sid` were enough and one wouldn't have to use the `-c` option to type `unstable-amd64-sbuild` manually). All of this can be solved by using schroot aliases. My sid schroot config says:

{{{
aliases=UNRELEASED,sid,rc-buggy,experimental
}}}

This means, that this schroot will be used for packages having UNRELEASED in their `debian/changelog`, for packages I build with `-d sid` (because writing out `unstable` is too long) as well as for packages I build for experimental.

== Adding extra packages ==

It is often necessary to add extra binary packages as build
dependencies. For example, you might want to make a package available
as a build dependency that is waiting in the NEW queue and so isn't
available from the mirrors. To do this, use the
`--extra-package=./foo.deb` option to sbuild.

You might find that the output from the resolver is not helpful in
determining which extra package you need to make available. In this
case, it can be useful to pass `--build-dep-resolver=aptitude` which
tends to provide more useful output (though you should remove it once
you've figured out the problem).

== Remote build servers ==

One advantage of cowbuilder over sbuild is that it supports offloading builds to a remote server with cowpoke. Unfortunately, that command is specifically crafted for cowbuilder, so you need something else for sbuild.

The trick is to create a source package and transfer it to the remote machine for building. The latter can be done with `dcmd`. Example:

{{{#!highlight bash
dpkg-buildpackage -S
dcmd scp ../foo-1.0.dsc example.net:build-area
ssh example.net sbuild build-area/foo-1.0.dsc
}}}

== Validate package cleanup ==

Packages will fail to build twice in a row if the clean target of
`debian/rules` do not restore the source directory to its initial state.
Adding the following to your `~/.sbuildrc` will help you to detect
modifications:

{{{
$external_commands = {
    "starting-build-commands" => [
        'bash -c \'find %SBUILD_PKGBUILD_DIR -print0 |
                  sort -z |
                  while read -d $\'\\\'\'\0\'\\\'\' file; do
                      echo -n "$(stat -c "%n %F %%s" "${file}") "
                      if [ -f "${file}" ]; then
                          sha256sum "${file}" |
                              cut -d " " -f 1
                      else
                              echo
                      fi
                  done > /tmp/file-list.pre-build\''
    ],
    "chroot-cleanup-commands" => [
        'cd %SBUILD_PKGBUILD_DIR && ./debian/rules clean',
        'bash -c \'find %SBUILD_PKGBUILD_DIR -print0 |
                  sort -z |
                  while read -d $\'\\\'\'\0\'\\\'\' file; do
                      echo -n "$(stat -c "%n %F %%s" "${file}") "
                      if [ -f "${file}" ]; then
                          sha256sum "${file}" |
                              cut -d " " -f 1
                      else
                              echo
                      fi
                  done > /tmp/file-list.post-build\'',
        'diff /tmp/file-list.pre-build /tmp/file-list.post-build'
    ]
};
}}}

== Associated schroot environment for development ==

For development, creating the latest minimal unstable chroot environment to execute programs in unstable environment as sbuild does is time consuming. Let’s create an associated package pre-loaded unstable chroot shell environment and use it directly from the schroot command. (This assumes you have already created chroot environment as above.)

{{{#!highlight bash
$ sudo cp -a /srv/chroot/unstable-amd64-sbuild-$suffix /srv/chroot/unstable-amd64-dev
$ sudo cp -a /etc/schroot/desktop /etc/schroot/dev
$ sudo tee /etc/schroot/chroot.d/unstable-amd64-dev << EOF
[unstable-dev]
description=Debian unstable/amd64 chroot development environment
groups=root,sbuild
root-groups=root,sbuild
source-groups=root,sbuild
source-root-groups=root,sbuild
profile=dev
type=directory
preserve-environment=true
directory=/srv/chroot/unstable-amd64-dev
union-type=overlay
command-prefix=eatmydata
EOF
}}}

Let’s install basic build environment packages to this chroot in `/srv/chroot/unstable-amd64-dev`.

{{{#!highlight bash
$ cd /
$ sudo schroot -c source:unstable-dev -u root
(unstable-dev)root@hostname:/# apt update; apt full-upgrade
(unstable-dev)root@hostname:/# apt install aptitude vim nano- mc devscripts sudo gitk
(unstable-dev)root@hostname:/# exit
}}}

Let's further customize mount points by editing `/etc/schroot/osamu/fstab` as needed. Let's also copy setting from the host PC to the chroot as needed.

Let's add 2 shell alias definitions to {{{~/.bashrc}}} on the host system as follows and enable convenient access to this chroot environment.

{{{
alias devs="schroot -c source:unstable-dev"
alias devx="xhost +si:localuser:$(id -un) ; schroot -c chroot:unstable-dev ; xhost -"
}}}

 * The `devs` command opens a shell prompt and allows us to modify the source chroot environment. The changes made are persistent. This is good for installing and upgrading packages in the chroot.
 * The `devx` command opens a shell prompt and allows us to work in a chroot session to build a source tree with access to X applications such as gitk as usual. The changes made aren’t persistent.


== Missing space in /build and /srv/chroot ==

It is possible that you lack space for builds even if `/srv/chroot` has plenty of space. This can happen at build time on large packages (e.g. Libreoffice) because sbuild decompresses the packages in `/build` in the chroot, which is a bind mount to `/var/lib/sbuild/build`. If you are on a workstation where `/var` is shared with the root partition, you may sometimes not have enough space for the largest packages. The symptom will be an error message like this:

{{{
E: Disc space is probably not sufficient for building.
I: Source needs 2703644 KiB, while 4089748 KiB is free.)
}}}

But then when you actually look at the `df`, you have plenty of space! This happens because sbuild cleans up after itself on failure and gives back the disk space. If you follow disk space usage more closely during the build, you will notice that `/var` will take up more and more space until that failure.

A workaround for this is to change the configuration of the `/build` bind mount. This is done in `/etc/schroot/sbuild/fstab`. For example, the following configuration uses `/home/build` instead:

{{{
# Mount a large scratch space for the build, so we don't use up
# space of the chroot itself.
#/var/lib/sbuild/build /build none rw,bind 0 0
/home/build /build none rw,bind 0 0
}}}

If `/srv/chroot` lacks space but some other directory has plenty of space, you can make space by adding the similar bind mount from the other directory to `/srv/chroot`.

== Using /var/cache/apt/archives/ as package cache ==

{{{#!highlight bash
sudo apt -o Dir::State::status=lock build-dep -d <pkg>
sudo apt -o Dir::State::status=lock install -d lintian
cd $(mktemp -d)
ln -s /var/cache/apt/archives/
apt-ftparchive packages . > Packages
apt-ftparchive release . > Release
python3 -m http.server 5678 --bind 127.0.0.1
sbuild --extra-repository="deb [trusted=yes] http://127.0.0.1:5678 ./" --chroot-setup-commands "rm /etc/apt/sources.list"
}}}

----

CategoryPackaging

Translation(s): none


What is sbuild

What is sbuild:

  • sbuild is a convenience wrapper script of schroot to build binary package easily under specified chroot.

  • sbuild is used on the official buildd network to build binary and source packages for all supported architectures.

    • This helps good and timely code updates along Debian changes.
  • sbuild can also be used by individuals to test that their package builds in a minimal installation of Debian Unstable

    • sbuild now can help ensure that you haven't missed any build dependencies. (This feature was the advantage for pbuilder/cowbuilder over buildd)

    • sbuild can help ensure the goodness of package by integrating lintian, piuparts, autopkgtest

    • sbuild can optionally produce a .changes file suitable for making a source-only upload using classic dput.

      • This last feature allows you to skip a separate dpkg-buildpackage -S execution after doing a final test binary build.

      • This last feature is irrelevant if you use modern dgit push-source for source-only uploads.

sbuild is fundamentally a tool for making binary builds. None of its isolation and testing features are actually relevant for making source-only uploads.

The alternative to sbuild are

This main part of this page is intended as a short guide to sbuild. It documents how to set up sbuild and build packages with it. The later parts of the page document optional enhancements to the simple setup described in the first section.

Setup

We assume you are running at least Debian 11 bullseye (testing) using a type=directory chroot.

Automatic setup using sbuild-debian-developer-setup

sbuild provides the package sbuild-debian-developer-setup which helps you setup an sbuild environment easily, if you're interested, make sure to take a look at the manpage at https://manpages.debian.org/unstable/sbuild/sbuild-debian-developer-setup.1

You may still want to further customize setting manually as the following to enjoy additional optional features.

Using unshare with mmdebstrap (no root needed)

   1 sudo apt install sbuild mmdebstrap
   2 mkdir -p ~/.cache/sbuild
   3 mmdebstrap --variant=buildd unstable ~/.cache/sbuild/unstable-amd64.tar.xz
   4 cat << "EOF" > ~/.sbuildrc
   5 $chroot_mode = 'unshare';
   6 $distribution = 'unstable';
   7 
   8 $run_autopkgtest = 1;
   9 $autopkgtest_root_args = '';
  10 $autopkgtest_opts = [ '--apt-upgrade', '--', 'unshare', '--release', '%r', '--arch', '%a' ];
  11 EOF

Manual setup of sbuild

This ~/.sbuildrc sets up sbuild in a way compatible with most contemporary Debian practices.

Tip: For creating a .sbuildrc file based on an example:

cp /usr/share/doc/sbuild/examples/example.sbuildrc $HOME/.sbuildrc

   1 sudo apt-get install sbuild schroot debootstrap apt-cacher-ng devscripts piuparts
   2 sudo tee ~/.sbuildrc << EOF
   3 ##############################################################################
   4 # PACKAGE BUILD RELATED (additionally produce _source.changes)
   5 ##############################################################################
   6 # -d
   7 $distribution = 'unstable';
   8 # -A
   9 $build_arch_all = 1;
  10 # -s
  11 $build_source = 1;
  12 # --source-only-changes (applicable for dput. irrelevant for dgit push-source).
  13 $source_only_changes = 1;
  14 # -v
  15 $verbose = 1;
  16 # parallel build
  17 $ENV{'DEB_BUILD_OPTIONS'} = 'parallel=5';
  18 ##############################################################################
  19 # POST-BUILD RELATED (turn off functionality by setting variables to 0)
  20 ##############################################################################
  21 $run_lintian = 1;
  22 $lintian_opts = ['-i', '-I'];
  23 $run_piuparts = 1;
  24 $piuparts_opts = ['--schroot', 'unstable-amd64-sbuild', '--no-eatmydata'];
  25 $run_autopkgtest = 1;
  26 $autopkgtest_root_args = '';
  27 $autopkgtest_opts = [ '--', 'schroot', '%r-%a-sbuild' ];
  28 
  29 ##############################################################################
  30 # PERL MAGIC
  31 ##############################################################################
  32 1;
  33 EOF
  34 sudo sbuild-adduser $LOGNAME
  35 sudo ln -sf ~/.sbuildrc /root/.sbuildrc

... *logout* and *re-login* or use newgrp sbuild in your current shell

   1 sudo sbuild-createchroot --include=eatmydata,ccache unstable /srv/chroot/unstable-amd64-sbuild http://127.0.0.1:3142/ftp.us.debian.org/debian

This invokes debootstrap with --variant=buildd option to install required, apt ,fakeroot and build-essential packages in addition to specified eatmydata, and ccache packages. This also configures /etc/hosts, /usr/sbin/policy-rc.d, and /etc/apt/sources.list in the newly created chroot environment. The schroot configuration /etc/schroot/chroot.d/sid-amd64-sbuild-$suffix for this newly created chroot is created in the host system as:

[sid-amd64-sbuild]
description=Debian sid/amd64 autobuilder
groups=root,sbuild
root-groups=root,sbuild
profile=sbuild
type=directory
directory=/srv/chroot/sbuild-createchroot
union-type=overlay
  • The schroot created using sbuild-createchroot has a few more packages available by default than the ones used by debci (for example build-essential) so if you are adding or removing dependencies, it is recommended to test the autopkgtest using autopkgtest-build-lxc and autopkgtest your_package_dsc.dsc -- lxc -s -e autopkgtest-unstable at least one to make sure your package won't fail when running on debci due to missing dependencies.

Now for a brief explanation on what these commands do.

  1. Install sbuild and other useful packages onto the system.
  2. Create the sbuild template configuration to your home folder. (This example is setting up sbuild for personal use, instead of as part of a build server.)
    1. This setting allows you to avoid long commandline options for typical workflow needs and run all post-build tests.
    2. Adjust parallel=5 to match your system resources.

    3. Setting the $run_lintian variable to 0 instead will disable running of lintian.

    4. Setting the $run_piuparts variable to 0 instead will disable running piuparts.

      1. '--no-eatmydata' for piuparts is needed when you configure schroot with "command-prefix=eatmydata" in /etc/schroot/chroot.d/unstable-amd64-sbuild-*.

    5. Setting the $run_autopkgtest variable to 0 instead will disable running autopkgtest.

    6. See the sbuild.conf manpage for more options.

  3. Update the active user group set to include sbuild.
    1. This will add your username so that it may use the sbuild command. Additional users may be added by running sudo sbuild-adduser USER1 USER2 .... sbuild-adduser will prompt you to copy the template sbuild configuration in /usr/share/doc/sbuild/examples/example.sbuildrc to each user's ~/.sbuildrc, to be used as their user sbuild configuration. You can customize sbuild settings here, but you usually won't need to customize anything. This should be done once per user.

  4. Use sbuild-createchroot to create a chroot used by sbuild meant for building packages targeting Debian unstable main and configured to be compatible with apt-cacher-ng. (See AptCacherNg )

    • Use of classic mirror server URLs such as http://ftp.us.debian.org instead of its modern http://deb.debian.org is intentional choice to avoid 986356 for apt-cacher-ng.

      • The chroot is saved in /srv/chroot/unstable-amd64-sbuild. It installs the packages ccache and eatmydata in the chroot in case you want to use some of the enhancements detailed below. The apt repository used is the mirror service http://ftp.us.debian.org/debian through apt-cacher-ng which will choose a suitable local mirror automatically. This can be changed to use a URL for a different mirror of the Debian archive. You can run this command once per distribution you want, and pass # --arch=i386 to create a chroot for a different architecture (the default is your host architecture).

        Each post build package test feature can be turned off by changing the corresponding variable to 0 .

Updating chroot manually

The chroot should be up-to-date before building packages. Use the sbuild-update to perform updates.

   1 sudo sbuild-update -udcar u

The arguments '-udcar' will tell sbuild-update to run an apt-get update, dist-upgrade, clean, autoclean, and autoremove in the chroot. "u" stands for unstable

All sbuild chroots built with sbuild-createchroot are created by schroot and will have a suffix of '-sbuild'. Thus to find the names of all sbuild chroots, run the following.

   1 schroot -l | grep sbuild

You can also pass --apt-update --apt-distupgrade to the individual sbuild invocation to update the temporary copy of the build chroot, but this won't cause any changes to happen in the persistent copy of the chroot (in the .tar.gz file). So if you are building more than once, you should run sbuild-update instead of relying on this.

Building packages

With properly configured ~/.sbuildrc as above, you can simply run the following to build a package from the source directory of a debianized package.

   1 sbuild

Alternatively, you may pass in the '.dsc' file of a package generated by dpkg-buildpackage, git-buildpackage, and so forth so that it may be built with sbuild. For example, to build sbuild from its '.dsc' file, do the following.

   1 sbuild sbuild_*.dsc

Integration with gbp (gbp-buildpackage)

Sbuild can be integrated into gbp's workflow with little effort with properly configured ~/.sbuildrc as above with ~/.gbp.conf which includes:

[DEFAULT]
# the default build command:
builder = sbuild

(You can alternatively specify --git-builder=sbuild option to gbp command.)

Then, you can build package for normal source only upload and for source and binary upload just with:

   1 gbp buildpackage

More information on git packaging at PackagingWithGit

Speeding up build process

Consider configuring sbuild/schroot with tmpfs or eatmydata. Otherwise, running dpkg especially by adt-run will slow the build a lot. (If modern fast NVMe SSD is used as storage device, you may not see much speed differences as you see with HDD.)

sbuild with eatmydata

Edit the corresponding /etc/schroot/chroot.d/$dist-$arch-sbuild-$suffix to append the line to use eatmydata:

command-prefix=eatmydata

Note that piuparts invokes eatmydata by default and nested eatmydata invocations don't work. To deal with this pass --no-eatmydata to piuparts in your ~/.sbuildrc, or set up a separate schroot profile for piuparts as described above.

sbuild with tmpfs (schroot)

You could configure /etc/fstab to mount all short term contents on tmpfs. (Added bonus: less access to precious SSD.)

# For speeding up sbuild/schroot and prevent SSD wear-out
none /var/lib/schroot/session        tmpfs uid=root,gid=root,mode=0755 0 0
none /var/lib/schroot/union/overlay  tmpfs uid=root,gid=root,mode=0755 0 0
none /var/lib/sbuild/build           tmpfs uid=sbuild,gid=sbuild,mode=2770 0 0

sbuild with tmpfs (unshare)

   1 echo "$unshare_tmpdir_template = '/dev/shm/tmp.sbuild.XXXXXXXXXX';" >> ~/.sbuildrc

Using "ccache" with sbuild

ccache is a compiler wrapper that will cache compilation results (produced object files) from gcc and g++; if you repeatedly compile the same source code (or parts of it), ccache will greatly shorten compilation times by avoiding recompilation of files that it has cached earlier.

This is especially useful during package development, when you might have to rebuild a package with a long compilation phase several times. It is also effective with packages that are frequently updated, because often only a few files actually change during updates to a software.

In order to prepare your sbuild environment for ccache, first perform the following setup in the host environment (i.e., outside the chroot), as user root:

   1 dir=/var/cache/ccache-sbuild
   2 install --group=sbuild --mode=2775 -d $dir
   3 env CCACHE_DIR=$dir ccache --max-size 4G
   4 cat >>/etc/schroot/sbuild/fstab <<END
   5 $dir $dir none rw,bind 0 0
   6 END

This assumes that you trust all members of the sbuild group.

It is perfectly fine to share the cache among chroots, even for different architectures. ccache honours the compiler name, size and timestamp as well as the command line when calculating hash values. At least one of these will differ between builds of the same file for different architectures.

Next place the following script into $dir/sbuild-setup:

   1 cat >$dir/sbuild-setup <<END
   2 #!/bin/sh
   3 export CCACHE_DIR=$dir
   4 export CCACHE_UMASK=002
   5 export CCACHE_COMPRESS=1
   6 unset CCACHE_HARDLINK
   7 export PATH="/usr/lib/ccache:\$PATH"
   8 exec "\$@"
   9 END

and make it executable:

   1 chmod a+rx $dir/sbuild-setup

Then for each chroot ($dist-$arch-sbuild) where you want to enable ccache

  1. Install ccache inside the chroot by running

       1  sudo sbuild-shell $dist-$arch
       2  apt-get install ccache
       3  exit
    
  2. and edit the corresponding configuration file in /etc/schroot/chroot.d/ by appending the line

     command-prefix=/var/cache/ccache-sbuild/sbuild-setup

    (Multiple command-prefix can be joined with commas, in case you already have one configured; see eatmydata below.)

Using eatmydata and ccache with sbuild

If you want to combine eatmydata with the ccache instructions from above then use:

command-prefix=/var/cache/ccache-sbuild/sbuild-setup,eatmydata

Basic tips

Delete a chroot

   1 sudo rm -r /srv/chroot/unstable-amd64-sbuild/
   2 sudo rm /etc/schroot/chroot.d/unstable-amd64-sbuild-* /etc/sbuild/chroot/unstable-amd64-sbuild

Also see sbuild-destroychroot(1).

Note: You may be unable to completely delete a chroot sometimes with a failing message, "Device or resource busy". This may mean that chroot has open or active session(s) which you should close or end first. See below for how to end sessions.

Cleaning up schroot session

Sometimes you can end up with dangling chroot sessions potentially taking up valuable system resources (find them with `schroot -l --all`). This command is useful:

   1 sudo schroot --end-session --all-sessions

Customizations of sbuild chroots

Sometimes it is desirable to further customize your sbuild chroot environment (type=directory) permanently. Typical customizations done are installing more packages inside the chroot, modifying /etc/apt/sources.list, and installing custom scripts to be run inside the chroot.

You start a root shell prompt inside the unstable chroot.

   1 sudo sbuild-shell u

Here, "u" is short for "unstable". You can specify longer form such as unstable-amd64 here, too.

This starts a schroot session for source:unstable-$arch-sbuild. Any modifications done inside the chroot will be saved upon exiting. Inside this chroot, you may run apt-get commands to install or remove packages as desired. For example, to install ccache, do the following in the chroot session.

   1 apt-get install ccache
   2 exit

Modifications to /etc/apt/sources.list inside the unstable chroot can be done from outside the chroot with root privileges using your favorite editor as:

   1 sudo -e /srv/chroot/unstable-amd64-sbuild/etc/apt/sources.list

Proceed to edit the sources.list file as you see fit, then save.

To add scripts inside the chroot, simply place the scripts in the directory /srv/chroot/unstable-amd64-sbuild/usr/local/bin/

Be sure to make the scripts executable.

   1 sudo chmod a+x /srv/chroot/unstable-amd64-sbuild/usr/local/bin/*

Other modifications may be done to the chroot, either by running commands available within the chroot session, or by running commands with root privileges via the secondary terminal session. Once you are done making modifications to the chroot, simply exit the session. Inside the chroot session, do the following.

   1 exit

After exiting, your modifications will be saved and made available for every new chroot session created afterwards.

Think negative impacts of modifications to the chroot environment used by the sbuild command before making modifications to it. If you simply want to have a chroot environment with many pre-loaded packages for debugging packages, it is better to set up a custom chroot environment using Schroot directly and use it.

Advanced Tips

Cross compiling

sbuild also supports cross-compiling a package to build for a different processor architecture e.g arm64 or ppc64el etc( if a package has an arch:all package option, then there's no point trying to cross build, since that's already arch independent): to build a package which is e.g. only buildable on mips, in the amd64 chroot from the above example, you can use:

   1 sbuild --host=mips

To build packages available from the apt repositories used in the sbuild chroot, just pass in a package name (older versions of sbuild required $package_$version). For example to build the latest sbuild:

   1 sbuild -d unstable sbuild

Everything needed to build the latest sbuild version will be downloaded from the repositories and will be saved in your current directory after building of the packages is finished.

External Commands

sbuild supports running external commands at various stages of the build process. This is useful for cases such as running a script inside the chroot after it has been setup. As an example, to run a script in /usr/local/bin/myscript, edit the $external_commands in the sbuild configuration file ~/.sbuildrc as follows.

   1 $external_commands = {
   2                         'post-build-commands' => [],
   3                         'chroot-setup-commands' => ['/usr/local/bin/myscript'],
   4                         'chroot-cleanup-commands' => [],
   5                         'pre-build-commands' => []
   6                       };

sbuild can also translate certain percent escaped keywords for external commands during certain portions of a build. For example, in post build commands, %SBUILD_CHANGES is changed to the path of the '.changes' file for a successfully built package.

Here is an example of adding a post build command to run /usr/local/bin/postbuildscript with %SBUILD_CHANGES as an argument.

   1 $external_commands = {
   2                         'post-build-commands' => ['/usr/local/bin/postbuildscript', '%SBUILD_CHANGES'],
   3                         'chroot-setup-commands' => ['/usr/local/bin/myscript'],
   4                         'chroot-cleanup-commands' => [],
   5                         'pre-build-commands' => []
   6                       };

See the 'EXTERNAL COMMANDS' section of the sbuild man page for more information on external commands.

Enabling experimental

The Debian experimental repository can be added dynamically on top of an existing unstable chroot during each sbuild run that requires experimental. You can use either the aspcud resolver (in use on the experimental buildds) or aptitude (used by *-backports).

   1 sbuild --extra-repository='deb http://deb.debian.org/debian experimental main' --build-dep-resolver=aspcud mypkg.dsc

or

   1 sbuild --extra-repository='deb http://deb.debian.org/debian experimental main' --build-dep-resolver=aptitude mypkg.dsc

Another possibility is to have a dedicated ~/.sbuildrc.experimental configuration file and run sbuild via

   1 SBUILD_CONFIG=~/.sbuildrc.experimental sbuild mypkg.dsc

with ~/.sbuildrc.experimental containing for example:

   1 $extra_repositories = [ 'deb http://deb.debian.org/debian experimental main' ];
   2 $build_dep_resolver = 'aptitude'; 

If you need to test a build against a versioned Build-Depends from experimental, you can add --add-depends:

   1 sbuild --extra-repository='deb http://deb.debian.org/debian experimental main' --build-dep-resolver=aspcud --add-depends='foo-dev (>= 1.2.3-4)' mypkg.dsc

Build for experimental

If you want to build for unstable but upload to experimental. Use schroot -l --all-source-chroots to get the name of the chroot (unstable-amd64-sbuild in this case).

   1 sbuild -d experimental -c unstable-amd64-sbuild mypkg.dsc

Alternatively, if you know that you always want to build your packages for experimental in a sid chroot, just add experimental as an alias of your sid schroot to your sid schroot configuration:

aliases=experimental

Enabling incoming.debian.org

Another useful repository to add as --extra-repository option is deb http://incoming.debian.org/debian-buildd/ buildd-unstable main in case you want to build or do a new upload before the next dinstall

   1 sbuild --extra-repository='deb http://incoming.debian.org/debian-buildd/ buildd-unstable main' mypkg.dsc

Disabling network access for dpkg-buildpackage

Using iptables

<!> This doesn't seem to work with recent sbuild (2016/05)

Source packages must be buildable without accessing any remote machines. On the other hand, the build process needs network access for the installation of the build dependencies. Conveniently, the build dependencies are installed by the "root" user while dpkg-buildpackage is run under fakeroot by the user running sbuild. So the following will deny any network access during the build process through blocking traffic originating from any process owned by the sbuild group while the root user will still have network access:

   1 sudo iptables -I OUTPUT -m owner --gid-owner sbuild ! -d 127.0.0.0/8 -j DROP
   2 sudo -u sbuild sbuild mypkg.dsc

A better fix would be if schroot allowed to unshare the network namespace for the dpkg-buildpackage invocation. See bugs 802850 and 802849

Using linux network namespaces

As an alternative, you can also use linux network namespace to isolate sbuild. In its most basic form, it will create a namespace with only the loopback interface (in a down state). This means no network access outside the namespace. Your dependencies must already be installed in the chroot. The main drawback of this method is that it requires root privileges.

Run the following to build your package without network access:

   1 sudo ip netns add no-net
   2 sudo ip netns exec no-net sbuild --no-apt-update --no-apt-upgrade --no-apt-distupgrade mypkg.dsc

A more elaborate setup could include a pair of peer interfaces, shared between the main and no-net namespace, without default routes, fetching packages from a local proxy.

Using aliases

In Debian, an unstable chroot is used for building in a number of situations like building an unpacked source package with UNRELEASED in debian/changelog or building packages for experimental. Furthermore, distributions have alternative names like sid/unstable or experimental/rc-buggy. Sbuild selects the chroot to use either from debian/changelog (so it would be nice if UNRELEASED would trigger a build in an unstable chroot) or through the -d option which also sets the Distribution value in the resulting .changes file (so it would be handy if saying -d sid were enough and one wouldn't have to use the -c option to type unstable-amd64-sbuild manually). All of this can be solved by using schroot aliases. My sid schroot config says:

aliases=UNRELEASED,sid,rc-buggy,experimental

This means, that this schroot will be used for packages having UNRELEASED in their debian/changelog, for packages I build with -d sid (because writing out unstable is too long) as well as for packages I build for experimental.

Adding extra packages

It is often necessary to add extra binary packages as build dependencies. For example, you might want to make a package available as a build dependency that is waiting in the NEW queue and so isn't available from the mirrors. To do this, use the --extra-package=./foo.deb option to sbuild.

You might find that the output from the resolver is not helpful in determining which extra package you need to make available. In this case, it can be useful to pass --build-dep-resolver=aptitude which tends to provide more useful output (though you should remove it once you've figured out the problem).

Remote build servers

One advantage of cowbuilder over sbuild is that it supports offloading builds to a remote server with cowpoke. Unfortunately, that command is specifically crafted for cowbuilder, so you need something else for sbuild.

The trick is to create a source package and transfer it to the remote machine for building. The latter can be done with dcmd. Example:

   1 dpkg-buildpackage -S
   2 dcmd scp ../foo-1.0.dsc example.net:build-area
   3 ssh example.net sbuild build-area/foo-1.0.dsc

Validate package cleanup

Packages will fail to build twice in a row if the clean target of debian/rules do not restore the source directory to its initial state. Adding the following to your ~/.sbuildrc will help you to detect modifications:

$external_commands = {
    "starting-build-commands" => [
        'bash -c \'find %SBUILD_PKGBUILD_DIR -print0 |
                  sort -z |
                  while read -d $\'\\\'\'\0\'\\\'\' file; do
                      echo -n "$(stat -c "%n %F %%s" "${file}") "
                      if [ -f "${file}" ]; then
                          sha256sum "${file}" |
                              cut -d " " -f 1
                      else
                              echo
                      fi
                  done > /tmp/file-list.pre-build\''
    ],
    "chroot-cleanup-commands" => [
        'cd %SBUILD_PKGBUILD_DIR && ./debian/rules clean',
        'bash -c \'find %SBUILD_PKGBUILD_DIR -print0 |
                  sort -z |
                  while read -d $\'\\\'\'\0\'\\\'\' file; do
                      echo -n "$(stat -c "%n %F %%s" "${file}") "
                      if [ -f "${file}" ]; then
                          sha256sum "${file}" |
                              cut -d " " -f 1
                      else
                              echo
                      fi
                  done > /tmp/file-list.post-build\'',
        'diff /tmp/file-list.pre-build /tmp/file-list.post-build'
    ]
};

Associated schroot environment for development

For development, creating the latest minimal unstable chroot environment to execute programs in unstable environment as sbuild does is time consuming. Let’s create an associated package pre-loaded unstable chroot shell environment and use it directly from the schroot command. (This assumes you have already created chroot environment as above.)

   1 $ sudo cp -a /srv/chroot/unstable-amd64-sbuild-$suffix /srv/chroot/unstable-amd64-dev
   2 $ sudo cp -a /etc/schroot/desktop /etc/schroot/dev
   3 $ sudo tee /etc/schroot/chroot.d/unstable-amd64-dev << EOF
   4 [unstable-dev]
   5 description=Debian unstable/amd64 chroot development environment
   6 groups=root,sbuild
   7 root-groups=root,sbuild
   8 source-groups=root,sbuild
   9 source-root-groups=root,sbuild
  10 profile=dev
  11 type=directory
  12 preserve-environment=true
  13 directory=/srv/chroot/unstable-amd64-dev
  14 union-type=overlay
  15 command-prefix=eatmydata
  16 EOF

Let’s install basic build environment packages to this chroot in /srv/chroot/unstable-amd64-dev.

   1 $ cd /
   2 $ sudo schroot -c source:unstable-dev -u root
   3 (unstable-dev)root@hostname:/# apt update; apt full-upgrade
   4 (unstable-dev)root@hostname:/# apt install aptitude vim nano- mc devscripts sudo gitk
   5 (unstable-dev)root@hostname:/# exit

Let's further customize mount points by editing /etc/schroot/osamu/fstab as needed. Let's also copy setting from the host PC to the chroot as needed.

Let's add 2 shell alias definitions to ~/.bashrc on the host system as follows and enable convenient access to this chroot environment.

alias devs="schroot -c source:unstable-dev"
alias devx="xhost +si:localuser:$(id -un) ; schroot -c chroot:unstable-dev ; xhost -"
  • The devs command opens a shell prompt and allows us to modify the source chroot environment. The changes made are persistent. This is good for installing and upgrading packages in the chroot.

  • The devx command opens a shell prompt and allows us to work in a chroot session to build a source tree with access to X applications such as gitk as usual. The changes made aren’t persistent.

Missing space in /build and /srv/chroot

It is possible that you lack space for builds even if /srv/chroot has plenty of space. This can happen at build time on large packages (e.g. Libreoffice) because sbuild decompresses the packages in /build in the chroot, which is a bind mount to /var/lib/sbuild/build. If you are on a workstation where /var is shared with the root partition, you may sometimes not have enough space for the largest packages. The symptom will be an error message like this:

E: Disc space is probably not sufficient for building.
I: Source needs 2703644 KiB, while 4089748 KiB is free.)

But then when you actually look at the df, you have plenty of space! This happens because sbuild cleans up after itself on failure and gives back the disk space. If you follow disk space usage more closely during the build, you will notice that /var will take up more and more space until that failure.

A workaround for this is to change the configuration of the /build bind mount. This is done in /etc/schroot/sbuild/fstab. For example, the following configuration uses /home/build instead:

# Mount a large scratch space for the build, so we don't use up
# space of the chroot itself.
#/var/lib/sbuild/build  /build   none    rw,bind         0       0
/home/build  /build   none    rw,bind         0       0

If /srv/chroot lacks space but some other directory has plenty of space, you can make space by adding the similar bind mount from the other directory to /srv/chroot.

Using /var/cache/apt/archives/ as package cache

   1 sudo apt -o Dir::State::status=lock build-dep -d <pkg>
   2 sudo apt -o Dir::State::status=lock install -d lintian
   3 cd $(mktemp -d)
   4 ln -s /var/cache/apt/archives/
   5 apt-ftparchive packages . > Packages
   6 apt-ftparchive release . > Release
   7 python3 -m http.server 5678 --bind 127.0.0.1
   8 sbuild --extra-repository="deb [trusted=yes] http://127.0.0.1:5678 ./" --chroot-setup-commands "rm /etc/apt/sources.list"


CategoryPackaging