Differences between revisions 27 and 28
Revision 27 as of 2020-07-04 09:30:58
Size: 6018
Editor: ?Beuc
Comment: Clarify/de-dup pbuilder doc
Revision 28 as of 2020-07-05 02:59:14
Size: 6019
Editor: PaulWise
Comment: rework grammar
Deletions are marked like this. Additions are marked like this.
Line 77: Line 77:
If you build binaries for testing with sbuild and pbuilder, as a convenience feature, they can be made to emit a {{{_source.changes}}} using the {{{--source-only-changes}}} option. A typical build command would be: If you build binaries for testing with sbuild and pbuilder, as a convenience feature, they can be made to emit a {{{_source.changes}}} using the {{{--source-only-changes}}} option.

T
ypical build commands are:
Line 84: Line 87:

Source-only uploads

Summary

Source-only are uploads to the Debian archive that do not include a binary build of the Debian package. Then the buildd network handles the build and distributes it to the archive.

Since circa August 2014, the Debian archive accepts source-only uploads and since August 2015, Architecture: all packages are built on the buildds.

Since July 2019, binaries uploaded by maintainers are not allowed to migrate to testing. In other words, packages must have source-only uploads before they can reach the next release.

Restrictions

NEW uploads and uploads with NEW binaries currently cannot be source-only.

Uploads to jessie-security are supported except for Architecture: all packages.

Rationale

Historically, the uploads of packages required them to be built on the developer's machine. The binary, arch-specific packages would be taken as-it-is by the archive after the upload (along with all binary packages, which are architecture agnostic). This means in particular that when you install an amd64 package it is very likely the exact version that was compiled by the maintainer.

There are a few problems with this approach:

  • it happens sometimes that a developer uploads a miscompiled package by a mistake (e.g., for another suite); this needs then to be fixed with one-time binNMU

  • often the developer compile in an out-of-date environment (e.g., old chroots), and the package might end up linking against old libraries; this is particularly annoying during transitions as it requires investigating why it happens and issuing binNMUs
  • if the machine of a developer is compromised, the package may contain maliciously compiled code

Source-only uploads address these issues by essentially having a central authority to compile all code. However, the disadvantage is that if the archive is compromised then every package is compromised. This is true, but it is true even now for more than 90% packages that are built by the archive.

A complementary project is ReproducibleBuilds which aims to make builds fully, bit-per-bit, reproducible. In the future one can imagine that packages are built both by the developer and the archive, the results are tested for equality and the package is only accepted if they match. All this work is about distributing trust and detecting problems early.

How to make a source-only upload

The dpkg-buildpackage program accepts the --changes-option=-S flag which builds the packages as always, but the final .changes file will contain only the source code. You can then use dput to upload the .changes file (see this thread). Example:

$ cd nghttp2
$ dpkg-buildpackage --changes-option=-S
$ cd ..
$ ls *nghttp2*.*
libnghttp2-doc_1.3.4-1_all.deb  nghttp2_1.3.4-1_amd64.changes  nghttp2_1.3.4-1.dsc
nghttp2_1.3.4-1_amd64.build     nghttp2_1.3.4-1.debian.tar.xz  nghttp2_1.3.4.orig.tar.bz2
$ egrep '^ \S{32} ' nghttp2_1.3.4-1_amd64.changes 
 01c9325805a6fe7fc444c890cf43e0fa 2008 httpd optional nghttp2_1.3.4-1.dsc
 cce2f954f27981191e539f43066e939a 1504585 httpd optional nghttp2_1.3.4.orig.tar.bz2
 e0be575279e76a872eac15708374499e 10060 httpd optional nghttp2_1.3.4-1.debian.tar.xz

You can also use the -S flag which only creates the source-only upload (the package is not even built in this case). Please make sure that the package builds properly before.

If you like debuild command, you can do:

$ debuild -S

Generating _source.changes with sbuild and pbuilder

While you should not build binary packages to upload directly with dpkg-buildpackage, it is fine to use dpkg-buildpackage -S to prepare source-only uploads.

Using sbuild or pbuilder

If you build binaries for testing with sbuild and pbuilder, as a convenience feature, they can be made to emit a _source.changes using the --source-only-changes option.

Typical build commands are:

sbuild -A -s -d unstable --source-only-changes --run-lintian
pbuilder --build --source-only-changes package.dsc

Alternatively, you can set is globally in the configuration and avoid needing to give the option on the command-line:

git-buildpackage support

If you are using git-buildpackage without pbuilder or sbuild, it will happily accept -S or --changes-option=-S switch:

gbp buildpackage --changes-option=-S

This also works with --git-pbuilder too in jessie and below.

If you are using --git-pbuilder in stretch or later, you should use --git-pbuilder-options=--source-only-changes:

gbp buildpackage --git-pbuilder --git-pbuilder-options=--source-only-changes

This will give a binary+source .changes file and a source-only .changes file. Naturally, you must discard the .changes file that includes information about binary packages (.deb files) and only sign and upload the source-only .changes file in order to make a source-only upload.

When using git-buildpackage with sbuild (≥ 0.70.0-1), specify --source-only-changes in order to get both a binary .changes file and a source-only .changes file, e.g.:

gbp buildpackage --git-builder='sbuild --source-only-changes -v -As --dist=unstable'

You can also set it globally in the builder configuration (see above).

Sources