Differences between revisions 18 and 19
Revision 18 as of 2015-12-04 00:41:36
Size: 12771
Comment:
Revision 19 as of 2016-01-05 11:45:58
Size: 12771
Editor: Lunar
Comment: let's say we've reached 1.0
Deletions are marked like this. Additions are marked like this.
Line 34: Line 34:
Format: 1.9 Format: 1.0

Information on the system used during a build will be recorded in a new control file, a “buildinfo” file with suffix .buildinfo.

Discussion with ftpmasters is happening in 763822.

uses

The .buildinfo file has several goals which are related to each other:

  • It records information about the system environment used during a particular build -- packages installed (toolchain, etc), system architecture, etc. This can be useful for forensics/debugging.
  • It can also be used to try to recreate (partially or in full) the system environment when trying to reproduce a particular build.

We want a debian user (or derivative) to be able to find and fetch all .buildinfo files that were known to produce a given binary package so that they can try to reproduce the package themselves.

In the future, there may be more uses for .buildinfo files (or for collections of .buildinfo files related to a single binary artifact). For example:

  • A debian user could elect to only install binary packages that have been successfully built by multiple builders
  • A debian derivative could attest to the packages it has deterministically rebuilt
  • A triage process could identify toolchain changes that have an effect on large numbers of binary packages.
  • A cross-building process could demonstrate the correctness of the cross-compiling toolchain by reproducing the exact binary artifacts.

buildinfo example

a .buildinfo file is a UTF-8-encoded textfile, usually clearsigned with OpenPGP.

The following file would be named fweb_1.62-12+b2_brahms-20120530114812.buildinfo:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.0
Build-Architecture: i386
Source: fweb (1.62-12)
Binary: fweb fweb-doc
Architecture: all i386
Version: 1.62-12+b2
Changes:
 fweb (1.62-12+b2) sid; urgency=low, binary-only=yes
 .
   * Binary-only non-maintainer upload for amd64; no source changes.
   * Rebuild for multiarch sync
 .
  -- amd64 / i386 Build Daemon (brahms) <buildd_amd64-brahms@buildd.debian.org>  Wed, 30 May 2012 09:48:12 +0200 18:37:51 +0000
Checksums-Sha256:
 9921500c4c6159c0019d4b8b600d2d06eef6b1da056abd2f78e66a9f0c3843b9 879 fweb_1.62-12.dsc
 3a7492c2013fbeebff08bee0514481ec0f56d2c4d138188d1ef85156d08ded00 436982 fweb-doc_1.62-12_all.deb
 a916dbb1c63707eaf52a5cdd10769871d2f621848176dc8f7ab4f0dcd999af85 229990 fweb_1.62-12+b2_i386.deb
Build-Path: /usr/src/debian/fweb-1.62-12+b2
Build-Environment:
 acl (= 2.2.52-1),
 adduser (= 3.113+nmu3),
 base-files (= 7.5),
 base-passwd (= 3.5.33),
 bash (= 4.3-9),
 binutils (= 2.24.51.20140818-1),
 bsdmainutils (= 9.0.5),
 bsdutils (= 1:2.20.1-5.8),
 build-essential (= 11.7),
 bzip2 (= 1.0.6-7),
 coreutils (= 8.21-1.2),
 cpp (= 4:4.9.1-3),
 cpp-4.9 (= 4.9.1-9),
 dash (= 0.5.7-4),
 debconf (= 1.5.53),
 debhelper (= 9.20140817),
 debianutils (= 4.4),
 dh-buildinfo (= 0.11),
 diffutils (= 1:3.3-1),
 dmsetup (= 2:1.02.88-1),
 dpkg (= 1.17.13),
 dpkg-dev (= 1.17.13),
 e2fslibs (= 1.42.11-2),
 e2fsprogs (= 1.42.11-2),
 file (= 1:5.19-1),
 findutils (= 4.4.2-9),
 g++ (= 4:4.9.1-3),
 g++-4.9 (= 4.9.1-9),
 gcc (= 4:4.9.1-3),
 gcc-4.9 (= 4.9.1-9),
 gcc-4.9-base (= 4.9.1-9),
 gettext (= 0.19.2-1),
 gettext-base (= 0.19.2-1),
 grep (= 2.20-2),
 groff-base (= 1.22.2-6),
 gzip (= 1.6-3),
 hostname (= 3.15),
 init (= 1.21),
 initscripts (= 2.88dsf-53.4),
 insserv (= 1.14.0-5),
 intltool-debian (= 0.35.0+20060710.1),
 libacl1 (= 2.2.52-1),
 libasan1 (= 4.9.1-9),
 libasprintf0c2 (= 0.19.2-1),
 libatomic1 (= 4.9.1-9),
 libattr1 (= 1:2.4.47-1),
 libaudit1 (= 1:2.3.7-1),
 libaudit-common (= 1:2.3.7-1),
 libblkid1 (= 2.20.1-5.8),
 libbz2-1.0 (= 1.0.6-7),
 libc6 (= 2.19-10),
 libc6-dev (= 2.19-10),
 libcap2 (= 1:2.24-4),
 libcap2-bin (= 1:2.24-4),
 libc-bin (= 2.19-10),
 libc-dev-bin (= 2.19-10),
 libcilkrts5 (= 4.9.1-9),
 libcloog-isl4 (= 0.18.2-1),
 libcomerr2 (= 1.42.11-2),
 libcroco3 (= 0.6.8-3),
 libcryptsetup4 (= 2:1.6.6-1),
 libdb5.3 (= 5.3.28-6),
 libdbus-1-3 (= 1.8.6-2),
 libdebconfclient0 (= 0.191),
 libdevmapper1.02.1 (= 2:1.02.88-1),
 libdpkg-perl (= 1.17.13),
 libffi6 (= 3.1-2),
 libgcc1 (= 1:4.9.1-9),
 libgcc-4.9-dev (= 4.9.1-9),
 libgcrypt11 (= 1.5.4-2),
 libgcrypt20 (= 1.6.2-2),
 libgdbm3 (= 1.8.3-13),
 libglib2.0-0 (= 2.40.0-4),
 libgmp10 (= 2:6.0.0+dfsg-6),
 libgomp1 (= 4.9.1-9),
 libgpg-error0 (= 1.13-3),
 libintl-perl (= 1.23-1),
 libisl10 (= 0.12.2-2),
 libitm1 (= 4.9.1-9),
 libkmod2 (= 18-1),
 liblzma5 (= 5.1.1alpha+20120614-2),
 libmagic1 (= 1:5.19-1),
 libmount1 (= 2.20.1-5.8),
 libmpc3 (= 1.0.2-1),
 libmpfr4 (= 3.1.2-1),
 libncurses5 (= 5.9+20140712-2),
 libncurses5-dev (= 5.9+20140712-2),
 libncursesw5 (= 5.9+20140712-2),
 libpam0g (= 1.1.8-3.1),
 libpam-modules (= 1.1.8-3.1),
 libpam-modules-bin (= 1.1.8-3.1),
 libpam-runtime (= 1.1.8-3.1),
 libpcre3 (= 1:8.35-3),
 libpipeline1 (= 1.3.0-1),
 libprocps3 (= 1:3.3.9-7),
 libquadmath0 (= 4.9.1-9),
 libselinux1 (= 2.3-1),
 libsemanage1 (= 2.3-1),
 libsemanage-common (= 2.3-1),
 libsepol1 (= 2.3-1),
 libss2 (= 1.42.11-2),
 libstdc++-4.9-dev (= 4.9.1-9),
 libstdc++6 (= 4.9.1-9),
 libsystemd-journal0 (= 208-8),
 libsystemd-login0 (= 208-8),
 libtext-unidecode-perl (= 0.04-2),
 libtimedate-perl (= 2.3000-2),
 libtinfo5 (= 5.9+20140712-2),
 libtinfo-dev (= 5.9+20140712-2),
 libubsan0 (= 4.9.1-9),
 libudev1 (= 208-8),
 libunistring0 (= 0.9.3-5.2),
 libustr-1.0-1 (= 1.0.4-3),
 libuuid1 (= 2.20.1-5.8),
 libwrap0 (= 7.6.q-25),
 libxml2 (= 2.9.1+dfsg1-4),
 libxml-libxml-perl (= 2.0116+dfsg-1+b1),
 libxml-namespacesupport-perl (= 1.11-1),
 libxml-sax-base-perl (= 1.07-1),
 libxml-sax-perl (= 0.99+dfsg-2),
 linux-libc-dev (= 3.14.15-2),
 login (= 1:4.2-2+b1),
 lsb-base (= 4.1+Debian13),
 make (= 4.0-8),
 man-db (= 2.6.7.1-1),
 mawk (= 1.3.3-17),
 mount (= 2.20.1-5.8),
 ncurses-base (= 5.9+20140712-2),
 ncurses-bin (= 5.9+20140712-2),
 passwd (= 1:4.2-2+b1),
 patch (= 2.7.1-6),
 perl (= 5.20.0-4),
 perl-base (= 5.20.0-4),
 perl-modules (= 5.20.0-4),
 po-debconf (= 1.0.16+nmu3),
 procps (= 1:3.3.9-7),
 sed (= 4.2.2-4),
 sensible-utils (= 0.0.9),
 startpar (= 0.59-3),
 systemd (= 208-8),
 systemd-sysv (= 208-8),
 sysvinit-utils (= 2.88dsf-53.4),
 sysv-rc (= 2.88dsf-53.4),
 tar (= 1.27.1-2),
 texinfo (= 5.2.0.dfsg.1-4),
 tzdata (= 2014f-1),
 ucf (= 3.0030),
 udev (= 208-8),
 util-linux (= 2.20.1-5.8),
 xz-utils (= 5.1.1alpha+20120614-2),
 zlib1g (= 1:1.2.8.dfsg-2)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQJ8BAEBCgBmBQJWYNoZXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w
Dx1xK9OgUkDt+gwh9WK/QrvV7IOjAg/pl6j7px5u6MNKHWPW0tC9M5123Q2KmaGT
-----END PGP SIGNATURE-----

buildinfo specification

File name

buildinfo files should be named ${SOURCE}_${DEBIAN_VERSION}_${STRING}.buildinfo.

  • SOURCE is the source package name.

  • DEBIAN_VERSION is the full version of the Debian package without the epoch. This is the same as the Version: field in the corresponding .changes file.

  • STRING is an arbitrary string, unique within all ${SOURCE},${DEBIAN_VERSION} builds. The string should consist only of alphanumeric characters and hyphens. It's likely that debian buildd's would create this string as ${HOSTNAME}-${TIMESTAMP}, where TIMESTAMP is an ISO-8601-formatted numeric string. Debian developers may choose arbitrary strings here. The only requirement from the archive's perspective is that the STRING be unique for source package version, so that multiple .buildinfo files may exist for a given version of a package.

buildinfo field descriptions

Format

Same as in `*.changes`.

Build-Architecture

The Debian machine architecture that was used to perform the build. 1

Source

Same as in `*.changes`. If the source and binary versions differ (e.g. binNMUs), the source version is added between parenthesis.

Binary

Same as in `*.changes`.

Architecture

Same as in `*.changes` except source should not be specified: only concrete architectures, no wildcards or any.

Version

Same as in `*.changes`.

Changes

Close to the one in `*.changes`. When source and binary versions differ, the field is added with the content of the extra changelog entries.

Checksums-Sha256

Same format as other control files. Must list the *.dsc file and all files listed in `debian/files`.

Build-Path

Absolute path of the directory in which the package has been built. See ReproducibleBuilds/History#Giving_up_on_build_paths for a rationale.

Build-Environment

List of all packages forming the build environment, their architecture if different from build architecture, and their version. This includes Essential packages, build-essential, and Build-Depends and Build-Depends-Indep. For each packages, their dependencies should be recursively listed. The format is the same as Built-Using.

buildinfo signatures

.buildinfo files describe the state of a particular build. The operator having made the build may sign the file by wrapping the entire file in a cleartext OpenPGP signature.

Inclusion of *.buildinfo in the archive

We want an interested user (an individual, organization, or derivative) to be able to find the relevant .buildinfo files from the archive they already have configured. There are several ways this could be done, but we imagine that a relatively easy way is to ship all the .buildinfo files for a given architecture for a suite in a Buildinfos.tgz archive file placed next to Packages.gz. See 763822 for more details.

Previous ideas

.changes files looked like a good place to record the environment as they list the checksums of the build products and are signed by either the maintainer or the buildd operator.

But the meaning of *.changes files is pretty clear: they describe a transactional change operation on the archive. They are not saved directly in the archive: they are equivalent of a log entry. The name of *.changes file is also not specified and multiple operations can have the same name.

(See also 719854 for the first attempt which tried using XC- field in debian/control.)

We thought that .buildinfo files would be all the data required to do a rebuild, as opposed to a description of the state of the build system. However, it's likely that we don't know the actual requirements, and it's likely that the description will be more detailed than is necessary in some cases. This means that two different buildinfo files could attest to the same exact binary artifact.

We thought at some point that the .buildinfo could be referenced in the Packages index, but this does not seem necessary, and might be overkill, since only some users may actually have varied systems.

We do not need to specify the ordering of fields?

things we are not currently including

  • order in which the system packages (essential, build-essential, build-deps) were installed.
  • the cryptographic digest of the system packages themselves
  • the digest of the source code of the system packages
  1. This is, the build architecture in GNU terminology. See dpkg-architecture(1) for a definition. The target architecture for cross compilers is usually encoded in the package name. The host architecture is the binary package architecture (in the Architecture field and file name). Thus, target and host architecture do not need to be encoded here. (1)