Summary

Getting the .buildinfo files into the Debian archive has not yet happened, and it is one of the last infrastructure blockers for having full reproducible support in Debian.

Holger Levsen proposed in https://debconf20.debconf.org/talks/49-reproducing-bullseye-in-practice/ the possibility of bundling the .buildinfo files into the binary .deb archives, as a way to sidestep the deployment into the archive.

From the dpkg PoV while this seems like it could probably get implemented somehow, it also looks like the result might end up being impractical/cumbersome, more complex than needed, and backtracks on some of the design principles originally agreed on as good properties for the .buildinfo support (https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles). But it's still probably interesting to go through the pros and cons anyway, even if just for historical purposes.

Evaluation

While the proposal has definitely not really been fleshed out, we can still evaluate some possibilities.

Bundling the .buildinfo files inside the .deb archives would probably imply that:

• the .deb archive are generated by a call to dpkg-deb from within debian/rules, bundling .buildinfo inside would require appending those from something called by dpkg-buildpackage.
• the .buildinfo file contains references to .deb archives (size and checksums), which would become wrong once the .buildinfo files have been bundled into the .deb archives. This would make a simple file comparison (or checksum comparison) not feasible anymore, and would require some kind of tool to unbundle the .buildinfo from the .deb archive (which goes against one of the stated original design principles).
  • proposal: if the checksum and size and build date were removed, it might be possible to make the embedded .buildinfo reproducible.
  • counter: while a possibility this seems to make the .buildinfo less useful for verification purposes, also the signature makes this unreproducible, and removing it would also seem to make the file even less useful for some of the intended purposes of the reproducible initiative? But if all those verifiable expectations from the .buildinfo can be dropped as an acceptable trade-off, then this could be a workable alternative.
  • counter: A relevant use case for reproducible builds is answering the question "If I change X, does it affect the output artifact?" This use case would be essentially broken by embedded .buildinfo files.
  • counter: Embedding .buildinfo files fully breaks cross/native reproducibility, because the installed build-depends do differ for cross/native in practice. To make the .buildinfo reproducible, the installed build-depends would have to be removed.
  • counter: If Debian were at some point requiring diverse double compilation for archive packages, which of the two .debs would be included in the archive?
  • counter-counter: signatures would also need to be removed from the embedded .buildinfo. The whole .deb includes the information necessary to reproduce it so signatures could be done on the whole .deb.
• the .changes and the .buildinfo file would end up with inconsistent .deb archive references (one before the bundling the other after).
  • proposal: unless produced artifacts are removed from embedded .buildinfo.
• all .deb archives from source packages would end up with the same .buildinfo (per architecture) which seems wasteful, although the .buildinfo files are not too big, and could end up compressed within an ar member.
• a .changes for a source-only upload would be unable to provide a .buildinfo file, which would be a regression.
  • proposal: could still ship an external .buildinfo as done now in addition to embedded .buildinfo.
  • counter: while true, that would not seem to help given that one of the problems this change proposal is trying to workaround is the infrastructure not exposing those unbundled .buildinfo files.
• .udebs would not contain these given that they are size constrained. This would make udeb-only source packages not trackable as reproducible. Source packages generating .deb and .udeb would still make it possible to fetch the .buildinfo by downloading a .deb (if any was built for that arch) matching the arch_version of the .udeb.
• the current deployment problem gets partially fixed, and we also get mirroring for free.
• the .buildinfo information is always readily available for a .deb archive.

Some alternatives:

• there has been some talk about packing some build artifacts so that they are easier to disseminate after a build, such as build logs, or similar. Perhaps an alternative solution would be to automatically generate a new .deb that contains build artifacts such as (at least) .buildinfo and build logs). This though might still require infra changes for the archive, say to have something like the debug symbol packages archive. This would still seem somehow problematic for the source-only upload, as including a binary .deb with build artifacts might seem odd?
• add a per-arch Builds/BuildInfo file with one stanza per .buildinfo and drop stanzas from the file when dropping the corresponding .deb files from Packages
• add to each stanza in Sources or Packages a field containing references to the corresponding buildinfo files stored in the pool/ directory

CategorySpec