Reproducible Live images

This page tracks the progress on creating reproducible live images, using live-build from DebianLive

Contents

  1. Reproducible Live images
  2. Status
    1. Debian bookworm
    2. Debian sid (unstable)
    3. Debian Bullseye
  3. Configuration
    1. Modified configuration files
    2. Jenkins node osuosl173-amd64.debian.net
    3. Personal computer
    4. Preparing the build environment
    5. Preparing the build environment (old)
  4. Checking reproducibility
    1. Using diffoscope
    2. Using reprotest
  5. See also

Status

Debian bookworm

Closed:

Open:

Debian sid (unstable)

Debian Bullseye

Image

Installer

Reproducible

Notes

mini

none

yes

mini

live/text

yes

1

mini

live/gui

yes

1

mini

daily

yes

2

standard

none

yes

3, 4

standard

live

yes

1, 3, 4

GNOME

none

yes

3, 4, 5

GNOME

live

yes

1, 3, 4, 5

KDE

live

yes

1, 3, 4, 5

Cinnamon

live

yes

3, 4, 5, 6

Configuration

Modified configuration files

Required entries in sudoers (assuming you are thisuser): 

Defaults env_keep += "SOURCE_DATE_EPOCH"
Defaults env_keep += "LIVE_BUILD"
thisuser ALL=(root) NOPASSWD: /usr/bin/lb build
thisuser ALL=(root) NOPASSWD: /usr/bin/lb clean --purge

Jenkins node osuosl173-amd64.debian.net

Personal computer

The computer that is used for generating live images:

Snapshot

In order to get a reproducible image, there are two main options:

  1. If the live image is expected to be reproducible, the image can be built twice in rapid succession from deb.debian.org (within its regular 6 hour update cycle)

  2. If the live image is expected to be non-reproducible, a snapshot server needs to be used.
    • By using a snapshot server, it is easier to re-generate images to track the difference and to polish on the work-around hook.

      Note that for live images in production, deb.debian.org needs to be used.

The server snapshot.notset.fr is used since 2021-09-05, because the service at snapshot.debian.org occasionally rejects traffic when many requests are made.

Historical note: debian.notset.fr was used until 2021-09-05, snapshot.debian.org was used until 2021-08-25

Considerations

Preparing the build environment

Preparation of the build directory

As root: 

# You need `dev` and `exec` active on your mount point
# Slightly more than 16GB is required (on a machine with 32GB memory /dev/shm get half of it)
mount /dev/shm -odev,exec,remount,size=24G

As a regular user: 

mkdir /dev/shm/live
cd /dev/shm/live

Building

All command lines can be executed as a regular user.

Use case: rebuild an image with a given timestamp (as e.g. taken from a Jenkins job) from the snapshot server 

/home/roland/git.nobackup/live-build/test/rebuild.sh gnome sid 20220421T153504Z

Use case: get the latest image from the snapshot server 

# Replace the timestamp with the word 'snapshot' to fetch the latest snapshot
/home/roland/git.nobackup/live-build/test/rebuild.sh gnome sid snapshot

Use case: while preparing new hooks, using the local git checkout of live-build 

# When LIVE_BUILD is defined, the script will not checkout live-build from the timestamp, but will use the version pointed at by LIVE_BUILD
export LIVE_BUILD=/home/roland/git.nobackup/live-build
/home/roland/git.nobackup/live-build/test/rebuild.sh gnome sid 20220421T153504Z

Use case: build an image from the current Debian repository 

/home/roland/git.nobackup/live-build/test/rebuild.sh gnome sid

Preparing the build environment (old)

These steps below have migrated to a dedicated script in live-build. It is recommended to use that instead. This section is kept for documentation purposes.

All command lines must be executed as root. 

##
## Section that does not need to be modified, if you want to use /dev/shm
##
cd /dev/shm
mount /dev/shm -odev,exec,remount
mkdir live
cd live

##
## Section that you would like to adjust for your own setup
##
export LIVE_BUILD=/home/roland/git.nobackup/live-build
export http_proxy=http://127.0.0.1:3142/
export USE_DISTRIBUTION=sid

##
## Timestamp selection
##
# Bullseye: 20210101T083123Z (1609489883)
#export SOURCE_DATE_EPOCH=1609489883
# Buster: 20210210T031935Z (1612927175)
#export SOURCE_DATE_EPOCH=1612927175
# Bookworm and sid: Use the timestamp of the latest mirror snapshot (if the timestamp was not set already)
if [ -z "${SNAPSHOT_TIMESTAMP}" ]; then
  wget http://snapshot.notset.fr/mr/timestamp/debian/latest
  #
  # Extract the timestamp from the JSON file
  #
  # Input:
  # {
  #   "_api": "0.3",
  #   "_comment": "notset",
  #   "result": "20210828T083909Z"
  # }
  # Output:
  # 20210828T083909Z
  #
  export SNAPSHOT_TIMESTAMP=$(cat latest | awk '/"result":/ { split($0, a, "\""); print a[4] }')
fi
# Convert SNAPSHOT_TIMESTAMP to Unix time (and insert suitable formatting first)
export SOURCE_DATE_EPOCH=$(date -d $(echo $SNAPSHOT_TIMESTAMP | awk '{ printf "%s-%s-%sT%s:%s:%sZ", substr($0,1,4), substr($0,5,2), substr($0,7,2), substr($0,10,2), substr($0,12,2), substr($0,14,2) }') +%s)

##
## Prepare the basic configuration
##
export MIRROR=http://snapshot.notset.fr/archive/debian/${SNAPSHOT_TIMESTAMP}
# The basic configuration line:
lb config --apt-http-proxy ${http_proxy} --parent-mirror-bootstrap ${MIRROR} --parent-mirror-binary ${MIRROR} --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution ${USE_DISTRIBUTION} --debian-installer live --cache-packages false

##
## Install the hooks for reproducible builds
##
if [ -z "${LIVE_BUILD}" ]; then
  cp /usr/share/doc/live-build/examples/hooks/reproducible/* config/hooks/normal
else
  cp $LIVE_BUILD/examples/hooks/reproducible/* config/hooks/normal
fi

Package selection

Standard 

echo "live-task-standard" > config/package-lists/desktop.list.chroot

GNOME desktop 

echo "live-task-gnome" > config/package-lists/desktop.list.chroot

KDE desktop 

echo "live-task-kde" > config/package-lists/desktop.list.chroot

Some variants

Note: these command lines use bullseye, they were written when bullseye was still unstable.

No installer 

lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/snapshot.debian.org/archive/debian/20210101T083123Z --parent-mirror-binary http://snapshot.debian.org/archive/debian/20210101T083123Z --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution bullseye --debian-installer none --cache-packages false

Live/text-only installer 

lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/snapshot.debian.org/archive/debian/20210101T083123Z --parent-mirror-binary http://snapshot.debian.org/archive/debian/20210101T083123Z --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution bullseye --debian-installer live --cache-packages false --debian-installer-gui false

Latest installer 

unset SOURCE_DATE_EPOCH
lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/deb.debian.org/debian --parent-mirror-binary http://snapshot.debian.org/archive/debian/20210101T083123Z --security false --updates false --distribution bullseye --debian-installer live --cache-packages false --debian-installer-gui false --parent-debian-installer-distribution daily
# When doing comparisons, use the following line for the first run:
#   export SOURCE_DATE_EPOCH=`date +%s`; lb build

Live installer (to be used for production images) 

lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/snapshot.debian.org/archive/debian/20210101T083123Z --parent-mirror-binary http://snapshot.debian.org/archive/debian/20210101T083123Z --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution bullseye --debian-installer live --cache-packages false

Building images for Debian Buster

Mini, with live installer 

export SOURCE_DATE_EPOCH=1612927175
lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/snapshot.debian.org/archive/debian/20210210T031935Z --parent-mirror-binary http://snapshot.debian.org/archive/debian/202102101031935Z --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution buster --debian-installer live --cache-packages false

Checking reproducibility

Ideally reprotest will be used to check reproducibility. For now, 2 images are built and then compared with diffoscope.

Using diffoscope

# 1) Prepare the build environment
# 2) Run the command line to create the configuration (see above)
lb build
mv live-image-amd64.hybrid.iso run01.iso
lb clean --purge
lb config
lb build
mv live-image-amd64.hybrid.iso run02.iso
diffoscope run01.iso run02.iso --html-dir html_run01_02

Using reprotest

reprotest requires full access on /tmp, so it must be remounted (when applicable) 

# 1) Prepare the build environment
# 2) Run the command line to create the configuration (see above)
mount /tmp -odev,exec,suid,remount
reprotest  --variations=-all,+environment,+build_path,-kernel,+aslr,+num_cpus,+time,-user_group,-fileordering,-domain_host,+home,+locales,+exec_path,+timezone,-umask  "lb clean --purge&&lb config&&lb build" "*.iso"
# Disabled tests:
#  kernel        The variant with kernel 2.6 is too old for debootstrap
#  user_group    Live build only works for root, the variation needs a list
#  fileordering  Disorderfs needs to be mounted with dev,exec,suid
#  domain_host   Needs additional rights during debootstrap
#  umask         Some files get a different umask, further investigation is required

See also

Page maintainer: Roland Clobus