Differences between revisions 19 and 20
Revision 19 as of 2021-08-21 22:08:48
Size: 18814
Editor: Brian Potkin
Comment: Corrected link.
Revision 20 as of 2021-08-21 23:07:18
Size: 18892
Editor: Brian Potkin
Comment: Added a command. Checked links and tested on bullseye.
Deletions are marked like this. Additions are marked like this.
Line 303: Line 303:
bsdtar -C <mount_point>/debian -xf debian-11.0.0-i386-DVD-1.iso pool/contrib

Translation(s): English - español


Install a preseeded Debian from a USB stick that holds a package archive.

Introduction

The techniques described on this page have the objectives of

  • Producing an automated installation of a Debian operating system from a USB stick.
  • Having the USB stick also hold a Debian archive to allow package installation from it after the OS has been installed.

Automation is done via preseeding and the focus here will be on the i386 Debian 11 (bullseye) architecture to illustrate the process. The same techniques should be applicable to pre-Debian 11 (buster and earlier) and the amd64 architecture of all distributions.

  • There is a sample preseed.cfg file available to consult. It is tailored to install a minimal text mode OS that has an ethernet connection to the internet.

Requiring Root Privileges

For Debian 8 and later an ordinary user does not have the permissions to write to a raw device. Using the dd, fdisk, cfdisk, mkfs.vfat and grub-install commands will not work for such a user. The only way to have these commands complete successfully is for a user to gain root privileges with su or sudo or log in as root.

On Debian 7 an ordinary user does have the permissions to write to a raw device and the dd, fdisk, cfdisk, mkfs.vfat and grub-install commands will be successful. There is no requirement to use su or sudo for a device to be partitioned, formatted and have GRUB installed to its MBR on Debian 7.

An exception to the need to have root privileges is when it comes to mounting a partition on a USB stick. This can be done with or without becoming root.

Preparing the USB Stick

Without the stick plugged into a USB port execute lsblk (as a user). Now plug the stick in and note the extra entry under NAME when the command is executed again. This is the device name of the USB stick. It could be /dev/sdb, /dev/sdc etc. It will be needed later. /dev/sdb will be assumed here.

It is best to play safe and obliterate any record of previous partitioning and of the stick having possibly held an isohybrid ISO. GRUB will refuse to install in the MBR of a device if it detects that it has once held an isohybrid image.

dd if=/dev/zero of=/dev/sdb count=100

Partition the stick with fdisk or cfdisk to have a single FAT16 primary partition:

/sbin/fdisk /dev/sdb
or
/sbin/cfdisk /dev/sdb

Install the dosfstools package, put a vfat filesystem on the partition you have created and label the partition.

/sbin/mkfs.vfat /dev/sdb1
/sbin/dosfslabel /dev/sdb1 ARCHIVE

FAT and vfat are chosen because modules for them are available from the very beginning of the start the installer.

Installing GRUB on the USB Stick

Being able to mount and unmount the USB stick is a prerequisite to installing GRUB to its MBR. As root:

mount /dev/sdb1 /mnt
umount /mnt

If you are working within one of the desktop environments it is very likely the mounting can be done as a user from the file manager that comes with it. This is because udisks2 will be on the system. A label will possibly be used for the mount point.

Check the mount point from within the file manager or with

findmnt

Without a desktop environment but with pmount, udevil or udisks2 on the system, one of the following commands may be run as a user to mount the primary partition. The mount point for pmount is /media/ARCHIVE. The other two programs will indicate where the mount point is located.

pmount /dev/disk/by-label/ARCHIVE ARCHIVE
udevil mount /dev/disk/by-label/ARCHIVE

udisksctl loop-setup -f /dev/disk/by-label/ARCHIVE
udisksctl mount -b /dev/disk/by-label/ARCHIVE

Unmounting is with:

pumount /dev/disk/by-label/ARCHIVE
udevil unmount /dev/disk/by-label/ARCHIVE
udisksctl unmount -b /dev/disk/by-label/ARCHIVE

GRUB in the Master Boot Record (MBR) of the USB stick (assuming /dev/sdb as the device name) is achieved with:

/usr/sbin/grub-install --boot-directory=<mount_point>/boot /dev/sdb

Booting from the USB Stick

The method chosen here to boot a USB stick requires a kernel, an initrd and a grub.cfg. The first two items are obtainable for the Debian 11 i386 architecture from a Debian mirror and copied to <mount_point>/boot:

cp vmlinuz <mount_point>/boot
cp initrd.gz <mount_point>/boot
  • It is important that the kernel used to boot and the kernel in the installation image match. Ensure the image, vmlinuz and initrd.gz are the most up-to-date possible. A vmlinuz and intitrd.gz from the daily download page could help with a problem encountered after booting the installation image.

The Installation Image

The initrd contains the iso-scan program; this will search all disks for an installation image. Suitable images are the first image of a CD or DVD set (labelled as debian-something-1) or a netinst image. If you anticipate having to rely on non-free firmware for the installation, you could consider following the advice given later on this page or using an unofficial image containing non-free firmware.

As part of an automated process you can avoid all disks being searched by constructing a grub.cfg that designates where the image to be used for the installation is.

  • shared/ask_device=manual informs the installer that the device where the image is will be manually selected.

  • shared/enter_device=/dev/disk/by-label/ARCHIVE (if labelling is used) tells it what the selected device is.

With a connection to the internet, any package updates that are available (security updates, for example) would normally be pulled in during the installation of the image. Archive information would now become outdated and some packages from an archive would be uninstallable. If this is unwanted, use one of these two solutions:

  • Dispense with a network connection during the installation.
  • Preseed in grub.cfg with pkgsel/upgrade=none.

On the other hand, if the intention is to have access to bullseye updates and security fixes on the installed system while still using the archive as the primary source of packages, add

  • deb http://deb.debian.org/debian bullseye main contrib

  • deb http://deb.debian.org/debian-security bullseye-security main contrib

after the package archive entry.

Packages not available in the package archive will come from deb.debian.org.

The Preseed File

A directory to hold the installation image, the preseed.cfg and any other files is convenient:

mkdir <mount_point>/files
cp <installation_image> <mount_point>/files
cp preseed.cfg <mount_point>/files

The location for the installer to look for preseed.cfg is given as file=/hd-media/preseed.cfg on the linux line of grub.cfg.

preseed.cfg will be found and loaded by the installer because the partition holding the initrd is mounted on /hd-media after the questions about language and keyboard layout are asked. This implies language and keyboard layout cannot be preseeded in preseed.cfg. They can be preseeded, however, on the linux line with (for example) locale=en_GB.UTF-8 and keymap=gb.

A Sample grub.cfg File

Putting together what we have up to now, grub.cfg would look like this:

menuentry 'Debian 11.0.0 i386 automated install' {
  linux /boot/vmlinuz shared/ask_device=manual                       \
                      shared/enter_device=/dev/disk/by-label/ARCHIVE \
                      file=/hd-media/files/preseed.cfg               \
                      locale=en_GB.UTF-8 keymap=gb                   \
                      pkgsel/upgrade=none
  initrd /boot/initrd.gz
  }

Copy this file to the USB stick:

cp grub.cfg <mount_point>/boot/grub

Non-free Firmware

Non-free firmware is sometimes required

  • During an installation, particularly for wireless and ethernet adapters.
  • After installation, particularly for video, wireless and ethernet adapters.

Needed firmware files may be extracted from Debian packages. A collection of such packages is provided by firmware.tar.gz. The extracted files are put in a directory on the USB stick that must be named firmware. For example:

mkdir <mount_point>/files/firmware
cp <firmware_files> <mount_point>/files/firmware

The files are provided directly to the installer's kernel by adding

d-i preseed/early_command string cp -a /hd-media/files/firmware /lib

to preseed.cfg. The firmware files will be found and transferred to lib/firmware because the partition holding the files is mounted on /hd-media. They also remain available to the installed system.

The benefits of this technique are that the non-free firmware is given directly to the installer and may be tailored to an installation on a particular machine. The disadvantage, as opposed to using an unofficial installation image, is that a user has to take action on the installed system to alter /etc/apt/sources.list to obtain updated non-free firmware packages.

Summary-1

  1. After blanking the first 64 sectors of the USB stick, partition it, format the partition with vfat and label it.

  2. Install GRUB to the stick's MBR.

  3. Make a directory files on the stick. Copy the installation image and preseed.cfg to files.

  4. Copy initrd.gz and vmlinuz to /boot.

  5. Copy grub.cfg to /boot/grub. Do not forget to specify your label.

  6. Choose an installation image and deal, if necessary, with the provision of non-free firmware.

  7. Unmount the stick and boot from it.

Choose from Multiple Installation Images

It is tempting to have more than one installation image in files and anticipate choosing one of them. This will work with a single partition on the USB stick if you are prepared to forgo some of the preseeded automation of responses to the first few questions presented by the installer.

One method is to add

priority=medium

to the linux line of grub.cfg. The stick's primary partition will be scanned for ISO images and a choice of which one to use given. After the selection the debconf priority can be put back to high to take advantage of the rest of the preseeding.

This bug report suggests a second method when all ISOs are in one directory. The installation will halt and display the contents of morefiles. After the selection the preseeding takes over again without user intervention.

mkdir <mount_point>/files/morefiles

and move the installation images into the morefiles directory. Then add

iso-scan/ask_second_pass=true

to the grub.cfg.

A third method, which preserves all automation, is to make more partitions on the USB stick and put an ISO on each of them. Label the new partitions (LABEL-2, LABEL-3 etc) and construct more grub stanzas. They will only differ from the original one by having

shared/enter_device=/dev/disk/by-label/<LABEL-2>
shared/enter_device=/dev/disk/by-label/<LABEL-3>

  • Remember that the architectures of the kernel and initrd images and the installation image must match.

Using GRUB's Loopback Facility

This feature of GRUB allows the kernel and initrd images within an ISO to be used for booting an ISO. This means the hd-media vmlinuz and initrd.gz would apparently not be required. Unfortunately, the installer images (netinst, CD-1, DVD-1 etc) have initrds that do not contain iso-scan, so an installation image cannot be searched for.

What could be tried, though, is using the hd-media initrd with a loopback stanza in grub.cfg.

set isofile="files/firmware-10.2.0-i386-netinst.iso"
menuentry 'loopback' {
     loopback loop (hd0,1)/$isofile
     linux  (loop)/install.386/vmlinuz shared/ask_device=manual                       \
                                       shared/enter_device=/dev/disk/by-label/ARCHIVE \
                                       file=/hd-media/files/preseed.cfg               \
                                       locale=en_GB.UTF-8 keymap=gb
     initrd /boot/initrd.gz
}

  • There is no guarantee that mixing an hd-media initrd with an ISO's kernel will produce a desirable outcome in all circumstances.

Creating a Package Archive

Mount the USB stick and set up a package directory structure. Staying with the i386 architecture:

mkdir -p <mount_point>/debian/dists/bullseye/main/binary-i386
mkdir -p <mount_point>/debian/dists/bullseye/contrib/binary-i386

Binary packages for the archive are obtained from the CD or DVD sets, (labelled as debian-something-1, debian-something-2 etc). Please consider using jigdo for downloading packages and making the images.

The binaries are in the pool/main and pool/contrib directories of the downloaded images. They may be copied to the stick using one of the methods described here. For example, for DVD-1:

bsdtar -C <mount_point>/debian -xf debian-11.0.0-i386-DVD-1.iso pool/main
bsdtar -C <mount_point>/debian -xf debian-11.0.0-i386-DVD-1.iso pool/contrib

Repeat for other images.

The Packages Index Files

Install dpkg-dev to get dpkg-scanpackages. Then:

cd <mount_point>/debian

and

dpkg-scanpackages pool/main /dev/null > dists/bullseye/main/binary-i386/Packages

Repeat for the contrib packages.

dpkg-scanpackages pool/contrib /dev/null > dists/bullseye/contrib/binary-i386/Packages

An alternative technique is to use apt-ftparchive (which is already on the system) to produce the index files.

Telling the OS About the Package Archive

This is done as root in /etc/apt/sources.list of the installed OS.

deb [ trusted=yes ] file:/media/<mount_point>/debian bullseye main contrib

With a labelled partition <mount_point> could be /media/ARCHIVE or /media/$USER/ARCHIVE, depending on which utility is used to mount the USB stick.

Summary-2

  1. Carry out steps 1 to 7 of Summary-1

  2. Construct a package directory structure.

  3. Extract binary packages from Debian ISOs and copy them to the stick.

  4. Create Packages index files.

  5. Write a sources.list.

Accessing the Package Archive

Plug in the stick and mount it. As root:

apt update
apt install <packages>

Automatically mounting a partition on the device when it is accessed and automatically unmounting it when access to it is not wanted can be done with an entry in /etc/fstab. All the parameters necessary for this to happen are available from systemd 220 onwards and explained in the systemd.mount manual.

  • LABEL=ARCHIVE /media/ARCHIVE vfat defaults, noauto,x-systemd.automount,x-systemd.idle-timeout=5,x-systemd.device-timeout=1 0 0

  • x-systemd.automount mounts /media/ARCHIVE when any command (such as either of the two above) wants to use it.

  • systemd.idle-timeout leads to unmounting the partition a specified time period after the calling program ceases to access it.

  • noauto causes /media/ARCHIVE not to be mounted while the machine is booting.

  • x-systemd.device-timeout configures how long systemd should wait when no device is plugged in or an incorrect device is found.

On a running system, do

  • systemctl daemon reload

followed by

  • systemctl restart media-ARCHIVE.automount

after changing /etc/fstab.

See Also