Copying a Debian hybrid ISO safely onto a USB flash drive

Several of the Debian and Debian Live images, notably all i386, amd64 and arm64 images, are created using the isohybrid technology and thus may be written directly to USB flash drives to make them bootable via the BIOS / EFI firmware of most PCs.

In general you need to unmount all mounted filesystems of the target device and then to plainly copy the image file bytes onto the device, without using any of its partitions or filesystems.

This article shows how to do this by assistance of program xorriso-dd-target in order to minimize the risk of overwriting the wrong storage device.

Contents

  1. Copying a Debian hybrid ISO safely onto a USB flash drive
    1. Identify the device by plugging and copy if it looks safe enough
    2. How to overwrite a drive against the will of xorriso-dd-target
    3. How to verify the result
    4. lsblk and bootable ISOs on USB stick

Identify the device by plugging and copy if it looks safe enough

On a GNU/Linux machine, install package xorriso-dd-target.

If no such package exists, download https://dev.lovelyhq.com/libburnia/libisoburn/raw/master/xorriso-dd-target/xorriso-dd-target and put it into a directory where your shell looks for executable programs. For verification you may download the .sig file and use gpg --verify like described for the GNU xorriso tarball. Finally give x-permission to the downloaded file xorriso-dd-target.

The program will not overwrite the flash drive if it contains filesystems other than VFAT and ISO 9660. In this case you will not be done at the end of this chapter but will have to go on with the next one.

Have the USB flash drive unplugged and start the program with e.g 

file="debian-10.0.0-amd64-netinst.iso"

by 

xorriso-dd-target -with_sudo -plug_test -DO_WRITE -image_file "$file"

You will probably be asked by program sudo for your password. sudo power will only be used with runs of programs lsblk, umount, and dd. 

Testing sudo to possibly get password prompting done now:
[sudo] password for thomas: 
sudo /bin/lsblk seems ok.

Alternatively you may run the whole program with superuser authority and without option -with_sudo.

Dialog step 1 is to confirm that the drive is really not plugged in. Your still attached drives will be registered as unwanted for copying. 

Step 1:
Please make sure that the desired target device is plugged _out_ now.
If it is currently plugged in, make sure to unmount all its fileystems
and then unplug it.
Press the Enter key when ready.

Found and noted as _not_ desired:  sda sdb sdc

Step 2 asks you to plug in the drive and to press Enter again. Wait with the key press until you expect that the drive has been recognized by the operating system. After the key press the program looks for a drive which is not in the list of unwanted drives. 

Step 2:
Please plug in the desired target device and then press the Enter key.

Waiting up to 10 seconds for a new device to be listed ... found: sdd
Now waiting 5 seconds to let it settle .........
Found and noted as desired device:  sdd

If the drive is recognized as the only new device and looking suitable for being entirely overwritten, you will see a line like 

sdd : YES : usb+ has_iso9660+ has_vfat+ : Intenso Ultra Line

Dialog Step 3 asks you to enter the word "yes". The program will then try to unmount possibly automatically mounted filesystems. In case of success it will copy the image file onto the flash drive. This will overwrite the drive's old partition table and substantial parts of its content. 

Step 3:
Last chance to abort. Enter the word 'yes' to start REAL WRITING.
yes
Looking for mount points of sdd:
  /dev/sdd1 on /mnt/iso type iso9660 (ro,relatime)
  /dev/sdd2 on /mnt/fat type vfat (rw,...,errors=remount-ro)
Unmounted: /dev/sdd1
Unmounted: /dev/sdd2
Performing:
  sudo /bin/dd if=/dev/zero of=/dev/'sdd' bs=512 seek='245759999' count=1 status=none
  sudo /bin/dd if='debian-10.0.0-amd64-netinst.iso' bs=1M of=/dev/'sdd' status=progress oflag=dsync ; sync

The first dd run writes zeros into the last block of the drive in order to invalidate a possibly present GPT backup table. It might get omitted if program "expr" on an old 32 bit system cannot cope with large numbers.

The second dd run copies the ISO image to the drive. If dd is young enough, options  status=progress oflag=dsync  get added to the dd command to let it print progress messages at realistic speed.

If the drive does not look safe for being overwritten, then the program refuses instead of performing step 3, unmounting, GPT backup zeroing, and copying. The reason of refusal in this case is the presence of an ext2 filesystem. 

sdd : NO  : usb+ has_iso9660+ has_vfat+ has_ext2- : SanDisk Cruzer 
NO CANDIDATE FOR TARGET DEVICE AMONG THE GIVEN NAMES !
Overall available devices:
sda : NO  : not_usb- has_vfat+ has_ext4- : ATA Samsung SSD 850 
sdb : NO  : not_usb- has_swap- has_ext4- : ATA WDC WD20EFRX-68A 
sdc : YES : usb+ has_iso9660+ has_vfat+ : Intenso Ultra Line 
sdd : NO  : usb+ has_iso9660+ has_vfat+ has_ext2- : SanDisk Cruzer

How to overwrite a drive against the will of xorriso-dd-target

If anything looks suspicious, the program will refuse to unmount and to overwrite. You may ask it then for a detailed assessment of the drive and for showing the shell commands which it does not dare to perform. 

xorriso-dd-target -with_sudo -dummy_force -list_long -image_file "$file" sdd

will report something like 

sdd : NO  : usb+ has_iso9660+ has_vfat+ has_ext2-
NAME    SIZE FSTYPE  TRAN LABEL
sdd     3.8G iso9660 usb  d-live 9.5.0 xf i386
|-sdd1  1.9G iso9660      d-live 9.5.0 xf i386
|-sdd2  320K vfat
`-sdd3  512M ext2

Overriding any advice because of -dummy_force
Looking for mount points of sdd:
  /dev/sdd1 on /mnt/iso type iso9660 (ro,relatime)
  /dev/sdd2 on /mnt/fat type vfat (rw,...,errors=remount-ro)
  /dev/sdd3 on /mnt/ext type ext2 (rw,relatime)
AGAINST THE ADVICE BY THIS PROGRAM, a daring user could do:
  sudo /bin/umount /dev/sdd1
  sudo /bin/umount /dev/sdd2
  sudo /bin/umount /dev/sdd3
  sudo /bin/dd if=/dev/zero of=/dev/'sdd' bs=512 seek='7864318' count=1 status=none
  sudo /bin/dd if='debian-10.0.0-amd64-netinst.iso' bs=1M of=/dev/'sdd' status=progress oflag=dsync ; sync
BE SMART. BE CAUTIOUS. BEWARE.

Now it is up to you to judge whether you want to perform the five shown sudo commands.

Details and more use cases are given by 

man xorriso-dd-target

Online: xorriso-dd-target(1)

How to verify the result

Debian provides two stages of checksum files along with the ISO images. Currently these are files with the names "SHA256SUMS", "SHA256SUMS.sign", "SHA512SUMS", and "SHA512SUMS.sign".

The files with the suffix ".sign" contain the PGP signatures of the files without that suffix. These contain SHA256 or SHA512 checksums together with names of the .iso image files.

Verification of the downloaded ISO and of the readily written USB stick is described in detail by the Debian CD FAQ verification instructions . Summary: Use program "sha256sum" or "sha512sum" to compute the checksum of ISO image or USB stick. When verifying the USB stick, make sure to curb the data stream length to the size of the ISO image. There is a script named check_debian_iso which does curbing and checksum lookup for you.

If the verification attempt yields a non-matching checksum although you are quite sure that you got an original Debian ISO, it is possible to look for files in the ISO which got hit by the damage. This works only if the damage does not hamper mountability of the ISO filesystem.

Mount the damaged ISO with a mountpoint of your choice 

path_to_image_or_usb_device=/dev/sdc
mountpoint=/mnt/iso

sudo mount "$path_to_image_or_usb_device" "$mountpoint"

Let program "md5sum" verify the files listed in "$mountpoint"/md5sum.txt. Because its output is long, redirect it to a file of your choice 

report_file=/tmp/md5_report
cd "$mountpoint"
md5sum -c ./md5sum.txt >"$report_file" 2>&1

Open the report file and look for line ends which say ": FAILED". The file paths before that end snippet tell which files are damaged. Classic victim is the EFI System Partition image file: 

./boot/grub/efi.img: FAILED

It often gets altered by proprietary software when you plug the USB stick into a running MS-Windows system. Normally this damage is harmless and just indicates that some small files were added to the FAT filesystem in the system partition image. But such alteration of course weakens the credibility of your download and copy efforts.

PGP verification of the checksum files can be done by e.g. 

gpg --keyserver keyring.debian.org --recv-keys 64E6EA7D 6294BE9B 09EA8AC3
gpg --with-fingerprint --verify SHA512SUMS.sign SHA512SUMS

which in case of successful verification must report essentially 

gpg: Good signature from "Debian CD signing key <debian-cd@lists.debian.org>"
...
Primary key fingerprint: DF9B 9C49 EAA9 2984 3258  9D76 DA87 E80D 6294 BE9B

The key title and the key fingerprint must match one of the pairs "Key fingerprint =" and "uid" as listed on the Debian authenticity verification page .

lsblk and bootable ISOs on USB stick

The assessment of devices and filesystems is done by program lsblk. See 

man lsblk

A known problem of younger lsblk with bootable ISOs on USB stick is that filesystem type and label of the ISO is falsely reported with partitions without filesystem or with empty label text: 

NAME    SIZE FSTYPE  TRAN LABEL
sdd     3.8G iso9660 usb  d-live 9.5.0 xf i386
|-sdd1  1.9G iso9660      d-live 9.5.0 xf i386
|-sdd2  320K vfat         d-live 9.5.0 xf i386
`-sdd3  512M ext2         d-live 9.5.0 xf i386

This was caused by a udev rule which is now fixed upstream by https://github.com/systemd/systemd/commit/19212f27816686a5cac2c965301cea8624ac467f and hopefully will be fixed at latest in Debian 11.

The effect on xorriso-dd-target is only cosmetic and does not affect the safety of drive evaluation.