Differences between revisions 47 and 49 (spanning 2 versions)
Revision 47 as of 2016-08-12 12:54:39
Size: 15098
Comment: Factorise code for waiting instance to change status; remove legacy euca2ools code.
Revision 49 as of 2016-08-13 11:14:04
Size: 15285
Comment: getInstanceIp()
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
= Official Debian-Installer AMIs =

''Official Amazon Machine Images for the DebianInstaller''
= Debian-Installer AMIs =

''Amazon Machine Images for the DebianInstaller''
Line 45: Line 45:

getInstanceIp() {
  aws ec2 describe-instances \
    --instance-id $1 \
    --query 'Reservations[0].Instances[0].NetworkInterfaces[0].Association.PublicIp' \
    --output text | tee /dev/stderr

Debian-Installer AMIs

Amazon Machine Images for the DebianInstaller

Work in progress, thus the instructions below are not guaranteed to work at the moment.

AMI build script

This is run in the Asia North-East zone from the Debian AMI Account (account number 379101102735). The operations in the cloud are controlled from the local computer using programs from the awscli package.

The procedure below runs a helper instance with an attached volume of 1 GiB, partitions it, downwoads Debian-Installer on it and sets up GRUB2 to boot the installer. The snapshot of the resulting volume is registered as a machine image.

First, declare a few accessory functions:

runHelperInstance() {
  aws ec2 run-instances \
    --image-id $1 \
    --instance-initiated-shutdown-behavior terminate \
    --instance-type $INSTANCE_TYPE \
    --block-device-mappings "$INSTANCE_BLKD" \
    --user-data $INSTANCE_DATA \
    --key-name $SSH_KEY_NAME \
    --security-group-id $SECURITY_GROUP \
    --query Instances[0].InstanceId \
    --output text | tee /dev/stderr

getVolumeIdentifier() {
  aws ec2 describe-instances \
    --instance-id $1 \
    --query 'Reservations[0].Instances[0].BlockDeviceMappings[?DeviceName==`/dev/sdb`].Ebs[].VolumeId' \
    --output text | tee /dev/stderr)

waitForInstanceStatus() {
  while [ ! $( aws ec2 describe-instances \
                 --instance-id $1 \
                 --query Reservations[0].Instances[0].State.Name \
                 --output text | tee /dev/stderr) = $2 ]                                                
    do sleep 30

getInstanceIp() {
  aws ec2 describe-instances \
    --instance-id $1 \
    --query 'Reservations[0].Instances[0].NetworkInterfaces[0].Association.PublicIp' \
    --output text | tee /dev/stderr

Then, configure:

INSTANCE_BLKD='[{"DeviceName":"/dev/sdb", "Ebs":{"DeleteOnTermination":false, "VolumeSize":1}}]'
SECURITY_GROUP="Your security group's ID" # At this stage, the instance does not need to be accessible.

At the moment, using Ubuntu (Ubuntu 16.04 LTS Xenial amd64 HVM EBS (Asia North-East)t) as helper system; Debian images with cloud-init enabled by default being tested.

HELPER_AMI=ami-23b54e42 # Ubuntu 16.04 LTS Xenial amd64 HVM EBS (Asia North-East)
# Debian AMI being tested now.
#HELPER_AMI=ami-d7d4c5b9 # Debian Jessie (Asia North-East)

Start the instance with an extra volume of 1 GiB, which will persist after termination (/dev/sdb=:1:false). Pass the script that will format the volume and download Debian-Installer (see below).


Get the volume's identifier.


Wait that the scripts shuts down the instance.

waitForInstanceStatus $HELPER_INSTANCE terminated

Snapshot the volume and register it as a machine image.

TARGET_SNAPSHOT=$( aws ec2 create-snapshot \
  --volume-id $TARGET_VOLUME \
  --query SnapshotId \
  --output text | tee /dev/stderr)

while aws ec2 describe-snapshots --snapshot-ids $TARGET_SNAPSHOT --query Snapshots[0].State --output text  |
  tee /dev/stderr |
  grep -q pending
  sleep 10

Delete the snapshoted volume, not needed anymore.

aws ec2 delete-volume --volume-id $TARGET_VOLUME

Note that AMI names can not contain the + character.

aws ec2 register-image \
  --name debian-jessie-installer-amd64-20150422-deb8u4 \
  --description 'Debian-Installer 8 (jessie) version 20150422+deb8u4 for amd64' \
  --block-device-mappings '[{"DeviceName":"/dev/sda1","Ebs": {"SnapshotId":"'$TARGET_SNAPSHOT'"}}]' \
  --architecture x86_64 \
  --root-device-name /dev/sda1 \
  --virtualization-type hvm

The script install-debian-installer follows here.

   1 #!/bin/sh -ex
   2 # License: CC0
   4 parted -s /dev/xvdb mklabel msdos mkpart primary 0% 100%
   5 mke2fs -L debian-installer /dev/xvdb1 -F
   6 mount LABEL=debian-installer /mnt/
   8 cd /mnt
  10 ARCH=amd64
  11 DIST=stable
  12 DI_VERSION=20150422+deb8u4
  13 MIRROR=http://cloudfront.debian.net
  14 BASEURL=$MIRROR/debian/dists/$DIST/main/installer-$ARCH/$DI_VERSION/images/netboot/xen
  16 wget $BASEURL/initrd.gz $BASEURL/vmlinuz
  18 mkdir -p boot/grub
  20 cat > boot/grub/grub.cfg <<__END__
  21 set default='0'
  22 set timeout='3'
  23 menuentry 'Debian Installer ($DI_VERSION $ARCH)' {
  24 insmod part_msdos
  25 insmod ext2
  26 set root='(hd0,msdos1)'
  27 linux /vmlinuz root=LABEL=debian-installer ro console=tty1 console=ttyS0 auto=true priority=critical url= DEBIAN_FRONTEND=text
  28 initrd /initrd.gz
  29 }
  30 __END__
  32 grub-install /dev/xvdb --boot-directory=/mnt/boot
  34 sleep 30
  36 halt


Work in progress; may be deleted.


Coming soon !

















Replicated and made public by hand using the web console.

How to use the images

Instance a Debian-Installer image and pass it a preseed file via the user data.

Note the similarity between the commands below and the commands above. They probably can be factorised in a single script that takes appropriate parameters.

Chose an installer image (for instance here: 379101102735/debian-wheezy-installer-amd64-20130613_deb7u1 on ap-northeast-1).


Chose the size of the final image, in gibibytes.


If you use the network console, pass the name of your key (in this example, name).


This time, pass the pressed file as user data.


Run the instance in a security group that has the SSH port open.

SECURITY_GROUP="Your security group's ID" # At this stage, the instance does not need to be accessible.


Wait that the instance is running.

waitForInstanceStatus $INSTALLER_INSTANCE running

Get the volume's identifier.


The rest is work in progress: preseed correctly to have accurate menu.lst and fstab files, prevent the creation of a swap partition, install cloud-init from backports …

Example preseed file

#### Adapted from http://www.debian.org/releases/wheezy/example-preseed.txt
### Localization
# Preseeding only locale sets language, country and locale.
d-i debian-installer/locale string en_US

# The values can also be preseeded individually for greater flexibility.
d-i debian-installer/language string en
d-i debian-installer/country string NL
d-i debian-installer/locale string en_GB.UTF-8
# Optionally specify additional locales to be generated.
#d-i localechooser/supported-locales multiselect en_US.UTF-8, nl_NL.UTF-8

# Keyboard selection.
# keymap is an alias for keyboard-configuration/xkb-keymap
# did not work with recent d-i: d-i keymap select us
d-i keyboard-configuration/xkb-keymap select us
# d-i keyboard-configuration/toggle select No toggling

# netcfg will choose an interface that has link if possible. This makes it
# skip displaying a list if there is more than one interface.
d-i netcfg/choose_interface select auto

# Any hostname and domain names assigned from dhcp take precedence over
# values set here. However, setting the values still prevents the questions
# from being shown, even if values come from dhcp.
d-i netcfg/get_hostname string unassigned-hostname
d-i netcfg/get_domain string unassigned-domain

# Disable that annoying WEP key dialog.
d-i netcfg/wireless_wep string

### Network console
# Use the following settings if you wish to make use of the network-console
# component for remote installation over SSH. This only makes sense if you
# intend to perform the remainder of the installation manually.
d-i anna/choose_modules string network-console
d-i network-console/authorized_keys_url
# Fixme: get the SSH authentication without setting a password.
d-i network-console/password password r00tme
d-i network-console/password-again password r00tme

### Using the CloudFront mirror (local in the EC2).
d-i mirror/country string manual
d-i mirror/http/hostname string cloudfront.debian.net
d-i mirror/http/directory string /debian
d-i mirror/http/proxy string

# Suite to install.
#d-i mirror/suite string testing
# Suite to use for loading installer components (optional).
#d-i mirror/udeb/suite string testing

### Account setup
# Skip creation of a root account (use sudo with "administrator" account).
d-i passwd/root-login boolean false
# Create an "administrator" account
d-i passwd/user-fullname string administrator
d-i passwd/username string administrator
# FIXME: no password should be set after cloud-init is installed
d-i passwd/user-password password FIXME
d-i passwd/user-password-again password FIXME

### Clock and time zone setup
# Controls whether or not the hardware clock is set to UTC.
d-i clock-setup/utc boolean true

# You may set this to any valid setting for $TZ; see the contents of
# /usr/share/zoneinfo/ for valid values.
d-i time/zone string US/Eastern

# Controls whether to use NTP to set the clock during the install
d-i clock-setup/ntp boolean true

### Partitioning
d-i partman-auto/disk string /dev/xvdb
# - regular: use the usual partition types for your architecture
d-i partman-auto/method string regular
# - atomic: all files in one partition
d-i partman-auto/choose_recipe select atomic
# This makes partman automatically partition without confirmation, provided
# that you told it what to do using one of the methods above.
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true

## Controlling how partitions are mounted
# Mount by label as UUIDs will change.
d-i partman/mount_style select label

### Base system installation
# The kernel image (meta) package to be installed; "none" can be used if no
# kernel is to be installed.
#d-i base-installer/kernel/image string linux-image-486

### Apt setup
# You can choose to install non-free and contrib software.
#d-i apt-setup/non-free boolean true
#d-i apt-setup/contrib boolean true
# Select which update services to use; define the mirrors to be used.
# Values shown below are the normal defaults.
#d-i apt-setup/services-select multiselect security, updates
#d-i apt-setup/security_host string security.debian.org

# Additional repositories, local[0-9] available
#d-i apt-setup/local0/repository string \
#       http://local.server/debian stable main
#d-i apt-setup/local0/comment string local server
# Enable deb-src lines
#d-i apt-setup/local0/source boolean true
# URL to the public key of the local repository; you must provide a key or
# apt will complain about the unauthenticated repository and so the
# sources.list line will be left commented out
#d-i apt-setup/local0/key string http://local.server/key

# By default the installer requires that repositories be authenticated
# using a known gpg key. This setting can be used to disable that
# authentication. Warning: Insecure, not recommended.
#d-i debian-installer/allow_unauthenticated boolean true

### Package selection
#tasksel tasksel/first multiselect standard, web-server
# If the desktop task is selected, install the kde and xfce desktops
# instead of the default gnome desktop.
#tasksel tasksel/desktop multiselect kde, xfce

# Individual additional packages to install
#d-i pkgsel/include string openssh-server build-essential
# Whether to upgrade packages after debootstrap.
# Allowed values: none, safe-upgrade, full-upgrade
#d-i pkgsel/upgrade select none

# Some versions of the installer can report back on what software you have
# installed, and what software you use. The default is not to report back,
# but sending reports helps the project determine what software is most
# popular and include it on CDs.
#popularity-contest popularity-contest/participate boolean false

### Finishing up the installation
# Avoid that last message about the install being complete.
d-i finish-install/reboot_in_progress note

# This will prevent the installer from ejecting the CD during the reboot,
# which is useful in some situations.
#d-i cdrom-detect/eject boolean false

# This is how to make the installer shutdown when finished, but not
# reboot into the installed system.
d-i debian-installer/exit/halt boolean true
# This will power off the machine instead of just halting it.
d-i debian-installer/exit/poweroff boolean true

### Preseeding other packages
# Depending on what software you choose to install, or if things go wrong
# during the installation process, it's possible that other questions may
# be asked. You can preseed those too, of course. To get a list of every
# possible question that could be asked during an install, do an
# installation, and then run these commands:
#   debconf-get-selections --installer > file
#   debconf-get-selections >> file

#### Advanced options
### Running custom commands during the installation
# d-i preseeding is inherently not secure. Nothing in the installer checks
# for attempts at buffer overflows or other exploits of the values of a
# preconfiguration file like this one. Only use preconfiguration files from
# trusted locations! To drive that home, and because it's generally useful,
# here's a way to run any shell command you'd like inside the installer,
# automatically.

# This first command is run as early as possible, just after
# preseeding is read.
#d-i preseed/early_command string anna-install some-udeb
# This command is run immediately before the partitioner starts. It may be
# useful to apply dynamic partitioner preseeding that depends on the state
# of the disks (which may not be visible when preseed/early_command runs).
#d-i partman/early_command \
#       string debconf-set partman-auto/disk "$(list-devices disk | head -n1)"
# This command is run just before the install finishes, but when there is
# still a usable /target directory. You can chroot to /target and use it
# directly, or use the apt-install and in-target commands to easily install
# packages and run commands in the target system.
#d-i preseed/late_command string apt-install zsh; in-target chsh -s /bin/zsh

See also Cloud/AmazonEC2Image, and Cloud.