Differences between revisions 33 and 34
Revision 33 as of 2016-08-08 21:51:06
Size: 15945
Comment: Actually, use EBS, as instance-store is not supported on t2.nano instances.
Revision 34 as of 2016-08-09 12:18:18
Size: 16320
Comment: Register D-I image with the `register-image` command.
Deletions are marked like this. Additions are marked like this.
Line 147: Line 147:
}}}

{{{
aws ec2 register-image \
  --name debian-wheezy-installer-amd64-20130613-deb7u3-b2 \
  --description 'Debian-Installer 7 (Wheezy) version 20130613+deb7u1 for amd64' \
  --block-device-mappings '[{"DeviceName":"/dev/sda1","Ebs": {"SnapshotId":"'$TARGET_SNAPSHOT'"}}]' \
  --architecture x86_64 \
  --root-device-name /dev/sda1 \
  --virtualization-type hvm

Official Debian-Installer AMIs

Official Amazon Machine Images for the DebianInstaller

AMI build script

Note: the commands in this page are currently being updated, to use awscli instead of euca2ools.

Not yet a script... This is run in the Asia North-East zone from the Debian AMI Account (account number 379101102735). The operations in the cloud are controlled from the local computer using programs from the awscli package.

The procedure below runs a helper instance with an attached volume of 1 GiB, partitions it, downwoads Debian-Installer on it and sets up a PV-GRUB configuration file to boot the installer. The snapshot of the resulting volume is registered as a machine image.

Using Ubuntu (12.04 LTS Precise amd64 EBS Asia North-East) as long as we do not have Debian images with cloud-init enabled by default.

HELPER_AMI=ami-7609bb77 # Ubuntu 12.04 LTS Precise amd64 EBS (Asia North-East)
# More recent Ubuntu AMI being tested now.
#HELPER_AMI=ami-23b54e42 # Ubuntu 16.04 LTS Xenial amd64 HVM EBS (Asia North-East)
# Debian AMI being tested now.
#HELPER_AMI=ami-d7d4c5b9 # Debian Jessie (Asia North-East)

Start the instance with an extra volume of 1 GiB, which will persist after termination (/dev/sdb=:1:false). Pass the script that will format the volume and download Debian-Installer (see below).

With euca2ools:

HELPER_INSTANCE=$( euca-run-instances \
        --instance-initiated-shutdown-behavior terminate \
        --instance-type t1.micro \
        --block-device-mapping /dev/sdb=:1:false \
        --user-data-file install-debian-installer \
        $HELPER_AMI |
        tee /dev/stderr | grep INSTANCE | awk '{print $2}')

With aws:

HELPER_INSTANCE=$( aws ec2 run-instances \
  --instance-initiated-shutdown-behavior terminate \
  --instance-type t2.nano \
  --block-device-mappings '[{"DeviceName":"/dev/sdb", "Ebs":{"DeleteOnTermination":false, "VolumeSize":1}}]' \
  --user-data file://install-debian-installer \
  --image-id $HELPER_AMI \
  --query Instances[0].InstanceId \
  --output text | tee /dev/stderr)

Wait that the instance is running.

while [ ! $(euca-describe-instances $HELPER_INSTANCE | grep INSTANCE | cut -f 6 | tee /dev/stderr) = "running" ]
    do sleep 30
done

# No need to wait with `aws`.

Get the volume's identifier.

TARGET_VOLUME=$( euca-describe-volumes |
        grep $HELPER_INSTANCE | grep '/dev/sdb' |
        tee /dev/stderr | awk '{print $2}')

TARGET_VOLUME=$( aws ec2 describe-instances \
  --instance-id $HELPER_INSTANCE \
  --output text \
  --query 'Reservations[0].Instances[0].BlockDeviceMappings[?DeviceName==`/dev/sdb`].Ebs[].VolumeId' |
  tee /dev/stderr)

Wait that the scripts shuts down the instance.

while [ ! $(euca-describe-instances $HELPER_INSTANCE | grep INSTANCE | cut -f 6 | tee /dev/stderr) = "terminated" ]
    do sleep 30
done

while [ ! $( aws ec2 describe-instances \
               --instance-id $HELPER_INSTANCE \
               --output text \
               --query Reservations[0].Instances[0].State.Name |
               tee /dev/stderr) = "terminated" ]                                                
  do sleep 30
done

Snapshot the volume and register it as a machine image.

PV_KERNEL=aki-176bf516 # Boot PV-Grub (Asia North-East)

TARGET_SNAPSHOT=$( euca-create-snapshot $TARGET_VOLUME |
        tee /dev/stderr | awk '{print $2}')

while euca-describe-snapshots $TARGET_SNAPSHOT | grep -q pending ; do sleep 30 ; done

TARGET_SNAPSHOT=$( aws ec2 create-snapshot \
  --volume-id $TARGET_VOLUME \
  --output text \
  --query SnapshotId |
  tee /dev/stderr)

while aws ec2 describe-snapshots --snapshot-ids $TARGET_SNAPSHOT --output text --query Snapshots[0].State |
  tee /dev/stderr |
  grep -q pending
do
  sleep 10
done

Delete the snapshoted volume, not needed anymore.

euca-delete-volume $TARGET_VOLUME

aws ec2 delete-volume --volume-id $TARGET_VOLUME

Note that AMI names can not contain the + character.

euca-register \
    --name 'debian-wheezy-installer-amd64-20130613_deb7u1' \
    --description 'Debian-Installer 7 (Wheezy) version 20130613+deb7u1 for amd64' \
    --snapshot $TARGET_SNAPSHOT \
    --kernel $PV_KERNEL \
    --architecture x86_64 \
    --root-device-name /dev/sda

aws ec2 register-image \
  --name debian-wheezy-installer-amd64-20130613-deb7u3-b2 \
  --description 'Debian-Installer 7 (Wheezy) version 20130613+deb7u1 for amd64' \
  --block-device-mappings '[{"DeviceName":"/dev/sda1","Ebs": {"SnapshotId":"'$TARGET_SNAPSHOT'"}}]' \
  --architecture x86_64 \
  --root-device-name /dev/sda1 \
  --virtualization-type hvm

The script install-debian-installer follows here.

   1 #!/bin/sh -ex
   2 # License: CC0
   3 
   4 parted -s /dev/xvdb mklabel msdos mkpart primary 0% 100%
   5 mke2fs -L debian-installer /dev/xvdb1 -F
   6 mount LABEL=debian-installer /mnt/
   7 
   8 cd /mnt
   9 
  10 ARCH=amd64
  11 DIST=wheezy
  12 DI_VERSION=20130613+deb7u3+b2
  13 MIRROR=http://cloudfront.debian.net
  14 BASEURL=$MIRROR/debian/dists/$DIST/main/installer-$ARCH/$DI_VERSION/images/netboot/xen
  15 
  16 wget $BASEURL/initrd.gz $BASEURL/vmlinuz
  17 
  18 mkdir -p boot/grub
  19 
  20 cat > boot/grub/menu.lst <<__END__
  21 default 0
  22 timeout 3
  23 
  24 title  Debian Installer ($DI_VERSION $ARCH)
  25 root   (hd0,0)
  26 kernel /vmlinuz root=LABEL=debian-installer ro console=hvc0 auto=true priority=critical url=http://169.254.169.254/latest/user-data DEBIAN_FRONTEND=text
  27 initrd /initrd.gz
  28 __END__
  29 
  30 sleep 30
  31 
  32 halt

Images

Work in progress; may be deleted.

379101102735/debian-wheezy-installer-amd64-20130613_deb7u1

Coming soon !

ap-northeast-1

ami-a1861ca0

ap-southeast-1

ami-e03164b2

ap-southeast-2

ami-fbb529c1

eu-west-1

ami-efb85b98

us-east-1

ami-29fea140

us-west-1

ami-0eaa9d4b

us-west-2

ami-1ece562e

sa-east-1

ami-bfb81ea2

Replicated and made public by hand using the web console.

How to use the images

Instance a Debian-Installer image and pass it a preseed file via the user data.

Note the similarity between the commands below and the commands above. They probably can be factorised in a single script that takes appropriate parameters.

Chose an installer image (for instance here: 379101102735/debian-wheezy-installer-amd64-20130613_deb7u1 on ap-northeast-1).

INSTALLER_IMAGE=ami-b38e14b2

Chose the size of the final image, in gibibytes.

VOLUME_SIZE=8

If you use the network console, pass the name of your key (in this example, name).

SSH_KEY_NAME="-k name"

INSTALLER_INSTANCE=$( euca-run-instances \
        --instance-initiated-shutdown-behavior terminate \
        --instance-type t1.micro \
        --block-device-mapping /dev/sdb=:${VOLUME_SIZE}:false \
        --user-data-file preseed.txt \
        $SSH_KEY_NAME \
        $INSTALLER_IMAGE |
        tee /dev/stderr | grep INSTANCE | awk '{print $2}')

Wait that the instance is running.

while [ ! $(euca-describe-instances $INSTALLER_INSTANCE | grep INSTANCE | cut -f 6 | tee /dev/stderr) = "running" ]
    do sleep 30
done

Get the volume's identifier.

TARGET_VOLUME=$( euca-describe-volumes |
        grep $INSTALLER_INSTANCE | grep '/dev/sdb' |
        tee /dev/stderr | awk '{print $2}')

The rest is work in progress: preseed correctly to have accurate menu.lst and fstab files, prevent the creation of a swap partition, install cloud-init from backports …

Example preseed file

#### Adapted from http://www.debian.org/releases/wheezy/example-preseed.txt
### Localization
# Preseeding only locale sets language, country and locale.
d-i debian-installer/locale string en_US

# The values can also be preseeded individually for greater flexibility.
d-i debian-installer/language string en
d-i debian-installer/country string NL
d-i debian-installer/locale string en_GB.UTF-8
# Optionally specify additional locales to be generated.
#d-i localechooser/supported-locales multiselect en_US.UTF-8, nl_NL.UTF-8

# Keyboard selection.
# keymap is an alias for keyboard-configuration/xkb-keymap
# did not work with recent d-i: d-i keymap select us
d-i keyboard-configuration/xkb-keymap select us
# d-i keyboard-configuration/toggle select No toggling

# netcfg will choose an interface that has link if possible. This makes it
# skip displaying a list if there is more than one interface.
d-i netcfg/choose_interface select auto

# Any hostname and domain names assigned from dhcp take precedence over
# values set here. However, setting the values still prevents the questions
# from being shown, even if values come from dhcp.
d-i netcfg/get_hostname string unassigned-hostname
d-i netcfg/get_domain string unassigned-domain

# Disable that annoying WEP key dialog.
d-i netcfg/wireless_wep string

### Network console
# Use the following settings if you wish to make use of the network-console
# component for remote installation over SSH. This only makes sense if you
# intend to perform the remainder of the installation manually.
d-i anna/choose_modules string network-console
d-i network-console/authorized_keys_url http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key
# Fixme: get the SSH authentication without setting a password.
d-i network-console/password password r00tme
d-i network-console/password-again password r00tme

### Using the CloudFront mirror (local in the EC2).
d-i mirror/country string manual
d-i mirror/http/hostname string cloudfront.debian.net
d-i mirror/http/directory string /debian
d-i mirror/http/proxy string

# Suite to install.
#d-i mirror/suite string testing
# Suite to use for loading installer components (optional).
#d-i mirror/udeb/suite string testing

### Account setup
# Skip creation of a root account (use sudo with "administrator" account).
d-i passwd/root-login boolean false
# Create an "administrator" account
d-i passwd/user-fullname string administrator
d-i passwd/username string administrator
# FIXME: no password should be set after cloud-init is installed
d-i passwd/user-password password FIXME
d-i passwd/user-password-again password FIXME

### Clock and time zone setup
# Controls whether or not the hardware clock is set to UTC.
d-i clock-setup/utc boolean true

# You may set this to any valid setting for $TZ; see the contents of
# /usr/share/zoneinfo/ for valid values.
d-i time/zone string US/Eastern

# Controls whether to use NTP to set the clock during the install
d-i clock-setup/ntp boolean true

### Partitioning
d-i partman-auto/disk string /dev/xvdb
# - regular: use the usual partition types for your architecture
d-i partman-auto/method string regular
# - atomic: all files in one partition
d-i partman-auto/choose_recipe select atomic
# This makes partman automatically partition without confirmation, provided
# that you told it what to do using one of the methods above.
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true

## Controlling how partitions are mounted
# Mount by label as UUIDs will change.
d-i partman/mount_style select label

### Base system installation
# The kernel image (meta) package to be installed; "none" can be used if no
# kernel is to be installed.
#d-i base-installer/kernel/image string linux-image-486

### Apt setup
# You can choose to install non-free and contrib software.
#d-i apt-setup/non-free boolean true
#d-i apt-setup/contrib boolean true
# Select which update services to use; define the mirrors to be used.
# Values shown below are the normal defaults.
#d-i apt-setup/services-select multiselect security, updates
#d-i apt-setup/security_host string security.debian.org

# Additional repositories, local[0-9] available
#d-i apt-setup/local0/repository string \
#       http://local.server/debian stable main
#d-i apt-setup/local0/comment string local server
# Enable deb-src lines
#d-i apt-setup/local0/source boolean true
# URL to the public key of the local repository; you must provide a key or
# apt will complain about the unauthenticated repository and so the
# sources.list line will be left commented out
#d-i apt-setup/local0/key string http://local.server/key

# By default the installer requires that repositories be authenticated
# using a known gpg key. This setting can be used to disable that
# authentication. Warning: Insecure, not recommended.
#d-i debian-installer/allow_unauthenticated boolean true

### Package selection
#tasksel tasksel/first multiselect standard, web-server
# If the desktop task is selected, install the kde and xfce desktops
# instead of the default gnome desktop.
#tasksel tasksel/desktop multiselect kde, xfce

# Individual additional packages to install
#d-i pkgsel/include string openssh-server build-essential
# Whether to upgrade packages after debootstrap.
# Allowed values: none, safe-upgrade, full-upgrade
#d-i pkgsel/upgrade select none

# Some versions of the installer can report back on what software you have
# installed, and what software you use. The default is not to report back,
# but sending reports helps the project determine what software is most
# popular and include it on CDs.
#popularity-contest popularity-contest/participate boolean false

### Finishing up the installation
# Avoid that last message about the install being complete.
d-i finish-install/reboot_in_progress note

# This will prevent the installer from ejecting the CD during the reboot,
# which is useful in some situations.
#d-i cdrom-detect/eject boolean false

# This is how to make the installer shutdown when finished, but not
# reboot into the installed system.
d-i debian-installer/exit/halt boolean true
# This will power off the machine instead of just halting it.
d-i debian-installer/exit/poweroff boolean true

### Preseeding other packages
# Depending on what software you choose to install, or if things go wrong
# during the installation process, it's possible that other questions may
# be asked. You can preseed those too, of course. To get a list of every
# possible question that could be asked during an install, do an
# installation, and then run these commands:
#   debconf-get-selections --installer > file
#   debconf-get-selections >> file

#### Advanced options
### Running custom commands during the installation
# d-i preseeding is inherently not secure. Nothing in the installer checks
# for attempts at buffer overflows or other exploits of the values of a
# preconfiguration file like this one. Only use preconfiguration files from
# trusted locations! To drive that home, and because it's generally useful,
# here's a way to run any shell command you'd like inside the installer,
# automatically.

# This first command is run as early as possible, just after
# preseeding is read.
#d-i preseed/early_command string anna-install some-udeb
# This command is run immediately before the partitioner starts. It may be
# useful to apply dynamic partitioner preseeding that depends on the state
# of the disks (which may not be visible when preseed/early_command runs).
#d-i partman/early_command \
#       string debconf-set partman-auto/disk "$(list-devices disk | head -n1)"
# This command is run just before the install finishes, but when there is
# still a usable /target directory. You can chroot to /target and use it
# directly, or use the apt-install and in-target commands to easily install
# packages and run commands in the target system.
#d-i preseed/late_command string apt-install zsh; in-target chsh -s /bin/zsh


See also Cloud/AmazonEC2Image, and Cloud.