updated interwikimap of backupninja, duplicity
|Deletions are marked like this.||Additions are marked like this.|
|Line 4:||Line 4:|
|[[http://packages.debian.org/duplicity|duplicity]] is a very useful tool to make remote unassisted backups. The backups can be incremental, encrypted, and over a variety of transports. Here is the example of setup to backup various directories remotely. Other tools such as [[http://packages.debian.org/backupninja|backupninja]] can be used for the same purpose.||[[DebianPkg:duplicity]] is a very useful tool to make remote unassisted backups. The backups can be incremental, encrypted, and over a variety of transports. Here is the example of setup to backup various directories remotely. Other tools such as [[DebianPkg:backupninja]] can be used for the same purpose.|
duplicity is a very useful tool to make remote unassisted backups. The backups can be incremental, encrypted, and over a variety of transports. Here is the example of setup to backup various directories remotely. Other tools such as backupninja can be used for the same purpose.
It is convenient to split the backups per directory, such as /etc, /var/lib/dpkg or /var/spool. This script will create small backups, one per directory defined later in BACKDIRS. The remote machine will contain the following directory tree in LPATH:
localmachine/etc/ localmachine/var/lib/dpkg localmachine/var/spool ...
and the commands we will run from localmachine will look like this:
duplicity --encrypt-key AE45AE12 --sign-key AE45AE12 \ remove-older-than 2W /var/spool \ scp://babar@remotehostname//home/babar/duplic/localmachine/var/spool
and then to remove the old backups:
duplicity --encrypt-key AE45AE12 --sign-key AE45AE12 \ remove-older-than 2W \ scp://babar@remotehostname//home/babar/duplic/localmachine/var/spool
You need first to know what directories to backup on your machine. A bare minimum to reconstruct the installation could be /etc and /var/lib/dpkg, then your mileage may vary. Think about twice, double check the size it takes, and the size you have one the server. How do the size vary along time? how long do you want to keep the old backups?
Here we keep 2 weeks of old backups, with a full backup every week and a daily increment.
Create the ssh and gpg keys
$ su # apt-get install duplicity keychain [...] # ssh-keygen -t dsa [...] # gpg --gen-key [...] # umask 077 # touch /root/backup.sh /root/.duplicity.conf # chmod u+x /root/backup.sh # ls -la /root/backup.sh /root/.duplicity.conf -rwx------ 1 root root 0 2006-01-16 06:47 /root/backup.sh -rw------- 1 root root 0 2006-01-16 06:47 /root/.duplicity.conf
Now you should have:
- a password protected gpg public/private key pair
localmachine:~# gpg --list-secret-keys /root/.gnupg/secring.gpg ------------------------ sec 1024D/AE45AE12 2005-08-26 [expires: 2016-08-26] uid Backup signature (localmachine) <root@localmachine> ssb 2048g/AE45AE12 2005-08-26
- an ssh private/public key pair
# ls -la /root/.ssh total 16 drwx------ 2 root root 54 2006-01-14 01:30 . drwxr-xr-x 13 root root 4096 2006-01-16 07:48 .. -rw------- 1 root root 1264 2006-01-13 21:30 id_dsa -rw-r--r-- 1 root root 1113 2006-01-13 21:30 id_dsa.pub
On the remote machine, the user used to receive the backups could be created like this:
# apt-get install scponly # adduser --disabled-password --shell /usr/bin/scponly babar # getent passwd babar babar:x:1002:1002:,,,:/home/babar:/usr/bin/scponly
Example of duplicity script file
First, the script file to run the backups: /root/backup.sh
# uncomment for debug #set -x . /root/.duplicity.conf # duplicity command DUPEXEC="duplicity --encrypt-key $ENCRKEY --sign-key $SIGNKEY $DUPOPTS $*" # loop on directories echo -n "---- Incremental backup of $HOSTNAME ---- "; date for i in $BACKDIRS do echo Starting backup of directory /$i # create directory, then backup, then erase old backups $MKDIR $LPATH/$i && $DUPEXEC /$i $RPATH/$i && $DUPEXEC $DUPOPTS_CLEANUP $RPATH/$i # verify backup integrity #$DUPEXEC --verify $RPATH/$i /$i done # if local, fix permissions if [ -z $HOST ]; then chown -R $NAME.$NAME $LPATH; fi echo -n "---- Finished backup on $HOSTNAME ---- "; date
Example of settings
Edit and complete the file /root/.duplicity.conf (XXXX must be replaced):
# path to backup to LPATH=/home/babar/XXX/$HOSTNAME ## 1. remote settings # remote host HOST=remotehostnameXXX # remote login (user for backup on server) NAME=babarXXX # send over ssh RPATH=scp://$NAME@$HOST/$LPATH ## 2. local settings (use another disk!) # remote host *empty* #HOST= # user name to change ownership too to #NAME=babar # RPATH now uses file:// #RPATH=file://$LPATH # complete with root gpg signature and encryption key SIGNKEY=XXXXXXXX ENCRKEY=$SIGNKEY # yes, we need to store the gpg pass phrase in clear somewhere export PASSPHRASE='XXXXXXXXXXXXXXXXXXXXXXXXXXXXX' # local list of directories to backup BACKDIRS='etc var/lib/dpkg var/log var/mail usr/local/XXX' # full '/' backup setting #BACKDIRS=/ #DUPOPTS='--exclude /proc --exclude /mnt --exclude /tmp' # duplicity options (backup.sh also accepts command line arguments) DUPOPTS= # cleanup (really needs --force) DUPOPTS_CLEANUP="remove-older-than 2W" # exclude patterns #DUPOPTS="$DUPOPTS --exclude **/pictures/XXX" # load ssh agent info using keychain [[ -f /root/.keychain/$HOSTNAME-sh ]] && \ source /root/.keychain/$HOSTNAME-sh export GNUPGHOME=/root/.gnupg if [ -z $HOST ]; then MKDIR="mkdir -p" else MKDIR="ssh $NAME@$HOST mkdir -p" fi
Testing the script
Add the following to your localmachine:/root/.bashrc
keychain --nogui --clear id_dsa . ~/.keychain/$HOSTNAME-sh
and login as root again. You should be prompted for the ssh passphrase.
The new ssh key needs to be installed on the server (i.e. add the content of localmachine:/root/.ssh/id_dsa.pub into remotemachine:/home/babar/.ssh/authorized_keys (-rw-------) ), and the directory LPATH created.
Then try running the script:
# /root/backup.sh ---- Incremental backup of localmachine ---- Mon Jan 16 06:49:25 GMT 2006 Starting backup of directory /home/barfoo No signatures found, switching to full backup. --------------[ Backup Statistics ]-------------- StartTime 1137394351.70 (Mon Jan 16 06:52:31 2006) EndTime 1137394838.14 (Mon Jan 16 07:00:38 2006) ElapsedTime 486.44 (8 minutes 6.44 seconds) SourceFiles 26330 SourceFileSize 677134571 (646 MB) NewFiles 4185 NewFileSize 125189830 (119 MB) DeletedFiles 0 ChangedFiles 1 ChangedFileSize 2190 (2.14 KB) ChangedDeltaSize 0 (0 bytes) DeltaEntries 4186 RawDeltaSize 99926952 (95.3 MB) TotalDestinationSizeChange 45211649 (43.1 MB) Errors 0 ------------------------------------------------- No old backup sets found, nothing deleted. ---- Finished backup on localmachine ---- Mon Jan 16 07:00:57 GMT 2006
The set -x line can be uncommented to debug the script.
Saving the keys
Now you have it working, you will need to backup the key, away from the machine to backup and from the remote backup storage. Somewhere safe, such as a usb key:
$ cd /media/usbdisk $ sudo tar zcvf root-$HOSTNAME.tar.gz /root
You may also need to store the ssh passphrase close to it.
To make the remote machine safer, you should also set the script of the backup user to scponly.
Making it automagic
You can now run manualy the script with
$ sudo /root/backup.sh
(you will need to add the ssh key to keychain after a reboot).
To recreate a full backup, add the --full flag. To delete the old backups, use --force.
To add it to cron:
$ sudo crontab -e
and put something like:
# m h dom mon dow command 33 23 * * 1-6 /root/backup.sh # full backup every sunday, deleting old ones 33 23 * * 7 /root/backup.sh --full --force
See also the duply package maybe.