5233
Comment: created new page to setup duplicity
|
9567
|
Deletions are marked like this. | Additions are marked like this. |
Line 2: | Line 2: |
[[TableOfContents(1)]] It is convenient to split the backups per directory, such as {{{/etc}}}, {{{/var/lib/dpkg}}} or {{{/var/spool}}}. This script will create small backups, one per directory defined later in BACKDIRS. The remote machine will contain the following directory tree in LPATH: {{{ localmachine/etc/ localmachine/var/lib/dpkg localmachine/var/spool ... }}} and the commands we will run from localmachine will look like this: {{{ duplicity --encrypt-key AE45AE12 --sign-key AE45AE12 \ --remove-older-than 2W /var/spool \ scp://babar@remotehostname//home/babar/duplic/localmachine/var/spool }}} and then to remove the old backups: {{{ duplicity --encrypt-key AE45AE12 --sign-key AE45AE12 \ --remove-older-than 2W \ scp://babar@remotehostname//home/babar/duplic/localmachine/var/spool }}} You need first to know what directories to backup on your machine. A bare minimum to reconstruct the installation could be {{{/etc}}} and {{{/var/lib/dpkg}}}, then your mileage may vary. Think about twice, double check the size it takes, and the size you have one the server. How do the size vary along time? how long do you want to keep the old backups? Here we keep 2 weeks of old backups, with a full backup every week and a daily increment. = Create the ssh and gpg keys = |
|
Line 13: | Line 43: |
# chmod u+x /root/backup.sh | |
Line 18: | Line 49: |
= Example of duplicity script file /root/backup.sh = | Now you should have: * a password protected gpg public/private key pair {{{ localmachine:~# gpg --list-secret-keys /root/.gnupg/secring.gpg ------------------------ sec 1024D/AE45AE12 2005-08-26 [expires: 2016-08-26] uid Backup signature (localmachine) <root@localmachine> ssb 2048g/AE45AE12 2005-08-26 }}} * an ssh private/public key pair {{{ # ls -la /root/.ssh total 16 drwx------ 2 root root 54 2006-01-14 01:30 . drwxr-xr-x 13 root root 4096 2006-01-16 07:48 .. -rw------- 1 root root 1264 2006-01-13 21:30 id_dsa -rw-r--r-- 1 root root 1113 2006-01-13 21:30 id_dsa.pub }}} On the remote machine, the user used to receive the backups could be created like this: {{{ # apt-get install scponly # adduser --disabled-password --shell /usr/bin/scponly babar # getent passwd babar babar:x:1002:1002:,,,:/home/babar:/usr/bin/scponly }}} = Example of duplicity script file = First, the script file to run the backups: /root/backup.sh |
Line 30: | Line 92: |
echo -n ---- Incremental backup of $HOSTNAME ---- ;date | echo -n "---- Incremental backup of $HOSTNAME ---- "; date |
Line 41: | Line 103: |
echo -n ---- Finished backup on $HOSTNAME ---- ;date }}} = Example of settings file /root/.duplicity.conf = |
echo -n "---- Finished backup on $HOSTNAME ---- "; date }}} = Example of settings = Edit and complete the file /root/.duplicity.conf (XXXX must be replaced): |
Line 50: | Line 114: |
LPATH=/home/babar/precious/$HOSTNAME | LPATH=/home/babar/XXX/$HOSTNAME |
Line 54: | Line 118: |
HOST=remotehostname | HOST=remotehostnameXXX |
Line 56: | Line 120: |
NAME=babar | NAME=babarXXX |
Line 60: | Line 124: |
## 2. local settings | ## 2. local settings (use another disk!) |
Line 75: | Line 139: |
BACKDIRS='etc var/lib/dpkg var/log var/mail usr/loca/more/XXX opt/another/dirXXXXXXX' |
BACKDIRS='etc var/lib/dpkg var/log var/mail usr/local/XXX' |
Line 85: | Line 148: |
DUPOPTS='--remove-older-than 2W' | DUPOPTS="$DUPOPTS --remove-older-than 2W" |
Line 87: | Line 150: |
DUPOPTS='$DUPOPTS --exclude **/pictures/XXX' | #DUPOPTS="$DUPOPTS --exclude **/pictures/XXX" |
Line 98: | Line 161: |
MKDIR="ssh $HOST mkdir -p" | MKDIR="ssh $NAME@$HOST mkdir -p" |
Line 102: | Line 165: |
= Testing the whole thing = | = Testing the script = |
Line 111: | Line 174: |
and login as root again. You should be prompted for the ssh passphrase. Once you have also added the content of {{{localmachine:/root/.ssh/id_dsa.pub}}} into {{{remotemachine:/home/babar/.ssh/authorized_keys (-rw-------)}}}, try running the script: |
and login as root again. You should be prompted for the ssh passphrase. The new ssh key needs to be installed on the server (i.e. add the content of {{{localmachine:/root/.ssh/id_dsa.pub}}} into {{{remotemachine:/home/babar/.ssh/authorized_keys (-rw-------)}}} ), and the directory LPATH created. Then try running the script: |
Line 153: | Line 218: |
To make the remote machine safer, you could also set the script of the backup user to scponly. | You may also need to store the ssh passphrase close to it. To make the remote machine safer, you should also set the script of the backup user to scponly. |
Line 157: | Line 224: |
You can now run manualy the script with | You can now run manualy the script with |
Line 174: | Line 241: |
33 23 * * 7 /root/backup.sh --full --force | 33 23 * * 7 /root/backup.sh --full --force |
Line 180: | Line 247: |
[http://homepage.mac.com/pallors1/73.html hottie webcam] | [http://sum.freewebsitehosting.com/36.html hardcore female stripers] | [http://karin.bravepages.com/49.html grannies pantyhose] | [http://homepage.mac.com/gismo3/23.html webcam chat hack] | [http://dissuasive.ibnsites.com/49.html gay watersport pee] | [http://gasolines.envy.nu/35.html female ejaculation movie] | [http://shitted.exactpages.com/48.html hot chat webcam] | [http://northwests.envy.nu/8.html orgy stitches video] | [http://nightstick.angelcities.com/56.html wife story gangbang] | [http://homepage.mac.com/zip17/18.html college webcam stripping] | [http://yeshivoth.150m.com/3.html housewives tits] | [http://radiosonde.g0g.net/19.html perfect pussy] | [http://homepage.mac.com/arks2/12.html webcam software gratis] | [http://homepage.mac.com/swazi2/32.html gangbang cream pies] | [http://shitted.exactpages.com/96.html young webcam boobs] | [http://sedately.1sweethost.com/67.html xxx final fantasy] | [http://homepage.mac.com/seawards/46.html girl masturbate webcam] | [http://pagodas.freewebpages.org/76.html teen chair webcam] | [http://leviers.ibnsites.com/57.html webcams sex] | [http://haricot.kogaryu.com/3.html young women masturbating] | [http://towtruck.freewebsitehosting.com/37.html blow webcam] | [http://sailings.1sweethost.com/9.html sapphic vid jo] | [http://homepage.mac.com/canola/89.html sex bare feet] | [http://homepage.mac.com/kibbutz1/78.html s bathroom webcam] | [http://saran.freewebsitehosting.com/57.html pics men pissing] | [http://forfend.freecities.com/87.html webcam dorm] | [http://headways.angelcities.com/70.html bondage handcuffs whips] | [http://homepage.mac.com/platoon1/25.html personnal webcam sites] | [http://precinct.100freemb.com/86.html fucked my grandma] | [http://sedition.1accesshost.com/44.html women breasts dick] |
[http://packages.debian.org/duplicity duplicity] is a very useful tool to make remote unassisted backups. The backups can be incremental, encrypted, and over a variety of transports. Here is the example of setup to backup various directories remotely. Other tools such as [http://packages.debian.org/backupninja backupninja] can be used for the same purpose.
?TableOfContents(1)
It is convenient to split the backups per directory, such as /etc, /var/lib/dpkg or /var/spool. This script will create small backups, one per directory defined later in BACKDIRS. The remote machine will contain the following directory tree in LPATH:
localmachine/etc/ localmachine/var/lib/dpkg localmachine/var/spool ...
and the commands we will run from localmachine will look like this:
duplicity --encrypt-key AE45AE12 --sign-key AE45AE12 \ --remove-older-than 2W /var/spool \ scp://babar@remotehostname//home/babar/duplic/localmachine/var/spool
and then to remove the old backups:
duplicity --encrypt-key AE45AE12 --sign-key AE45AE12 \ --remove-older-than 2W \ scp://babar@remotehostname//home/babar/duplic/localmachine/var/spool
You need first to know what directories to backup on your machine. A bare minimum to reconstruct the installation could be /etc and /var/lib/dpkg, then your mileage may vary. Think about twice, double check the size it takes, and the size you have one the server. How do the size vary along time? how long do you want to keep the old backups?
Here we keep 2 weeks of old backups, with a full backup every week and a daily increment.
Create the ssh and gpg keys
$ su # apt-get install duplicity keychain [...] # ssh-keygen -t dsa [...] # gpg --gen-key [...] # umask 077 # touch /root/backup.sh /root/.duplicity.conf # chmod u+x /root/backup.sh # ls -la /root/backup.sh /root/.duplicity.conf -rwx------ 1 root root 0 2006-01-16 06:47 /root/backup.sh -rw------- 1 root root 0 2006-01-16 06:47 /root/.duplicity.conf
Now you should have:
- a password protected gpg public/private key pair
localmachine:~# gpg --list-secret-keys /root/.gnupg/secring.gpg ------------------------ sec 1024D/AE45AE12 2005-08-26 [expires: 2016-08-26] uid Backup signature (localmachine) <root@localmachine> ssb 2048g/AE45AE12 2005-08-26
- an ssh private/public key pair
# ls -la /root/.ssh total 16 drwx------ 2 root root 54 2006-01-14 01:30 . drwxr-xr-x 13 root root 4096 2006-01-16 07:48 .. -rw------- 1 root root 1264 2006-01-13 21:30 id_dsa -rw-r--r-- 1 root root 1113 2006-01-13 21:30 id_dsa.pub
On the remote machine, the user used to receive the backups could be created like this:
# apt-get install scponly # adduser --disabled-password --shell /usr/bin/scponly babar # getent passwd babar babar:x:1002:1002:,,,:/home/babar:/usr/bin/scponly
Example of duplicity script file
First, the script file to run the backups: /root/backup.sh
# uncomment for debug #set -x source /root/.duplicity.conf # duplicity command DUPEXEC="duplicity --encrypt-key $ENCRKEY --sign-key $SIGNKEY $DUPOPTS $*" # loop on directories echo -n "---- Incremental backup of $HOSTNAME ---- "; date for i in $BACKDIRS do echo Starting backup of directory /$i # create directory, then backup, then erase old backups $MKDIR $LPATH/$i && $DUPEXEC /$i $RPATH/$i && $DUPEXEC $RPATH/$i # verify backup integrity #$DUPEXEC --verify $RPATH/$i /$i done # if local, fix permissions if [ -z $HOST ]; then chown -R $NAME.$NAME $LPATH; fi echo -n "---- Finished backup on $HOSTNAME ---- "; date
Example of settings
Edit and complete the file /root/.duplicity.conf (XXXX must be replaced):
# path to backup to LPATH=/home/babar/XXX/$HOSTNAME ## 1. remote settings # remote host HOST=remotehostnameXXX # remote login (user for backup on server) NAME=babarXXX # send over ssh RPATH=scp://$NAME@$HOST/$LPATH ## 2. local settings (use another disk!) # remote host *empty* #HOST= # user name to change ownership too to #NAME=babar # RPATH now uses file:// #RPATH=file://$LPATH # complete with root gpg signature and encryption key SIGNKEY=XXXXXXXX ENCRKEY=$SIGNKEY # yes, we need to store the gpg pass phrase in clear somewhere export PASSPHRASE='XXXXXXXXXXXXXXXXXXXXXXXXXXXXX' # local list of directories to backup BACKDIRS='etc var/lib/dpkg var/log var/mail usr/local/XXX' # full '/' backup setting #BACKDIRS=/ #DUPOPTS='--exclude /proc --exclude /mnt --exclude /tmp' # duplicity options (backup.sh also accepts command line arguments) DUPOPTS= # cleanup (really needs --force) DUPOPTS="$DUPOPTS --remove-older-than 2W" # exclude patterns #DUPOPTS="$DUPOPTS --exclude **/pictures/XXX" # load ssh agent info using keychain [[ -f /root/.keychain/$HOSTNAME-sh ]] && \ source /root/.keychain/$HOSTNAME-sh export GNUPGHOME=/root/.gnupg if [ -z $HOST ]; then MKDIR="mkdir -p" else MKDIR="ssh $NAME@$HOST mkdir -p" fi
Testing the script
Add the following to your localmachine:/root/.bashrc
keychain --nogui --clear id_dsa . ~/.keychain/$HOSTNAME-sh
and login as root again. You should be prompted for the ssh passphrase.
The new ssh key needs to be installed on the server (i.e. add the content of localmachine:/root/.ssh/id_dsa.pub into remotemachine:/home/babar/.ssh/authorized_keys (-rw-------) ), and the directory LPATH created.
Then try running the script:
# /root/backup.sh ---- Incremental backup of localmachine ---- Mon Jan 16 06:49:25 GMT 2006 Starting backup of directory /home/barfoo No signatures found, switching to full backup. --------------[ Backup Statistics ]-------------- StartTime 1137394351.70 (Mon Jan 16 06:52:31 2006) EndTime 1137394838.14 (Mon Jan 16 07:00:38 2006) ElapsedTime 486.44 (8 minutes 6.44 seconds) SourceFiles 26330 SourceFileSize 677134571 (646 MB) NewFiles 4185 NewFileSize 125189830 (119 MB) DeletedFiles 0 ChangedFiles 1 ChangedFileSize 2190 (2.14 KB) ChangedDeltaSize 0 (0 bytes) DeltaEntries 4186 RawDeltaSize 99926952 (95.3 MB) TotalDestinationSizeChange 45211649 (43.1 MB) Errors 0 ------------------------------------------------- No old backup sets found, nothing deleted. ---- Finished backup on localmachine ---- Mon Jan 16 07:00:57 GMT 2006
The set -x line can be uncommented to debug the script.
Saving the keys
Now you have it working, you will need to backup the key, away from the machine to backup and from the remote backup storage. Somewhere safe, such as a usb key:
$ cd /media/usbdisk $ sudo tar zcvf root-$HOSTNAME.tar.gz /root
You may also need to store the ssh passphrase close to it.
To make the remote machine safer, you should also set the script of the backup user to scponly.
Making it automagic
You can now run manualy the script with
$ sudo /root/backup.sh
(you will need to add the ssh key to keychain after a reboot).
To recreate a full backup, add the --full flag. To delete the old backups, use --force.
To add it to cron:
$ sudo crontab -e
and put something like:
# m h dom mon dow command 33 23 * * 1-6 /root/backup.sh # full backup every sunday, deleting old ones 33 23 * * 7 /root/backup.sh --full --force
More infos
For more, man duplicity, or read the [http://www.nongnu.org/duplicity/duplicity.1.html online manpage] at the [http://www.nongnu.org/duplicity/ duplicity webpage]
[http://homepage.mac.com/pallors1/73.html hottie webcam] | [http://sum.freewebsitehosting.com/36.html hardcore female stripers] | [http://karin.bravepages.com/49.html grannies pantyhose] | [http://homepage.mac.com/gismo3/23.html webcam chat hack] | [http://dissuasive.ibnsites.com/49.html gay watersport pee] | [http://gasolines.envy.nu/35.html female ejaculation movie] | [http://shitted.exactpages.com/48.html hot chat webcam] | [http://northwests.envy.nu/8.html orgy stitches video] | [http://nightstick.angelcities.com/56.html wife story gangbang] | [http://homepage.mac.com/zip17/18.html college webcam stripping] | [http://yeshivoth.150m.com/3.html housewives tits] | [http://radiosonde.g0g.net/19.html perfect pussy] | [http://homepage.mac.com/arks2/12.html webcam software gratis] | [http://homepage.mac.com/swazi2/32.html gangbang cream pies] | [http://shitted.exactpages.com/96.html young webcam boobs] | [http://sedately.1sweethost.com/67.html xxx final fantasy] | [http://homepage.mac.com/seawards/46.html girl masturbate webcam] | [http://pagodas.freewebpages.org/76.html teen chair webcam] | [http://leviers.ibnsites.com/57.html webcams sex] | [http://haricot.kogaryu.com/3.html young women masturbating] | [http://towtruck.freewebsitehosting.com/37.html blow webcam] | [http://sailings.1sweethost.com/9.html sapphic vid jo] | [http://homepage.mac.com/canola/89.html sex bare feet] | [http://homepage.mac.com/kibbutz1/78.html s bathroom webcam] | [http://saran.freewebsitehosting.com/57.html pics men pissing] | [http://forfend.freecities.com/87.html webcam dorm] | [http://headways.angelcities.com/70.html bondage handcuffs whips] | [http://homepage.mac.com/platoon1/25.html personnal webcam sites] | [http://precinct.100freemb.com/86.html fucked my grandma] | [http://sedition.1accesshost.com/44.html women breasts dick]