Monday, 27 July 2009

HOWTO: Backup with Amanda on 10.04 "Lucid"

I have updated this guide taking into account some changes that have been introduced into the latest versions amanda which ships with Ubuntu 10.04 "Lucid Lynx". The version at the time of writing is amanda 2.6.1p1.

Before installing amanda, your system should be able to send emails

Scenario
I have a machine setup as a server/NAS and I have another machine which runs amanda for backing up user home directories. As this is only for a couple of home users I am happy for the backup to only run once a week on a 28 day cycle (roughly 4 backups a month). The NAS server is called "callisto" and the amanda server is named "ganymede"

The user data currently sits at around 60Gb. I intend to set a maximum size of 100Gb for user data on my server therefore I will configure full weekly backups to single 100Gb virtual tapes of which there will be 5 in total. These virtual tapes will be directories on an 2Tb external HDD which is mounted on /backups.

The settings here are for my own network and are intended to be a personal reference for when I need to set things up again. Feel free to follow these steps but take note that you will need to modify ip address and disk path details to suit your own setup.

Here is the procedure;

Install packages
sudo apt-get install xinetd amanda-server amanda-client dump

Create an xinetd entry for amanda
sudo vi /etc/xinetd.d/amanda

/etc/xinetd.d/amanda
# default: on
# description: The amanda service

service amanda
{
socket_type = stream
protocol = tcp
wait = no
user = backup
group = backup
groups = yes
server = /usr/lib/amanda/amandad
server_args = -auth=bsdtcp amdump amindexd amidxtaped
disable = no
}


Restart xinetd
sudo /etc/init.d/xinetd restart


Create the main holding disk directory
mkdir -m 770 /dumps

Set file permissions;
sudo chown backup:backup /etc/amanda
sudo chown backup:backup /backups
sudo chown backup:backup /dumps
sudo chown backup:backup /etc/amandahosts


Change to backup user
sudo -u backup -s

Create the "weekly" directory where our amanda configs will be kept;
mkdir -m 770 /etc/amanda/weekly

Create amanda.conf
vi /etc/amanda/weekly/amanda.conf

/etc/amanda/weekly/amanda.conf
org "tuxnetworks.com"       # your organization name for reports
mailto "brett.glasson@gmail.com" # space separated list
dumpuser "backup" # the user to run dumps under
displayunit "g" # Possible values: "k|m|g|t"

netusage 10000 Kbps # maximum net bandwidth for Amanda, in KB per sec

dumpcycle 28 # the number of days in the normal dump cycle
runspercycle 4 # the number of amdump runs in dumpcycle days
tapecycle 5 tapes # the number of tapes in rotation
usetimestamps yes

bumpsize 20 Mb # minimum savings (threshold) to bump level 1 -> 2
bumppercent 20 # minimum savings (threshold) to bump level 1 -> 2
bumpdays 1 # minimum days at each level
bumpmult 4 # threshold = bumpsize * bumpmult^(level-1)

inparallel 8

etimeout 300 # number of seconds per filesystem for estimates.
dtimeout 1800 # number of idle seconds before a dump is aborted.
ctimeout 30 # number of seconds that amcheck waits per host

runtapes 1 # number of tapes to be used in a single run of amdump
tpchanger "chg-disk" # the tape-changer glue script

tapedev "file:/backups/weekly/slots"

changerfile "/etc/amanda/weekly/changer"

maxdumpsize -1 # Maximum number of bytes the planner will schedule

tapetype HARDDISK

define tapetype HARDDISK {
length 100 gbytes
}

amrecover_do_fsf yes
amrecover_check_label yes
amrecover_changer "changer"

holdingdisk hd1 {
comment "main holding disk"
directory "/dumps" # where the holding disk is
use 200 Gb # how much space can we use on it
chunksize 2Gb # size of chunks
}

reserve 25 # percent reserved for degraded backups

autoflush no

infofile "/etc/amanda/weekly/curinfo" # database DIRECTORY
logdir "/etc/amanda/weekly" # log directory
indexdir "/etc/amanda/weekly/index" # index directory

define dumptype global {
program "GNUTAR"
comment "Global definitions"
# exclude list "/etc/amanda/exclude.gtar"
auth "bsd"
}

define dumptype full {
global
comment "Full dump of this filesystem always"
priority medium
compress none
dumpcycle 0
index
}

define dumptype full-compress {
global
comment "Full dump of this filesystem always"
priority high
compress server fast
dumpcycle 0
}

define dumptype normal {
global
comment "partitions dumped with tar"
index
priority low
compress none
}

define dumptype normal-compress {
global
comment "dump with tar, compress with gzip"
index
priority low
compress server fast
}

define interface local {
comment "a local disk"
use 2000 kbps
}

define interface eth0 {
comment "1000 Mbps ethernet"
use 1000 kbps
}


Create the disklist file

The format for the disklist file is :
host directory dumptype
Note: We defined the dumptypes in /etc/amanda/amanda.conf

vi /etc/amanda/weekly/disklist


/etc/amanda/weekly/disklist
callisto /exports/homes normal


Virtual Tapes

Create an empty tapelist file
touch /etc/amanda/weekly/tapelist


Create the location and set permissions for the virtual tapes
mkdir -p -m 770 /backups/weekly/slots


CD to the new directory
cd /backups/weekly/slots


Create the tape directories
for ((i=1; $i<=5; i++)); do mkdir slot$i; done


Create symlink for the data directory to point to the first tape
ln -s slot1 data

Label the tapes
for ((i=1; $i<=5; i++)); do amlabel weekly weekly-0$i slot $i; done

You should see 5 tapes are labeled which looks like this:
labeling tape in slot 1 (file:/backups/weekly/slots):
Reading label...
Found an empty tape.
Writing label weekly-01..
Checking label...
Success!


We need to reset the changer back to slot 1
amtape weekly reset

Now we need to configure the client. Log on to the client "callisto.tuxnetworks.com"

ssh brettg@callisto

Install the amanda client package;

sudo apt-get install amanda-client

Edit the amandahosts file
vi /etc/amandahosts


The format of this file is;
host user
where “host” refers to the client that is to be backed up and “user” is the user account that is authorised to do the backup.

This is what my /etc/amandahosts looks like;

ganymede backup amdump
ganymede.tuxnetworks.com backup amdump
ganymede root amindexd amidxtaped
ganymede.tuxnetworks.com root amindexd amidxtaped


If you want to exclude certain files or locations from the backup you will need to create an "excludes.gtar" file which lists your exclusions. Place this file in /etc/amanda/ on the client and uncomment the appropriate line in the "Global" dump definition in your amanda.conf file.


Run amcheck on Server to verify configuration files, connections, etc.
amcheck weekly


If all went well you should see this:

backup@ganymede:/backups/weekly/slots$ amcheck weekly
Amanda Tape Server Host Check
-----------------------------
Holding disk /dumps: 485 GB disk space available, using 200 GB as requested
slot 1:read label `weekly-01', date `X'.
NOTE: skipping tape-writable test
Tape weekly-01 label ok
NOTE: host info dir /etc/amanda/weekly/curinfo/callisto does not exist
NOTE: it will be created on the next run.
NOTE: index dir /etc/amanda/weekly/index/callisto does not exist
NOTE: it will be created on the next run.
Server check took 15.763 seconds

Amanda Backup Client Hosts Check
--------------------------------
Client check: 1 host checked in 1.864 seconds. 0 problems found.

(brought to you by Amanda 2.6.1p1)


And that's it. Amanda is setup and ready to go!

Doing a manual backup (log in as backup user)
amdump weekly
You can add that command to the backup users crontab for automated backups

Restoring the backup (log in as backup user)

Change to the target directory (backup user must have write permissions here)

cd /store/tmp/restore

Select the virtual tape to restore from;

/usr/sbin/amtape weekly slot 1

Restore from the virtual tape

/usr/sbin/amrestore file:/backups/weekly/slots callisto /exports/homes

Finally, to unpack the files simply use tar

sudo tar xvf callisto._store_users.20090802.0

So there you have it, backing up user files to virtual tapes hosted on an external USB is all up and running. Go grab yourself a beer!

3 comments:

Divya said...

I am getting error while doing amcheck weekly


NOTE: skipping tape-writable test
NOTE: host info dir /etc/amanda/weekly/curinfo/192.168.0.19 does not exist
NOTE: it will be created on the next run.
NOTE: index dir /etc/amanda/weekly/index/192.168.0.19 does not exist
NOTE: it will be created on the next run.
Server check took 0.885 seconds

Amanda Backup Client Hosts Check
--------------------------------
WARNING: 192.168.0.19: selfcheck request failed: Connection refused
Client check: 1 host checked in 10.076 seconds. 1 problem found.

(brought to you by Amanda 3.3.0)

Brett said...

Divya, you should check to see if amanda is listening on your client.

netstat -a | grep amanda

Anonymous said...

amcheck dailyset
Amanda Tape Server Host Check
-----------------------------
found in slot 1: contains an empty volume
slot 1: contains an empty volume
found in slot 2: volume 'dailyset-2'
slot 2: volume 'dailyset-2'
Will write to volume 'dailyset-2' in slot 2.
NOTE: skipping tape-writable test
Server check took 0.941 seconds

Amanda Backup Client Hosts Check
--------------------------------
WARNING: ldap.orkash.net: selfcheck request failed: Connection timed out
Client check: 1 host checked in 199.370 seconds. 1 problem found.

(brought to you by Amanda 3.3.0)