Backup your Linux box with rsync
In this Linux backup series we have taken a look at Flyback ("Quick and easy backups with Flyback"), Backerupper ("Simple gui backup tool Backerupper"), and Back In Time ("Linux Back In Time: Backup made easy"). But what Linux series would be complete without a command line entry? Not this one.
There is one thing that most Linux backup tools have in common and that is their underlying technologies. In most cases one of the tools that make the GUI backup tools possible is the venerable rsync. Rsync is an incredibly fast and lightweight file copy tool that can not only copy files to and from a local machine, it can also copy over a network connection - which makes rsync an ideal candidate for user-generated backup scripts or cron jobs.
In this tutorial you will learn how easy it is to use rysnc to not only back up specified directories to an external usb drive, but also to backup over a network connection via ssh.
Command structure
The structure of the rsync command is:
rsync [OPTIONS] SOURCE DESTINATION
Where SOURCE is the location of the directory to be backed up and DESTINATION is where the backup will be placed.
Now the structure of the command changes when you are employing a network facility such as ssh. At that point the command structure would look like:
rsync [OPTIONS] ssh SOURCE user@destination:/directory
Where user is the user name on the remote machine, destination would be either an IP address or domain, and /directory is the explicit path to the directory you want to back up to.
Usage
For the first example we are going to backup the directory /home/jlwallen/Documents to the directory /media/disk/BACKUPS. This destination is a directory located on an external USB drive obviously mounted to /media/disk. The command for this backup will be:
rsync -avh /home/jlwallen/Documents /media/disk/BACKUPS
This is where we run into our first "gotcha". What happens with the above command is that any subdirectory in /home/jlwallen/Documents will be created on /media/disk/BACKUPS. So if you want to create a similar directory structure on the destination you should first create a parent directory similar to that of the source. So before you run the rsync command issue this command:
mkdir /media/disk/BACKUPS/Documents
The new rsync command would be:
rsync -avh /home/jlwallen/Documents /media/disk/BACKUPS/Documents
The options used in the above command are:
- a: Archive mode
- v: Verbose mode
- h: Output in human readable format.
Now let's backup the same source to a remote location with the help of secure shell. It will help your cause to first make sure you can log into the remove machine via ssh. Once you have that working you are ready to backup. Using our same example we are going to backup to user jlwallen at the IP address 192.168.1.10 to the directory /home/jlwallen/BACKUPS/Documents. To do this the command would look like:
rsync -avhe ssh /home/jlwallen/Documents jlwallen@192.168.1.10:/home/jlwallen/BACKUPS/Documents
The added option is e which allows you to specify the remote shell to use.
You will be prompted for the remote users' password and then the coping will begin. But what if you don't want to have to use a password? If you are wanting to set up automated, remote backups you will have to allow this process to happen without entering a password. To do this you have to create an SSH key without a password. Here are the steps for this:
create an ssh key on the source machine with the command:
ssh-keygen -t dsa
Press enter when prompted for a password.
Once the key is created copy that key to the destination key with the following command:
ssh-copy-id -i .ssh/id_dsa.pub username@destination
Where username is the user on the remote machine and destination is the IP or domain of the remote machine.
Now rsync copying can be done without having to enter a password.
Final thoughts
The nice thing about this setup is you can now use rsync to create a cron job for backup automation. Rsync is an incredibly flexible and reliable means for backing up your directories and files. It should be since it is the foundation that so many other backup tools were based on.
Advertisement
Here is a shell script that I wrote to make snapshot backups of your full filesystem with the speed of rsync.
It creates hard-links between the backups (deduplication) in order to have a full backup taking as little disk space as if it would be an incremental backup.
It also come with tuning parameters like MD5 integrity signature, ‘chattr’ protection, filter rules, restriction of disk quota, retention policy with exponential distribution (backups rotation while saving more recent backups than older).
check it here: http://blog.pointsoftware.ch/?p=527&preview=true
It’s free! ^^
Cheers
Francois Scheurer
sorry the direct URL is http://blog.pointsoftware.ch/index.php/howto-local-and-remote-snapshot-backup-using-rsync-with-hard-links/
emjoy!
francois
@David: The reason I didn’t mention the –delete option was because I find it can be mis-used. And, generally speaking, since most don’t read the man pages they won’t use the –delete option because they won’t know it’s even there. thanks for mentioning it though.
Very useful tutorial, thanks.
Perhaps it should be pointed out though, that if you delete a file from the source directories it will also be deleted from the back directories when you next rsync if you use the –delete option. It’s just a matter of reading the man page carefully so that you get what you really want, as opposed to what your really told it!
Hoi Jack a great (read) this series about Linux backup.
Is it correctly to say that between the lines your saying that you think that rsync is the best? Or is there a outher rabit comming out of your high head?