upr00ted

These scripts allows for remote incremental daily snapshot backups of linux, mac, and windows based servers.
The theory is that we want a full snapshot of any given day without taking up the full disk space required. The solution us to use rsync with hard links.
In addition I’ve made it backup to an encrypted disk image stored on a usb drive in OS X.
This makes it fairly portable and secure.
So far the only down-side is that the password for the encrypted drive must be stored in the script (or in another text file) as plain text. To avoid prying eyes the script has been chmod’ded 700, and is owned by root.

The setup:

1. Create a public and private key pair and distribute the public to all linux servers to be backed up, and make sure sudo rsync can be run by the local backup user without requiring a password.
2. Create an encrypted disk image using Disk Utility under Mac OS X. Ours is on a 250GB removable USB drive, but if you have access to a RAID drive with more space make it as large as you like.

Our setup creates a daily snapshot and maintains the last 30 days, rotating out the oldest to save space, adjust to the availability of your space.

4. Because rsync doesn’t run natively or easily on Windows I’m using XCOPY to backup windows servers to an intermediary Samba share on a linux box and then rsync’ing that. While not perfect, it does allow for hard-linking, and daily snapshots.

The script:
#!/bin/sh

# Tell the log file we're done:
echo "Backup Finished: "
echo "==========================================="
echo " "
date

Backing up the Windows to a Samba share is relatively straight forward. For security purposes I created Samba shares that are only accessible to one user, the account that runs the script on the Windows servers.

Samba Share:
[global]
workgroup = yourdomain
security = user
wins support = yes
[WinBackups]
path = /var/winbackups
comment = Windows Backups
public = no
write list = administrator

The Window script:
echo off
:: variables
set drive="\\linux-box\sambashare\backupfolder"
set backup1="E:\FirstLocation"
set backup2="E:\SecondLocation"
set backupcmd=xcopy /s /c /d /e /h /i /r /k /y

echo ### Backing up Second Location...
%backupcmd% %backup2% %drive%

I’ve scheduled the xcopy script to run nightly well before the rsync script, with large sets of data you might have to run them on different days.

I’ve also scheduled the rsync backup script to run nightly and send me an email detailing what it wrote to the log.
MAILTO=" email@yourdomain.com"
30 22 * * * /private/var/root/backup-scripts/backup-daily.sh 2>&1 | tee -a /private/var/root/backup-scripts/logs/backup-daily.log | mail -s "Daily Backup" email@yourdomain.com

If you wish to make a backup of your backup, simply copy the entire disk image to another drive. Make sure the image isn’t mounted when you copy it.

Creating ssh keys:

This is based on this article.

create a key pair on your backup server machine.

This section should only be done once.

ssh-keygen -t rsa -f ~/rsync-key

This creates two files public and private keys.

place the private key into /etc/ssl/private/rsync.key and secure it.
mv rsync-key /etc/ssl/private/rsync.key
chown root:root /etc/ssl/private/rsync.key
chmod 600 /etc/ssl/private/rsync.key

The public key is the file that you place in the /home/(rsync_user)/.ssh/authorized_keys file on all your clients so that they know to allow you to use rsync as that user without a password. (Note that (rsync_user) should be replaced with the user you are using to connect with so a command that looks like this: rsync -avz -e “ssh -i /etc/ssl/private/rsync.key” user@server:/files/to/backup /backup/location, you would replace it with eanders.

Before you place it on your client machines you’ll want to tell it that it should only be valid if used from your backup server and that it is only valid for the rsync command. This is security for your client not your server. Having this public key means that anyone on a machine with the private key can connect and execute stuff on your machine without a password.

You should place the following at the beginning of the rsync-key.pub file:
from="ip.of.backup.server",command="/home/(rsync_user)/.rsync/validate-rsync.sh"
The file would then look something like this:
from="10.0.100.5",command="/home/(rsync_user)/.rsync/validate-rsync.sh" ssh-rsa AAAAB3NzaC1yc2...

Repeat the following section as needed:

In most circumstances you will only be backing up information that the user you connect as has permission to. In these cases you should copy the edited rsync-key.pub file to the users home directory and place it in the .ssh/authorized_keys file. If your user was eanders you would move the file to eanders’ home directory and do something like this:

mkdir /home/(rsync-user)/.ssh
cat rsync-key.pub >> .ssh/authorized_keys
This will place the key at the end of the authorized_keys file. It is now safe to delete the key.
Finally you’ll need to place the validate-rsync script in the /home/(rsync-user)/.rsync/ directory on your client computer so that you can allow rsync to be run without a password, but nothing else will work.
You can just create the .rsync directory and then paste the following into validate-rsync.sh within that file.
mkdir /home/(rsync-user)/.rsync
vi /home/(rsync-user)/.rsync/validate-rsync.sh

#!/bin/sh
case "$SSH_ORIGINAL_COMMAND" in
*\&*)
echo "Rejected"
;;
*\;*)
echo "Rejected"
;;
rsync*)
$SSH_ORIGINAL_COMMAND
;;
*)
echo "Rejected"
;;
esac

Then we tell it that the validate-rsync.sh file should be executable
chmod 755 /home/(rsync-user)/.rsync/validate-rsync.sh

At this point you should be able to run an rsync command from your backup server as root to backup anything that (rsync-user) has permission to backup. Something like:
rsync -avz -e "ssh -i /etc/ssl/private/rsync.key" (rsync-user)@client_ip_address:/files/to/backup /backup/location

If you want to backup a machine which contains files that are only readable by root you need to do the following

This will be the case where you have a file server that hosts files which are only readable by inidividual users (root always has access) but your administrator user can’t access their files without the use of sudo.

First you will create an unprivileged user on the client machine, we are using the username rsync:
adduser rsync
Give this user whatever password you want.

Then follow the instructions above replacing (rsync-user) with the user rsync.

Next you will need to login and gain root access to the server and we’ll move the rsync application and replace it with a script.
mv /usr/bin/rsync /usr/bin/rsync_real
vi /usr/bin/rsync

Fill the file with this script:

#!/bin/sh
/usr/bin/sudo /usr/bin/rsync_real "$@"

Then make it executable:

chmod 755 /usr/bin/rsync

This tells the computer that when you ask it to run rsync it should actually run it as root, so issuing:
rsync
will result in this command actually running:
sudo rsync_real

Finally you need to tell your client computer that the (rsync-user) should be able to run rsync_real as sudo without a password since we’ll be running it without an interactive shell. To do this run, as root:
visudo
and add the following line:
(rsync-user) ALL= NOPASSWD: /usr/bin/rsync_real

For our purposes we are making a single script that will run various rsync commands to backup servers across the wan, this script will reside in /var/usb1/backup-district.sh
In addition we will rsync /var/usb1 to /var/usb2 each night with an rsync command.

If you are having problems where it is continually asking for your password (even though you’ve done all the key stuff) make sure that the line you added to visudo is at the bottom. That file appears to be processed from the top down instead of going from broad to granular.
This is especially a problem when your user is also an administrator and already has an entry in visudo.