I recently wanted to syncronise the contents of our backup VPS (for the ZPanel project) that we used to backup all of our servers and services that we use (MySQL databases, websites and SVN data) from multiple servers to a server sitting at my house – just for double protection as our backup VPS is donated to use free of charge :).
The solution had to be secure and be able to run automatically from a CRON job on my home server at regular intervals without the need for human interaction. I also wanted to be able to archive daily data to another part of the system (as Rysnc synchronises data, it should not be relied upon as the total backup solution!)
The solution was to use Rsync over SSH whilst using a private/public key pair for authentication (thus not requiring password prompting for SSH access)..
So here is how I did it…
Firstly we need to generate a public/private key pair on your local server (in this case, my server at home), to generate the key pair use the following command:-
ssh-keygen -t rsa -b 4096 -f $HOME/homeserver-rsync-key
The above will create a 4096 strengh RSA public/private key pair located in your home directory, hence ($HOME) called homeserver-rsync-key – obviously you can chance this is you’d like.
Now we need to copy the public key (the file in your home directory ending in .pub) to the remote host (the server which holds the data you want to backup) you wish to automate your logon to. scp (Secure copy) the file to remoteuser@remotehost:~/.ssh/authorized_keys. If you do not wish to overwrite the existing authorized_keys file on the remote host, then, copy the .pub file to a new file on the remote host and append the contents to authorized_keys with the command:-
cat homeserver-rsync-key.pub >> ~/.ssh/authorized_keys
You can now use the ‘-i’ option in the ssh command line utility to login to the remote host without having to supply a password.
ssh -i homeserver-rsync-key remoteuser@remotehost
If everything has worked correctly to this point you will be given a command line prompt without having to enter the password for the remote system. It is also recommended to add a password to the private key when it is generated. This will stop anyone from using the private key for authentication, should it fall into the wrong hands. However, because the idea behind this is to automate your rsync remote backups with cron, you will not want to enter the password every time. This situation can be resolved by using the ssh-agent utility that ships with OpenSSH. You can add the password for a private key to the ssh-agent with the ssh-add commmand. When the ssh-agent is running on the system public/private key authentication will be password protected, but passwords will not be prompted for. However, the ssh-agent and ssh-add commands must be re-entered after each reboot of the back-up server.
NOTE: This is very insecure, never distribute the private key for any reason, as it will enable anyone to gain access to your system. It is recommended to add a password to the private key and use the ssh-agent utility provided with OpenSSH.
So now you can autoamte your backups via. RSYNC using the following command: (which ofcourse is then intended to be added to the cron job – In this example the files and folders are Rsync’d remotely every hour, on the hour)
0 * * * * rsync -avz -e "ssh -i /root/homeserver-rsync-key" root@remoteserver.com:/remote/file/path/* /local/folder/to/copy/to/
On the server at home, you’ll then want to create anoter cronjob that will archive the current data into a tar.gz file (dated!) using a shell script that I have written, the contents of the crontab would like as follows (in the example the data is archived every day at midnight)…
0 0 * * * /root/makeback.sh /path/to/rsync_folder /path/to/place/to/store/backups
You can download the shell script if you’d like to use it too from here (remember to chmod +x the file before attempting to run it!).