This month on my ‘main’ project (ZPanel) I have changed the download mirror network system, previously I wrote a PHP mirror syncronisation web app which would syncronise files to all pulic mirrors but after about 3 years of it being in use and the main repository getting rather full and slowing everything down I have decided to use rsync…
The concept here is to have a master server that stores the actual files BUT NOT used as a download server but runs rysnc to allow the download node servers to ‘pull’ the files and folders from the master server….
I thought I’d share with you how I got this configured on a Ubuntu 10.04 server…
Ok so first of all, you need Ubuntu server installed…. duhh
Next, as rysnc is already installed on Ubuntu there is no need for us to install anything etc. so lets move streight on…
The configuration is simple, we just create 2 files and edit one file and then restart the rsyncd service, so the first file we are going to create is the main rsync service configuration file…
I used nano to create the file with the following command:-
nano /etc/rsyncd.conf
Then we populate it with the following…
motd file = /etc/rsyncd.motd
[zdds]
path = /home/zpanel/mirrorfs
comment = ZPanel master mirror controller. (ZDDS)
uid = zpanel
gid = zpanel
read only = true
I hope you can work out for yourself what is happening above but just incase I’ll give you a quick breakdown, the [zdds] is the ‘module’ name that you’ll use on the remote client/server to ‘pull’ and syncronise the data, the path is the path in the file system to where the files and folders are that you want to mirror. The comment is self explainatory 🙂 – The uid and gid is the user and group for access rights, I set these to zpanel as that is the home directory in which the files are located. I have the repository read only as the download nodes will connect and ‘pull’ the data to their server which will actually be used by the public to download the files.
You can setup username and password access if you like but I don’t need it so I didnt bother but search google if you want to know how to do it!
Save the file and now create the (M)essage (O)f (T)he (D)ay file like so..
nano /etc/rsyncd.motd
Then pop a quick message in the file, for example:
Hello this is the Message of the day text on this Rsync server!
Now we just need to edit /etc/default/rsync edit and enable Rsync as a service…
So using Nano again…
nano /etc/default/rsync
and set the RSYNC_ENABLE to true and then save…
Then simply restart rsyncd by doing this…
/etc/init.d/rsyncd restart
Then now your download node servers will be able to syncronise with your master server, using the below command:
rsync -a -r –delete –delete-before example.com::zdds/ /path/to/local/folder
Hope you found this useful 🙂