How to setup MySQL Replication with SSL on Ubuntu Server 10.04

This tutorial describes how to set up database replication in MySQL using an SSL connection for encryption (to make it impossible for hackers to sniff out passwords and data transferred between the master and slave). MySQL replication allows you to have an exact copy of a database from a master server on another server (slave), and all updates to the database on the master server are immediately replicated to the database on the slave server so that both databases are in sync. This is not a backup policy because an accidentally issued DELETE command will also be carried out on the slave; but replication can help protect against hardware failures however.

In this tutorial I will show how to replicate the database exampledb from the server server1.example.com (master) with the IP address 192.168.0.100 to the server server2.example.com (slave) with the IP address 192.168.0.101. Both systems are running Ububuntu Server 10.04; however, the configuration should apply to almost all distributions with little or no modifications. The database exampledb with tables and data is already existing on the master, but not on the slave.

I’m running all the steps in this tutorial with root privileges, so make sure you’re logged in as root.

To be completed on BOTH Servers (Server1 and Server2)

Firstly we should install MySQL on both servers (if it isn’t already installed, we should do this by running the following command):-

apt-get install mysql-server mysql-client

You will be asked to provide a new password for the MySQL root user and confirm it during the installation of MySQL – the password is valid for the user root@localhost as well as root@server1.example.com / root@server2.example.com, so we don’t have to specify a MySQL root password manually later on.

Once MySQL is installed on both servers we need to check to see if both servers support SSL connections so we need to login to MySQL on the console first of all..

mysql -u root -p

… and then we just need to run the following command:

show variables like ‘%ssl%’;

The output of the above command should look like this…

mysql> show variables like ‘%ssl%’;
+—————+———-+
| Variable_name | Value    |
+—————+———-+
| have_openssl  | DISABLED |
| have_ssl      | DISABLED |
| ssl_ca        |          |
| ssl_capath    |          |
| ssl_cert      |          |
| ssl_cipher    |          |
| ssl_key       |          |
+—————+———-+
7 rows in set (0.00 sec)

mysql>#

The above tells us that MySQL was compiled with SSL Support but we now just need to enable it by editing the MySQL configuration file.

Firslt exit out of the MySQL command prompt as follows:-

quit;

Now we edit the MySQL configuration file using ‘Nano’ (which is my prefered text editor)

nano /etc/mysql/my.cnf

Scroll down to the * Security Features section (within the [mysqld] section) and add a line with the word ssl to it:

[…]
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI “tinyca”.
ssl
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem
[…]

Restart the MySQL Server…

/etc/init.d/mysql restart

and the we check again if SSL is now enabled:

mysql -u root -p

show variables like ‘%ssl%’;

Output should be as follows which means that SSL is now enabled:

mysql> show variables like ‘%ssl%’;
+—————+——-+
| Variable_name | Value |
+—————+——-+
| have_openssl  | YES   |
| have_ssl      | YES   |
| ssl_ca        |       |
| ssl_capath    |       |
| ssl_cert      |       |
| ssl_cipher    |       |
| ssl_key       |       |
+—————+——-+
7 rows in set (0.00 sec)

mysql>

To leave the MySQL command prompt we now type in:-

quit;

 

Configuring The Master Server (Server1)

To make sure that the replication can work, we must make MySQL listen on all interfaces on the master (server1), therefore we comment out the line bind-address = 127.0.0.1 in /etc/mysql/my.cnf:

nano /etc/mysql/my.cnf

And comment out by adding a ‘#’ (hash) in front of the bind-address argument as follows:-

[…]
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address           = 127.0.0.1
[…]

and now we restart the MySQL server again…

/etc/init.d/mysql restart

Then we check that the MySQL server is really listening on all interfaces and accepting connections by running this command:-

netstat -tap | grep mysql

The output should look like this:-

server1:~# netstat -tap | grep mysql
tcp        0      0 *:mysql                 *:*                     LISTEN      3771/mysqld
server1:~#

Now we create the CA, server, and client certificates that we need for the SSL connections. I create these certificates in the directory /etc/mysql/newcerts which I have to create first by running this command:-

mkdir /etc/mysql/newcerts && cd /etc/mysql/newcerts

Now we make sure that we have OpenSSL installed….

apt-get install openssl

Create CA certificate:

openssl genrsa 2048 > ca-key.pem
openssl req -new -x509 -nodes -days 1000 -key ca-key.pem > ca-cert.pem

Create server certificate:

openssl req -newkey rsa:2048 -days 1000 -nodes -keyout server-key.pem > server-req.pem
openssl x509 -req -in server-req.pem -days 1000 -CA ca-cert.pem -CAkey ca-key.pem -set_serial 01 > server-cert.pem

Create client certificate:

openssl req -newkey rsa:2048 -days 1000 -nodes -keyout client-key.pem > client-req.pem
openssl x509 -req -in client-req.pem -days 1000 -CA ca-cert.pem -CAkey ca-key.pem -set_serial 01 > client-cert.pem

We now check that we have the correct number of files by running this command:-

ls -l

If you do not see the same as below (total number of files and file names) then you should attempt to recreate the certificate files (as it appears you may have missed a couple of commands out)…

server1:/etc/mysql/newcerts# ls -l
total 32
-rw-r–r– 1 root root 1346 2010-08-18 20:13 ca-cert.pem
-rw-r–r– 1 root root 1675 2010-08-18 20:13 ca-key.pem
-rw-r–r– 1 root root 1099 2010-08-18 20:14 client-cert.pem
-rw-r–r– 1 root root 1675 2010-08-18 20:14 client-key.pem
-rw-r–r– 1 root root 956 2010-08-18 20:14 client-req.pem
-rw-r–r– 1 root root 1099 2010-08-18 20:14 server-cert.pem
-rw-r–r– 1 root root 1679 2010-08-18 20:14 server-key.pem
-rw-r–r– 1 root root 956 2010-08-18 20:14 server-req.pem
server1:/etc/mysql/newcerts# 

 We must now transfer ca-cert.pem, client-cert.pem, and client-key.pem to the slave (server2); before we do this, we create the directory /etc/mysql/newcerts on server2:

Server2

mkdir /etc/mysql/newcerts

Back on server1, we can transfer the three files to server2 as follows:

Server1

Now we copy the certificate files that we need over to the slave server using SCP (Secure Copy) using the below commands:-

scp /etc/mysql/newcerts/ca-cert.pem root@192.168.0.101:/etc/mysql/newcerts

scp /etc/mysql/newcerts/client-cert.pem root@192.168.0.101:/etc/mysql/newcerts

scp /etc/mysql/newcerts/client-key.pem root@192.168.0.101:/etc/mysql/newcerts

Now we need to tell the MySQL server what certificates to use when connecting over SSL, to do this we need to edit the main MySQL server configuration file as follows:-

nano /etc/mysql/my.cnf

We need to modify the * Security Features section; uncomment the ssl-ca, ssl-cert, and ssl-key lines and fill in the correct values:

[…]
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI “tinyca”.
ssl
ssl-ca=/etc/mysql/newcerts/ca-cert.pem
ssl-cert=/etc/mysql/newcerts/server-cert.pem
ssl-key=/etc/mysql/newcerts/server-key.pem
[…]

Save the changes and then restart the MySQL server as follows:-

/etc/init.d/mysql restart

Now we set up a replication user slave_user that can be used by server2 to access the MySQL database on server1:

mysql -u root -p

On the MySQL shell, run the following commands:

GRANT REPLICATION SLAVE ON *.* TO ‘slave_user’@’%’ IDENTIFIED BY ‘slave_password’ REQUIRE SSL;

The REQUIRE SSL string is optional; if you leave it out, slave_user will be allowed to connect through encrypted and also unencrypted connections. If you use REQUIRE SSL, then only encrypted connections are allowed.

(If you’ve already set up a replication user, and now want to modify it so that it can only connect through SSL, you can modify the user as follows:

GRANT USAGE ON *.* TO ‘slave_user’@’%’ REQUIRE SSL;

Now we should flush the privileges and exit out of the MySQL command prompt:-

FLUSH PRIVILEGES;
quit;

Furthermore we have to tell MySQL for which database it should write logs (these logs are used by the slave to see what has changed on the master), which log file it should use, and we have to specify that this MySQL server is the master. We want to replicate the database exampledb, so we add/enable the following lines in /etc/mysql/my.cnf (in the [mysqld]section):

nano /etc/mysql/my.cnf

[…]
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#       other settings you may need to change.
server-id               = 1
log_bin                 = /var/log/mysql/mysql-bin.log
expire_logs_days        = 10
max_binlog_size         = 100M
binlog_do_db            = exampledb
[…]

Then as normal, save the changes and restart the MySQL server:-

/etc/init.d/mysql restart

Next we lock the exampledb database on server1, find out about the master status of server1, create an SQL dump of exampledb (that we will import into exampledb on server2 so that both databases contain the same data), and unlock the database so that it can be used again:

mysql -u root -p

On the MySQL shell, run the following commands:

USE exampledb;
FLUSH TABLES WITH READ LOCK;
SHOW MASTER STATUS;

The last command should show something like this (please write it down, we’ll need it later on):

mysql> SHOW MASTER STATUS;
+——————+———-+————–+——————+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+——————+———-+————–+——————+
| mysql-bin.000001 |       98 | exampledb    |                  |
+——————+———-+————–+——————+
1 row in set (0.00 sec)

mysql>

Now don’t leave the MySQL shell, because if you leave it, the database lock will be removed, and this is not what we want right now because we must create a database dump now. While the MySQL shell is still open, we open a second command line window where we create the SQL dump snapshot.sql and transfer it to server2 (using scp; again, make sure that the root account is enabled on server2):

cd /tmp
mysqldump -u root -pyourrootsqlpassword –opt exampledb > snapshot.sql
scp snapshot.sql root@192.168.0.101:/tmp

Afterwards, you can close the second command line window. On the first command line window, we can now unlock the database and leave the MySQL shell:

UNLOCK TABLES;
quit;

 

Configuring the slave server (server2)

Now we must configure the slave. Open /etc/mysql/my.cnf and make sure you have the following settings in the [mysqld] section:

nano /etc/mysql/my.cnf

[…]
server-id=2
master-connect-retry=60
replicate-do-db=exampledb
[…]

The value of server-id must be unique and thus different from the one on the master!
Restart MySQL afterwards:

/etc/init.d/mysql restart

Before we start setting up the replication, we create an empty database exampledb on server2:

mysql -u root -p

To create the database we run the following commands:-

 CREATE DATABASE exampledb;
quit;

On server2, we can now import the SQL dump snapshot.sql like this:

/usr/bin/mysqladmin –user=root –password=yourrootsqlpassword stop-slave
cd /tmp
mysql -u root -pyourrootsqlpassword exampledb < snapshot.sql

Now connect to MySQL again…

mysql -u root -p

… and run the following command to make server2 a slave of server1 (it is important that you replace the values in the following command with the values you got from the SHOW MASTER STATUS; command that we ran on server1!):

CHANGE MASTER TO MASTER_HOST=’192.168.0.100′, MASTER_USER=’slave_user’, MASTER_PASSWORD=’slave_password’, MASTER_LOG_FILE=’mysql-bin.000001′, MASTER_LOG_POS=98, MASTER_SSL=1, MASTER_SSL_CA = ‘/etc/mysql/newcerts/ca-cert.pem’, MASTER_SSL_CERT = ‘/etc/mysql/newcerts/client-cert.pem’, MASTER_SSL_KEY = ‘/etc/mysql/newcerts/client-key.pem’;

  • MASTER_HOST is the IP address or hostname of the master (in this example it is 192.168.0.100).
  • MASTER_USER is the user we granted replication privileges on the master.
  • MASTER_PASSWORD is the password of MASTER_USER on the master.
  • MASTER_LOG_FILE is the file MySQL gave back when you ran SHOW MASTER STATUS; on the master.
  • MASTER_LOG_POS is the position MySQL gave back when you ran SHOW MASTER STATUS; on the master.
  • MASTER_SSL makes the slave use an SSL connection to the master.
  • MASTER_SSL_CA is the path to the ca-cert.pem file on the slave.
  • MASTER_SSL_CERT is the path to the client-cert.pem file on the slave.
  • MASTER_SSL_KEY is the path to the client-key.pem file on the slave.

Then we finally start the slave…

START SLAVE;

Now we should check the slave status but running the following command:-

SHOW SLAVE STATUS \G

It is important that both Slave_IO_Running and Slave_SQL_Running have the value Yes in the output (otherwise something went wrong, and you should check your setup again and take a look at /var/log/syslog to find out about any errors); as you’re using an SSL connection now, you should also find values in the fields Master_SSL_Allowed, Master_SSL_CA_File, Master_SSL_Cert, and Master_SSL_Key:

mysql> SHOW SLAVE STATUS \G
*************************** 1. row ***************************
             Slave_IO_State: Waiting for master to send event
                Master_Host: 192.168.0.100
                Master_User: slave_user
                Master_Port: 3306
              Connect_Retry: 60
            Master_Log_File: mysql-bin.000001
        Read_Master_Log_Pos: 98
             Relay_Log_File: mysqld-relay-bin.000002
              Relay_Log_Pos: 235
      Relay_Master_Log_File: mysql-bin.000001
           Slave_IO_Running: Yes
          Slave_SQL_Running: Yes
            Replicate_Do_DB: exampledb
        Replicate_Ignore_DB:
         Replicate_Do_Table:
     Replicate_Ignore_Table:
    Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
                 Last_Errno: 0
                 Last_Error:
               Skip_Counter: 0
        Exec_Master_Log_Pos: 98
            Relay_Log_Space: 235
            Until_Condition: None
             Until_Log_File:
              Until_Log_Pos: 0
         Master_SSL_Allowed: Yes
         Master_SSL_CA_File: /etc/mysql/newcerts/ca-cert.pem
         Master_SSL_CA_Path:
            Master_SSL_Cert: /etc/mysql/newcerts/client-cert.pem
          Master_SSL_Cipher:
             Master_SSL_Key: /etc/mysql/newcerts/client-key.pem
      Seconds_Behind_Master: 0
1 row in set (0.00 sec)

mysql>

Afterwards, you can leave the MySQL shell on server2:

quit;

That’s it! Now whenever exampledb is updated on the master, all changes will be replicated to exampledb on the slave.