Tungsten Replicator cookbook. Advanced replication topologies made easy

I have been asked many times to provide an easy way of deploying fan-in and star schema replication schemas. So far, I have been delayed by more pressing duties.

Now the time has come. Since we are about to release a new version of Tungsten Replicator, I made the effort of putting together the steps for an easy deployment.

Recipes

The package (with downloads and svn code available at Tungsten-Replicator Toolbox) includes some juicy goodies. There are recipes to install.

• Master/slave, the classic replication topology. Nothing fancy, but with the tools mentioned in the next section, it becomes as valuable as the other topologies.
• All-masters. This is the Tungsten no-SPOF topology. Every node is a master, and every node has a direct slave service to every other node. A bit heavy on the network, but quite robust.
• Fan-in. The legendary multiple-source replication, where one slave can get instant updates from many masters.
• Star schema. The most efficient multiple-master deployment, where all the nodes are connected through a central hub. Here the trade-off is less traffic in exchange for a SPOF.

For each topology, there is a NODES_xxxx.sh file, which you need to edit, to add the list of your nodes. The nodes must be reachable by the O.S. account used for the installation, using ssh wit an authentication key (and no password).

Once you have set the list of nodes (the README file has more details), you can run the installer using the corresponding ./cookbook/install_xxxxx.sh.

Easier administration

In addition to the recipes, there are some tools that come with the package. For each topology, there is a script that shows the cluster, one that performs a simple replication test, checking that data generated in the masters reaches all the slaves, and a script that removes all replication in one go. Again, the README file has all the details.

All the scripts are written in (hopefully simple) Bash shell language. You can use them as they are, or use them as a basis to create additional administration tools.

Happy hacking!