
Glossary of Terms: Data Replication
Definition: The practices of copying information between redundant servers and keeping the copies consistent to improve reliability, fault-tolerance, or accessibility.
Reproduced from SIOS
SIOS SANless clusters High-availability Machine Learning monitoring

Definition: The practices of copying information between redundant servers and keeping the copies consistent to improve reliability, fault-tolerance, or accessibility.
Reproduced from SIOS

Chris O’Brien Lifehouse (www.mylifehouse.org.au) is an integrated and focused center of excellence specializing in state-of-the-art treatment and research for patients who are suffering from rare and complex cancer cases. Lifehouse offers everything a cancer patient might need in one place, including advanced oncology-surgery, chemotherapy, radiation therapy, clinical trials, research, education, complementary therapies and psychosocial support. Situated alongside Royal Prince Alfred Hospital and the University of Sydney in Camperdown, the not-for-profit hospital sees more than 40,000 patients annually for screening, diagnosis and treatment. As one of Australia’s largest clinical trial centers, Lifehouse also provides its patients access to the world’s latest cancer treatment breakthroughs.
Lifehouse uses the MEDITECH healthcare Electronic Medical Record and patient administration system, which stores the electronic health records for all patients in a database.
“The health information system and database are vital to the care we provide, and if either goes down, patient records would not be accessible, and that would paralyze the hospital’s operations,” explains Peter Singer, Director Information Technology at Lifehouse.
In the hospital’s datacenter, mission-critical uptime has been provided by Windows Server Failover Clustering (WSFC) running on a Storage Area Network (SAN). But like many organizations, Lifehouse wanted to migrate to the cloud to take advantage of its superior agility and affordability.
Lifehouse chose Amazon Web Services as its cloud service provider, and had hoped to “lift and shift” its environment directly to the AWS cloud. To simulate its on-premises configuration, Peter chose a “cloud volumes” service available in the AWS Marketplace. Failover clusters were configured using software defined storage volumes to share data between active and standby instances, and testing proved that the approach could provide the automatic failover needed to satisfy the hospital’s demanding recovery point and recovery time objectives.
There was a problem, however: The use of software-defined cloud volumes had a substantial adverse impact on throughput performance. With so many elements and layers involved, performance problems are notoriously difficult to troubleshoot in software defined configurations deployed in the cloud. With the “No Protection” option specified, the cloud volumes performed well. But “No Protection” was not really an option for the Chris O’Brien Lifehouse Ensures High Availability in the AWS Cloud with SIOS DataKeeper
“We were able to go from testing to production in a matter of days. Ongoing maintenance is also quite simple, which we expect will minimize our operational expenditures associated with high availability and disaster recovery,” said Peter who is responsible for mission-critical MEDITECH application and its database. “We made every reasonable effort to find and fix the root cause, and eventually concluded that software-defined storage would never be able to deliver the throughput performance we needed,” Peter recalls. So the team at Lifehouse began looking for another solution.
In its search for another solution capable of providing both high availability and high performance, Lifehouse established three criteria:
Validation was important to minimize risk associated with using a third-party solution in the cloud. The ability to work across multiple Availability Zones would assure business continuity in the event an entire AWS datacenter was impacted by a localized disaster. The sub-millisecond latency AWS delivers between Availability Zones would be critical to being able to replicate data synchronously to “hot” standby instances to meet the hospital’s demanding recovery time and recovery point objectives.
After conducting an exhaustive search, Peter concluded that the best available solution was SIOS DataKeeper Cluster Edition from SIOS Technology. SIOS DataKeeper was available on the AWS Marketplace, which assured it was proven to operate reliably in the AWS cloud. And because it did not use software-defined storage, Peter was confident SIOS DataKeeper would be able to deliver the performance Lifehouse needed.
SIOS DataKeeper provides the high-performance, synchronous data replication Lifehouse needs. By using real-time, block-level data mirroring between the local storage attached to all active and standby instances, the solution overcomes the problems caused by the lack of a SAN in the cloud, including the poor performance that often plagues software-defined storage. The resulting SANless cluster is compatible with Windows Server Failover Clustering, provides continuous monitoring for detecting failures at the application and database levels, and offers configurable policies for failover and failback.
Lifehouse currently has eight instances in SANless failover clusters to support its MEDITECH application and database across different AWS availability zones to protect against widespread disasters. The latency inherent across the long distances involved normally requires the use of asynchronous data replication to avoid delaying commits to the active instance of the database. But the real-time, block level data mirroring technology used in SIOS DataKeeper still enables Peter Singer to achieve a near-zero recovery point.
Unlike software-defined shared storage, SIOS DataKeeper is purpose-built for high performance high availability, so it came as no surprise to Peter Singer that the cloudbased configuration now works as needed. What was a bit surprising was just how easy the solution has been to implement and operate: “We were able to go from testing to production in a matter of days. Ongoing maintenance is also quite simple, which we expect will minimize our operational expenditures associated with high availability and disaster recovery.”
SIOS DataKeeper has enabled Lifehouse to take full advantage of the economies of scale afforded in the cloud without sacrificing uptime or performance. “If it were not for SIOS, we might not have been able to migrate our environment to the cloud,” Peter Singer concluded.

Two common platforms to replicate data are from the server host that operates against the data and from the storage array that holds the data.
When creating remote replicas for business continuity, the decision whether to deploy a host- or storage-based solution depends heavily on the platform that is being replicated and the business requirements for the applications that are in use. If the business demands zero impact to operations in the event of a site disaster, then host-based techniques provide the only feasible solution.
One of the two platforms to replicate data is Host-based replication. It doesn’t lock users into a particular storage array from any one vendor. SIOS SteelEye DataKeeper, for example, can replicate from any array to any array, regardless of vendor. This ability ultimately lowers costs and provides users the flexibility to choose what is right for their environment. Most host-based replication solutions can also replicate data natively over IP networks, so users don’t need to buy expensive hardware to achieve this functionality.
Host-based solutions are storage-agnostic, providing IT managers complete freedom to choose any storage that matches the needs of the enterprise. The replication software functions with any storage hardware that can be mounted to the application platform, offering heterogeneous storage support. It can operate at the block or volume level are also ideally suited for cluster configurations.
One disadvantage is that host-based solutions consume server resources and can affect overall server performance. Despite this possibility, a host-based solution might still be appropriate when IT managers need a multi-vendor storage infrastructure or have a legacy investment or internal expertise in a specific host-based application.
Another platforms to replicate data is the storage-based replication is OS-independent and adds no processing overhead. However, vendors often demand that users replicate from and to similar arrays. This requirement can be costly, especially when you use a high-performance disk at your primary site — and now must use the same at your secondary site. Also, storage-based solutions natively replicate over Fibre Channel and often require extra hardware to send data over IP networks, further increasing costs.
A storage-based alternative does provide the benefit of an integrated solution from a dedicated storage vendor. These solutions leverage the controller of the storage array as an operating platform for replication functionality. The tight integration of hardware and software gives the storage vendor unprecedented control over the replication configuration and allows for service-level guarantees that are difficult to match with alternative replication approaches. Most storage vendors have also tailored their products to complement server virtualization and use key features such as virtual machine storage failover. Some enterprises might also have a long-standing business relationship with a particular storage vendor; in such cases, a storage solution might be a relevant fit.
High quality of service comes at a cost, however. Storage-based replication invariably sets a precondition of like-to-like storage device configuration. This means that two similarly configured high-end storage arrays must be deployed to support replication functionality, increasing costs and tying the organization to one vendor’s storage solution.
This locking in to a specific storage vendor can be a drawback. Some storage vendors have compatibility restrictions within their storage-array product line, potentially making technology upgrades and data migration expensive. When investigating storage alternatives, IT managers should pay attention to the total cost of ownership: The cost of future license fees and support contracts will affect expenses in the longer term.
Cost is a key consideration, but it is affected by several factors beyond the cost of the licenses. Does the solution require dedicated hardware, or can it be used with pre-existing hardware? Will the solution require network infrastructure expansion and if so, how much? If you are using replication to place secondary copies of data on separate servers, storage, or sites, realize that this approach implies certain hardware redundancies. Replication products that provide options to redeploy existing infrastructure to meet redundant hardware requirements demand less capital outlay.
Before deciding between a host- or storage-based replication solution, carefully consider the pros and cons of each, as illustrated in the following table.
| Host-Based Replication | Storage-Based Replication | |
| Pros |
|
|
| Cons |
|
|
| Best Fit |
|
|
To understand how SIOS can work on platforms to replicate data, do read our success stories
Reproduced with permission from Linuxclustering

When you want to replicate data across multi-site or wide area network (WAN) configurations, you first need to answer one important question: Is there sufficient bandwidth to successfully replicate the partition and keep the mirror in the mirroring state as the source partition is updated throughout the day? Keeping the mirror in the mirroring state is crucial. A partition switchover is allowed only when the mirror is in the mirroring state.
Therefore, an important early step to figure out how much Bandwidth To Support Real-Time Replication is determining your network bandwidth requirements. How can you measure the rate of change—the value that indicates the amount of network bandwidth needed to replicate your data?
First, use these commands to determine the basic daily rate of change for the files or partitions that you want to mirror; for example, to measure the amount of data written in a day for /dev/sda3, run this command at the beginning of the day:
MB_START=`awk ‘/sda3 / { print $10 / 2 / 1024 }’ /proc/diskstats`
Wait for 24 hours, then run this command:
MB_END=`awk ‘/sda3 / { print $10 / 2 / 1024 }’ /proc/diskstats`
The daily rate of change, in megabytes, is then MB_END – MB_START.
The amounts of data that you can push through various network connections are as follows:
What’s next to calculate Bandwidth To Support Real-Time Replication? You’ll need to measure detailed rate of change. The best way to collect this data is to log disk write activity for some period (e.g., one day) to determine the peak disk write periods. To do so, create a cron job that will log the timestamp of the system followed by a dump of /proc/diskstats. For example, to collect disk stats every 2 minutes, add this link to /etc/crontab:
*/2 * * * * root ( date ; cat /proc/diskstats ) >> /path_to/filename.txt
Wait for the determined period (e.g., one day, one week), then disable the cron job and save the resulting /proc/diskstats output file in a safe location.
Next you should analyze the detailed rate of change data. You can use the roc-calc-diskstats utility for this task. This utility takes the /proc/diskstats output file and calculates the rate of change of the disks in the dataset. To run the utility, use this command:
# ./roc-calc-diskstats <interval> <start_time> <diskstats-data-file> [dev-list]
For example, the following dumps a summary (with per-disk peak I/O information) to the output file results.txt:
# ./roc-calc-diskstats 2m “Jul 22 16:04:01” /root/diskstats.txt sdb1,sdb2,sdc1 > results.txt
Here are sample results from the results.txt file:
Sample start time: Tue Jul 12 23:44:01 2011
Sample end time: Wed Jul 13 23:58:01 2011
Sample interval: 120s #Samples: 727 Sample length: 87240s
(Raw times from file: Tue Jul 12 23:44:01 EST 2011, Wed Jul 13 23:58:01 EST 2011)
Rate of change for devices dm-31, dm-32, dm-33, dm-4, dm-5, total
dm-31 peak:0.0 B/s (0.0 b/s) (@ Tue Jul 12 23:44:01 2011) average:0.0 B/s (0.0 b/s)
dm-32 peak:398.7 KB/s (3.1 Mb/s) (@ Wed Jul 13 19:28:01 2011) average:19.5 KB/s (156.2 Kb/s)
dm-33 peak:814.9 KB/s (6.4 Mb/s) (@ Wed Jul 13 23:58:01 2011) average:11.6 KB/s (92.9 Kb/s)
dm-4 peak:185.6 KB/s (1.4 Mb/s) (@ Wed Jul 13 15:18:01 2011) average:25.7 KB/s (205.3 Kb/s)
dm-5 peak:2.7 MB/s (21.8 Mb/s) (@ Wed Jul 13 10:18:01 2011) average:293.0 KB/s (2.3 Mb/s)
total peak:2.8 MB/s (22.5 Mb/s) (@ Wed Jul 13 10:18:01 2011) average:349.8 KB/s (2.7 Mb/s)
To help you understand your specific bandwidth needs over time, you can graph the detailed rate of change data. The following dumps graph data to results.csv (as well as dumping the summary to results.txt):
# export OUTPUT_CSV=1
# ./roc-calc-diskstats 2m “Jul 22 16:04:01” /root/diskstats.txt sdb1,sdb2,sdc1 2> results.csv > results.txt
SIOS has created a template spreadsheet, diskstats-template.xlsx, which contains sample data that you can overwrite with your data from roc-calc-diskstats. The following series of images show the process of using the spreadsheet.
The Bandwidth vs ROC graph updates. Analyze your results to determine whether you have sufficient bandwidth to support data replication.
If your Rate of Change exceeds your available bandwidth, you will need to consider some of the following points to ensure your replication solution performs optimally:
For quick how-tos like figuring Bandwidth To Support Real-Time Replication, read our blog
Reproduced with permission from Linuxclustering

The previous post introduced the advantages of running a MySQL cluster, using a shared-nothing storage configuration. We also began walking through the process of setting up the cluster, using data replication and SteelEye Protection Suite (SPS) for Linux. In this post, we complete the process to Create a 2-Node MySQL Cluster Without Shared Storage. Let’s get started.
Now it’s time to access the SteelEye LifeKeeper GUI. LifeKeeper is an integrated component of SPS for Linux. The LifeKeeper GUI is a Java-based application that can be run as a native Linux app or as an applet within a Java-enabled Web browser. (The GUI is based on Java RMI with callbacks, so hostnames must be resolvable or you might receive a Java 115 or 116 error.)
To start the GUI application, enter this command on either of the cluster nodes: /opt/LifeKeeper/bin/lkGUIapp & Or, to open the GUI applet from a Web browser, go to http://<hostname>:81.
The first step is to make sure that you have at least two TCP communication (Comm) paths between each primary server and each target server, for heartbeat redundancy. This way, the failure of one communication line won’t cause a split-brain situation. Verify the paths on the primary server. The following screenshots walk you through the process of logging into the GUI, connecting to both cluster nodes, and creating the Comm paths.
Step 1: Connect to primary server
Step 2: Connect to secondary server
Step 3: Create the Comm path
Step 4: Choose the local and remote servers
Step 5: Choose device type
Next, you are presented with a series of dialogue boxes. For each box, provide the required information and click Next to advance. (For each field in a dialogue box, you can click Help for additional information.)
Step 6: Choose IP address for local server to use for Comm path
Step 7: Choose IP address for remote server to use for Comm path
Step 8: Enter Comm path priority on local server
After entering data in all the required fields, click Create. You’ll see a message that indicates that the network Comm path was successfully created.
Step 9: Finalize Comm path creation
Click Next. If you chose multiple local IP addresses or remote servers and set the device type to TCP, then the procedure returns you to the setup wizard to create the next Comm path. When you’re finished, click Done in the final dialogue box. Repeat this process until you have defined all the Comm paths you plan to use.
Verify that the communications paths are configured properly by viewing the Server Properties dialogue box. From the GUI, select Edit > Server > Properties, and then choose the CommPaths tab. The displayed state should be ALIVE. You can also check the server icon in the right-hand primary pane of the GUI. If only one Comm path has been created, the server icon is overlayed with a yellow warning icon. A green heartbeat checkmark indicates that at least two Comm paths are configured and ALIVE.
Step 10: Review Comm path state
In the LifeKeeper GUI, create an IP resource and extend it to the secondary server by completing the following steps. This virtual IP can move between cluster nodes along with the application that depends on it. By using a virtual IP as part of your cluster configuration, you provide seamless redirection of clients upon switchover or failover of resources between cluster nodes because they continue to access the database via the same FQDN/IP.
Step 11: Create resource hierarchy
Step 12: Choose IP ARK
Enter the appropriate information for your configuration, using the following recommended values. (Click the Help button for further information.) Click Next to continue after entering the required information.
|
Field |
Tips |
| Resource Type | Choose IP Address as the resource type and click Next. |
| Switchback Type | Choose Intelligent and click Next. |
| Server | Choose the server on which the IP resource will be created. Choose your primary server and click Next. |
| IP Resource | Enter the virtual IP information and click Next.(This is an IP address that is not in use anywhere on your network. All clients will use this address to connect to the protected resources.) |
| Netmask | Enter the IP subnet mask that your TCP/IP resource will use on the target server. Any standard netmask for the class of the specific TCP/IP resource address is valid. The subnet mask, combined with the IP address, determines the subnet that the TCP/IP resource will use and should be consistent with the network configuration.This sample configuration 255.255.255.0 is used for a subnet mask on both networks. |
| Network Connection | Enters the physical Ethernet card with which the IP address interfaces. Chose the network connection that will allow your virtual IP address to be routable. Choose the correct NIC and click Next. |
| IP Resource Tag | Accept the default value and click Next. This value affects only how the IP is displayed in the GUI. The IP resource will be created on the primary server. |
LifeKeeper creates and validates your resource. After receiving the message that the resource has been created successfully, click Next.
Step 13: Review notice of successful resource creation
Now you can complete the process of extending the IP resource to the secondary server.
Step 14: Extend IP resource to secondary server
The process of extending the IP resource starts automatically after you finish creating an IP address resource and click Next. You can also start this process from an existing IP address resource, by right-clicking the active resource and selecting Extend Resource Hierarchy. Use the information in the following table to complete the procedure.
|
Field |
Recommended Entries or Notes |
| Switchback Type | Leave as intelligent and click Next. |
| Template Priority | Leave as default (1). |
| Target Priority | Leave as default (10). |
| Network Interface | This is the physical Ethernet card with which the IP address interfaces. Choose the network connection that will allow your virtual IP address to be routable. The correct physical NIC should be selected by default. Verify and then click Next. |
| IP Resource Tag | Leave as default. |
| Target Restore Mode | Choose Enable and click Next. |
| Target Local Recovery | Choose Yes to enable local recovery for the SQL resource on the target server. |
| Backup Priority | Accept the default value. |
After receiving the message that the hierarchy extension operation is complete, click Finish and then click Done.
Your IP resource (example: 192.168.197.151) is now fully protected and can float between cluster nodes, as needed. In the LifeKeeper GUI, you can see that the IP resource is listed as Active on the primary cluster node and Standby on the secondary cluster node.
Step 15: Review IP resource state on primary and secondary nodes
Halfway to Create a 2-Node MySQL Cluster Without Shared Storage! You’re ready to set up and configure the data replication resource, which you’ll use to synchronize MySQL data between cluster nodes. For this example, the data to replicate is in the /var/lib/mysql partition on the primary cluster node. The source volume must be mounted on the primary server, the target volume must not be mounted on the secondary server, and the target volume size must be equal to or larger than the source volume size.
The following screenshots illustrate the next series of steps.
Step 16: Create resource hierarchy
Step 17: Choose Data Replication ARK
Use these values in the Data Replication wizard.
|
Field |
Recommended Entries or Notes |
| Switchback Type | Choose Intelligent. |
| Server | Choose LinuxPrimary (the primary cluster node or mirror source). |
| Hierarchy Type | Choose Replicate Existing Filesystem. |
| Existing Mount Point | Choose the mounted partition to replicate; in this example, /var/lib/mysql. |
| Data Replication Resource Tag | Leave as default. |
| File System Resource Tag | Leave as default. |
| Bitmap File | Leave as default. |
| Enable Asynchronous Replication | Leave as default (Yes). |
Click Next to begin the creation of the data replication resource hierarchy. The GUI will display the following message.
Step 18: Begin creation of Data Replication resource
Click Next to begin the process of extending the data replication resource. Accept all default settings. When asked for a target disk, choose the free partition on your target server that you created earlier in this process. Make sure to choose a partition that is as large as or larger than the source volume and that is not mounted on the target system.
Step 19: Begin extension of Data Replication resource
Eventually, you are prompted to choose the network over which you want the replication to take place. In general, separating your user and application traffic from your replication traffic is best practice. This sample configuration has two separate network interfaces, our “public NIC” on the 192.168.197.X subnet and a “private/backend NIC” on the 192.168.198.X subnet. We will configure replication to go over the back-end network 192.168.198.X, so that user and application traffic is not competing with replication.
Step 20: Choose network for replication traffic
Click Next to continue through the wizard. Upon completion, your resource hierarchy will look like this:
Step 21: Review Data Replication resource hierarchy
You need to create a MySQL resource to protect the MySQL database and make it highly available between cluster nodes. At this point, MySQL must be running on the primary server but not running on the secondary server.
From the GUI toolbar, click Create Resource Hierarchy. Select MySQL Database and click Next. Proceed through the Resource Creation wizard, providing the following values.
|
Field |
Recommended Entries or Notes |
| Switchback Type | Choose Intelligent. |
| Server | Choose LinuxPrimary (primary cluster node). |
| Location of my.cnf | Enter /var/lib/mysql. (Earlier in the MySQL configuration process, you created a my.cnf file in this directory.) |
| Location of MySQL executables | Leave as default (/usr/bin) because you’re using a standard MySQL install/configuration in this example. |
| Database tag | Leave as default. |
Click Create to define the MySQL resource hierarchy on the primary server. Click Next to extend the file system resource to the secondary server. In the Extend wizard, choose Accept Defaults. Click Finish to exit the Extend wizard. Your resource hierarchy should look like this:
Step 22: Review MySQL resource hierarchy
Next, you’ll configure MySQL to depend on a virtual IP (192.168.197.151) so that the IP address follows the MySQL database as it moves.
From the GUI toolbar, right-click the mysql resource. Choose Create Dependency from the context menu. In the Child Resource Tag drop-down menu, choose ip-192.168.197.151. Click Next, click Create Dependency, and then click Done. Your resource hierarchy should now look like this:
Step 23: Review MySQL IP resource hierarchy
At this point in the evaluation, you’ve fully protected MySQL and its dependent resources (IP addresses and replicated storage). Test your environment, and you’re ready to go.
You can find much more information and detailed steps for every stage of the evaluation process in the SIOS SteelEye Protection Suite for Linux MySQL with Data Replication Evaluation Guide. To download an evaluation copy of SPS for Linux, visit the SIOS website or contact SIOS at info@us.sios.com.
Interested to learn to Create a 2-Node MySQL Cluster Without Shared Storage, here’s our past success stories with satisfied clients.
Reproduced with permission from Linuxclustering