dbShards/Migrate is an exciting new offering in the dbShards Suite, supporting data-center to cloud, cloud-to-cloud and Disaster Recovery sites for critical databases. With dbShards/Migrate, users can migrate databases of any size between environments with near-zero downtime. This means no more major service interruptions when changing cloud or database vendors.
“dbShards/Migrate can move our database to a new environment without having any downtime while that move is occurring. This gives us the freedom we need to use the cloud environments that will best serve our customers.”
Kim Albee, CEO, Genoo LLC
Read the full case study here.
Migrating a large database to a new hosting environment is hard. At the core of the process is the need to dump the database from the source environment, transfer it to the new environment, and finally perform a load into the new database. With databases in the 10s to 100s of GBs it is challenging enough – when terabyte-size databases need to be moved, the task can be truly daunting.
Combine these challenges with the fact that most applications are 24X7 “always-on”, and it’s obvious that the process must be smooth, reliable and able to be performed without incurring major service outages. From our experience with many customers, extended outages cost more than just revenue – they damage reputation, often at the cost of end-users failing to return to the site or service.
Despite these obstacles, having the flexibility to migrate your database is crucial to long-term success and stability. For example, you might want to migrate your database in these circumstances:
- Changing between Cloud vendors.
- Move a database from a private data-center environment to the Cloud.
- Setting up a live Disaster Recovery database environment in a secondary Cloud or hosting environment.
- Geographic replication of your database across regions.
Here is a list of common problems and questions you may run into when you need to migrate your database:
- How to reliably migrate your database from one data-center/cloud to another?
- How to avoid downtime when migrating your database?
- How to test your database in the target migration environment?
- How to maintain a Disaster Recovery replica of your database in the Cloud?
- How to migrate your database from a locked-down Database-as-a-Service provider?
To answer these questions and meet the challenge, the underlying technology must be reliable, capable of recovering from network outages (common between hosting environments), and fast.
In short, finding a reliable database migration solution is highly important for fast-moving organizations with large “always-on” databases.
How does dbShards/Migrate work?
This diagram shows the dbShards/Migrate components and basic flow:
In the above diagram, you can see there are two Clouds: Cloud 1 (Source Cloud) and Cloud 2 (Target Cloud). The objective is to migrate the database from Cloud 1 to Cloud 2, without incurring extensive risk or damaging service outages.
The core enabling technology for dbShards/Migrate is our patent-pending Continuous Replication technology. With Continuous Replication, database replication is managed outside of the DBMS engine, supporting highly available transaction replication with excellent performance. The Continuous Replication engine has built-in High Availability support (HA), including tolerance for network outages during the migration process.
The process starts by installing dbShards/Migrate in both the Source and Target Clouds. dbShards/Migrate consists of two main components:
- dbShards/Client: The dbShards/Client is plug-compatible with your native database driver, including support for both native or Java applications. dbShards/Client is an intelligent client, and participates in the Continuous Replication process.
- dbShards/Replicate Agent: The agent controls reliable replication via the Continuous Replication method, the key to successful, smooth database migration. Each dbShards/Replicate instance maintains it’s own reliable transaction log, outside the DBMS engine. This means that dbShards/Migrate can function even when native database vendor replication is not exposed (such as a Database as a Service vendor).
Once the components are configured, reliable replication of all database write transactions can be started. When an application sends transactions to dbShards/Client, the rest is automatic, and all transactions are reliably logged in both the Source and Target Clouds.
After replication is started and transactions are flowing, a point-in-time dump of the Source Cloud database is performed. The dump can be created with any tool in use by the customer (generally the standard DBMS engine dump utility). dbShards/Migrate automatically manages the exact point-in-time transaction for the dump. Once the dump is complete, it is transferred to the Target Cloud and loaded into the database in that environment. During this entire process, dbShards/Migrate is reliably replicating all transactions, so even if this phase takes hours or days, the process maintains reliability.
Next, dbShards/Migrate applies all transactions that occurred during the Dump/Load process. The transactions continue as new transactions are added to the Source Cloud.
At the conclusion of the dbShards/Migrate process, the Target Cloud database is in synch, and the application can safely be moved from the Source Cloud to the Target Cloud. Customers can continue using dbShards in the Target Cloud for HA and other capabilities, or may uninstall dbShards to run with the native DBMS vendor configuration.
Because the process is reliable, you can perform the migration as many times as needed, allowing you to fully test the Target Cloud environment.
dbShards/Migrate has been used by customers to move databases data-center to cloud, cloud to cloud and cloud to data-center. In addition to customer usage, our support team tested moving the dbShards/Angry Shards database with continuous live transactions, from Amazon’s RDS service to Google’s Cloud SQL.
The result of using dbShards/Migrate includes:
- Seamlessly migrate your database from one data-center/cloud to another
- Near-zero downtime during the migrate process
- Typically <1 minute when switching to new environment
- Avoid vendor lock-in
- Cloud or Database-as-a-Service (DaaS)
- Perform your migration for test purposes as often as you like
- Test out your new environment or vendor to be sure you are satisfied
- Maintain a Disaster Recovery site in the Cloud
- Continuously replicate your data around the globe
Contact us for more information so we can discuss your specific requirements.
To receive more information on dbShards/Migrate please fill out the following form:
Google Cloud SQL is a trademark of Google Inc. Amazon is a trademark of Amazon, Inc. or its affiliates.