Data migration has been a critical part of rolling out TrustYou CXP.
“Let’s just move the data, it’ll be easy,” said no engineer ever.
Data migration is always challenging, and our case includes two factors that make it even harder: a large volume of data and different models in the source and target platforms.
6 Tips for Data Migration
Here are six tips that have helped us navigate this process.
Tip #1: Idempotency is your ally. You won’t move your data with a single job. Design your migration process to be rerun without hassle—no previous clean-ups, no post-processing actions. Strive for idempotency.
Tip #2: Reuse UUIDs for entities that exist in both platforms. Even with a new model, some entities will have the same underlying business concept in both the source and target platforms. Reusing UUIDs makes sense for direct mapping and traceability, even if the definitions differ in some details.
Tip #3: Enable traceability—both ways. Be ready for data integrity checks and debugging. You’ll need step-by-step traceability. Sometimes, reusing UUIDs is enough; when it’s not an option, consider storing additional metadata to maintain traceability.
Tip #4: Performance matters. Don’t overlook performance just because data migration is an “internal process” invisible to users. Poor performance can lead to long data freeze periods—some data freeze is common—or drain resources that should be serving users.
Tip #5: Pay attention to data migration requirements. Even when designing a new platform from scratch, consider migration requirements from day one. Identify what you can do during modeling to simplify the process and prevent data loss.
Tip #6: Plan ahead—and execute early when feasible. Don’t leave data migration until the final stages of your rollout. Unexpected issues can cause delays. Starting early also provides valuable insights and allows for real-world testing of your new platform with actual data.
Applying these six tips will make your data migration smoother!
Give Data Migration Its Place in Your Plans
Data migration isn’t the flashiest task, but it’s one of the most important. Give it the priority it deserves from day one.
Add data migration tasks to your plan early. Estimate and allocate the capacity you need to prepare and execute them—they won’t come for free. Think carefully about who will be needed to properly design, implement, and run the migration; some of the engineers building your new solution will likely need to support it.
Never Migrate Alone!
For the most critical steps, reinforce the four-eyes principle with pairing sessions. This makes it much easier to prevent—or detect and fix—any errors during data migration in your production environment. In particular, run cross-checks to catch hidden errors that won’t be evident during execution but could cause trouble later.
As our Principal engineer, Felix Hassert, says: “Never migrate alone“.
Wishing you a smooth and successful data migration!
