Data & AI
Data migration between databases, engines, versions and systems
A data migration means moving data from one system to another without losing it, without altering it and without stopping the business longer than necessary. It may be an engine change (MySQL to PostgreSQL), a major version upgrade, a move to the cloud, the takeover of data from a legacy business application or the merge of several sources.
This service is for businesses changing tools or infrastructure that cannot afford inconsistencies: order history, customer records, accounting data, health or production data. It is also for software vendors who must migrate all of their customers' data during a redesign.
Our method relies on reproducible transformation scripts, continuous synchronization through change data capture (CDC) when downtime must be minimal, and systematic validation: row counts, checksums, sample comparison and business tests before cutover.
When do we step in?
Database engine change
You are moving from MySQL to PostgreSQL, from Oracle to PostgreSQL or from MongoDB to a relational engine. Types, encodings, sequences, constraints and SQL behavior differ and must be converted, then verified.
Taking over data from a legacy application
You are replacing a business application or an ERP and need to recover years of data, often from partial exports, Excel files or a poorly documented database, with duplicates and implicit rules.
Migration to the cloud or a new host
The database must change server or move to Amazon RDS, Aurora or Atlas. Volume and network latency make a simple export impossible within the available maintenance window.
Application redesign with a new schema
The new version of your application uses a different data model. Existing data must be transformed, enriched or cleaned to fit, without users losing their history.
Merging or consolidating sources
After an acquisition or a reorganization, several databases must be brought together. Identifiers collide, reference data diverges, and deduplication rules must be validated by business owners.
How we work
- 1
Data mapping
We inventory sources, volumes, tables or collections, relations, sensitive data and implicit business rules. This step often reveals poor quality data that must be either corrected or excluded.
- 2
Transformation rules
We write field-by-field mappings, type and encoding conversions, cleaning and deduplication rules, validated with your business referents.
- 3
Reproducible ETL scripts
Transformations are coded (Python, SQL, tools such as pgloader, AWS DMS or Debezium), versioned and replayable at will on a copy. Every run produces a report.
- 4
Dry runs
We replay the full migration on a test environment as many times as needed, measuring duration and fixing discrepancies, until the result is stable.
- 5
Validation
Row counts per table, checksums, random sample comparison, reconciliation of financial totals and functional tests of the application on the migrated data.
- 6
Cutover and monitoring
Depending on the need: short outage with a final migration, or continuous CDC synchronization followed by a zero-downtime cutover. A rollback plan is ready, and the new database is closely monitored during the first days.
Technologies we use
- pgloader
- AWS Database Migration Service
- Debezium (change data capture)
- PostgreSQL logical replication
- Python (pandas, SQLAlchemy)
- Apache Airflow
- dbt
- MongoDB Change Streams
- Talend / Pentaho
- Great Expectations
Why choose Agencei?
Validation above all
We consider a migration finished only when row counts, checksums and business tests agree. The validation report is delivered with the migration.
Downtime kept to a minimum
Thanks to change data capture and logical replication, the cutover takes minutes rather than a night of maintenance, whenever the context allows it.
Replayable and reversible
Our scripts are idempotent and replayed several times before the big day. The old system stays available until full validation.
Understanding of both sides
We know the source and target engines, but also the applications that use them. We anticipate ORM, collation or time zone incompatibilities before they show up in production.
In brief
- What is this service?
- Data migration service between databases, engines, versions, hosts or applications, with ETL scripts, change data capture, validation and cutover.
- Who is it for?
- Businesses changing software, database engine or hosting, software vendors redesigning their application, organizations consolidating several sources.
- What problem does it solve?
- Risk of data loss or corruption, excessive downtime, incompatibilities between engines, poor data quality and undocumented business rules.
- How long does it usually take?
- Preparation usually takes from a few days to several weeks depending on volume and data quality. The final cutover is short, often a few minutes to a few hours, once it has been rehearsed.
- What factors influence the price?
- Price depends on volume, number of sources and targets, transformation complexity, data quality, service continuity requirements and regulatory constraints.
- How does the engagement run?
- Mapping, transformation rules, reproducible ETL scripts, dry runs on a test environment, validation through row counts and checksums, cutover with a planned rollback.
- What are the risks?
- Lost or corrupted data without validation, badly converted encodings and time zones, colliding identifiers, application untested on migrated data, intermediate copies not deleted.
- What alternatives exist?
- Manual export-import migration for very small volumes, migration tools provided by the target software vendor, or keeping the old system in read-only mode for history.
Frequently asked questions
Can we migrate without stopping the application?
Often, yes. We load the initial data, then synchronize changes continuously (CDC, logical replication or change streams) until the cutover, which only takes a few minutes. This requires the application to tolerate a short read-only period or dual writes, which we validate with you.
How do you guarantee that no data is lost?
Through multi-level validation: row counts per table, checksums on key columns, random sample comparison, reconciliation of financial amounts and functional tests. Discrepancies are explained or fixed before cutover.
What about poor quality data?
The migration brings it to light: duplicates, impossible dates, orphan references. We document each case and you decide with us whether to fix, exclude or keep as is. We do not make those decisions for you.
How long does a data migration take?
It depends on volume, number of sources, data quality and transformations. A simple migration between two versions is prepared in a few days; a takeover from a legacy ERP with cleaning is often measured in weeks. The final cutover itself is short if it has been rehearsed.
Is personal data protected during the migration?
Yes. Test environments use anonymized or pseudonymized data whenever possible, transfers are encrypted, access is restricted and logged, and we comply with GDPR obligations, including deleting intermediate copies after validation.
Related services
Database services
Design, optimization, administration and backups for your PostgreSQL, MySQL, MongoDB and Redis databases.
See this servicePostgreSQL expertise
Schema design, optimization, replication, version upgrades and migration to PostgreSQL.
See this serviceMongoDB expertise
Document modeling, indexes, aggregations, migration and performance on MongoDB and Atlas.
See this serviceCloud Migration
Migration of applications and infrastructure to the public cloud, from lift-and-shift to re-architecture.
See this serviceServer Migration
Full transfer of a dedicated server or VPS to a new machine, with OS upgrade and a controlled cutover.
See this serviceCommon problems
Tell us about your project
Describe your need in a few lines: we come back to you with a first analysis and the next steps.