data aws migration claude-curated
AWS Database Migration Service migrates data between Databases — both one-shot loads and ongoing replication. Source and target don’t have to be the same engine.
Components
- Replication instance — EC2 instance that runs the migration. Sized by source data volume and CDC throughput.
- Source endpoint — connection details for the source database.
- Target endpoint — connection details for the target.
- Replication task — defines what to migrate (which schemas/tables) and how.
The instance reads from source, applies to target. It’s stateless — you can recreate it from the task definition.
Migration types
| Type | What it does |
|---|---|
| Full Load | One-shot bulk copy of existing data. Stops when done. |
| Full Load + CDC | Bulk copy, then stream ongoing changes from the source’s transaction log. |
| CDC only | Skip the bulk copy (already done somehow). Just replicate ongoing changes. |
Full Load + CDC is the common pattern for zero-downtime migrations: bulk-copy historical data, replicate ongoing writes to keep target in sync, cut over when target catches up.
Homogeneous vs heterogeneous
- Homogeneous — same engine on both sides (PostgreSQL → PostgreSQL, MySQL → MySQL). DMS handles types, indexes, sequences mostly transparently. For RDS/Aurora homogeneous moves, native tools (
pg_dump+ logical replication) are often simpler than DMS. - Heterogeneous — different engines (Oracle → PostgreSQL). DMS converts the data; schema conversion is a separate concern. Use AWS Schema Conversion Tool (SCT) to translate procedures, types, etc.
CDC under the hood
DMS reads source-specific change streams:
- PostgreSQL: logical replication slots (requires
wal_level = logical) - MySQL: binary logs (requires
binlog_format = ROW) - Oracle: LogMiner or Oracle Binary Reader
- SQL Server: MS-CDC or transaction log
If the source can’t expose its log (or the log is rotated before DMS reads it), CDC fails. Check log retention before starting.
Common gotchas
- Views, materialised views, sequences, stored procedures — DMS only migrates data. Schema objects need to be created on the target separately, ideally before DMS starts. Missing dependencies show up as opaque errors mid-load.
- Large objects (LOBs) — DMS truncates LOBs by default. Set Limited LOB mode with a generous max size, or Full LOB mode (slower).
- Foreign keys & triggers — disable on target during full load, re-enable for CDC. DMS can do this with
target_applysettings. - Type mismatches — heterogeneous migrations silently truncate or coerce. Check schema-conversion warnings carefully.
- CDC lag — if target is slower than source’s write rate, replication falls behind permanently. Size the target and replication instance for peak source throughput, not average.
- Identity / sequence values — after CDC cutover, sequences on the target need to be advanced past the highest value migrated, or new inserts collide.
Cutover playbook (Full Load + CDC)
- Pre-create schema on target (DDL only, no data).
- Start DMS task — Full Load runs.
- CDC starts after Full Load finishes; tail of source writes replicates to target.
- Monitor CDC lag — wait for it to be ~0.
- Stop application writes to source (small downtime window).
- Verify CDC has caught up.
- Stop DMS task.
- Reconfigure application to point at target.
- Resume writes.
Validation
DMS has built-in data validation — compares row counts and content between source and target. Enable it; otherwise you find out about silent corruption months later.