Adding Is Safe, Renaming Is Not

Concepts covered: Schema Evolution

3 classes of schema change

The compatibility test judges a single change. A proposal also needs a verdict on its plan: can it ship today, or does it need a schedule agreed with other teams? Each proposal falls into 1 of 3 classes.

A rename looks like a delete to every reader

A rename feels small to the team making it. To every reader it is 2 changes at once: a field they rely on disappears, and an unknown field appears. Suppose the trip service adopts a naming standard and renames fare_usd to fare_amount_usd in a release at 15:00 on March 10. The loader runs in ignore mode, so it skips the unknown name and stores null in fare_usd. Every load succeeds.

SELECT
  date(completed_at) AS trip_date,
  sum(fare_usd) AS revenue_usd
FROM trips
WHERE date(completed_at) BETWEEN DATE '2026-03-08' AND DATE '2026-03-11'
GROUP BY date(completed_at)
ORDER BY trip_date
trip_daterevenue_usd
2026-03-08612,400
2026-03-09598,900
2026-03-10214,700
2026-03-11null

March 10 holds only the fares from before 15:00, and March 11 sums nothing but nulls, which is null. Revenue appears to fall by almost two thirds and then vanish. No run fails, and a finance analyst is the first to notice. A reject loader would stop on March 10 instead. That is the better failure because it is louder, but readers break either way, so a single-step rename is destructive.

Which of these proposals is coordinated?

The Safe Rename: Add, Copy, Switch, Drop

A coordinated rename reaches the same end state in 4 steps, and nothing a reader uses disappears until that reader has stopped using it.

StepProducer writesReaders useWhy nothing breaks
1. AddBoth `fare_usd` and `fare_amount_usd``fare_usd`A new field passes both compatibility directions
2. CopyBoth`fare_usd`Rows from before step 1 get `fare_amount_usd` filled from `fare_usd`
3. SwitchBoth`fare_amount_usd`, a reader at a timeBoth names hold the same values, so each reader moves on its own schedule
4. Drop`fare_amount_usd` only`fare_amount_usd`No reader is left on the old name
SELECT
  count(*) AS rows_missing_new_name
FROM trips
WHERE fare_amount_usd IS NULL
  AND fare_usd IS NOT NULL

Run the steps out of order and readers break. Drop before switch, and any reader still on fare_usd reads nulls: the March 11 dashboard again. Switch before the copy finishes, and a switched reader sees nulls for every row older than step 1. A type change uses the same steps under a new name: to store fares as whole cents, add fare_cents, copy, switch, then drop the old field.

Do
  • Ship new fields with a default so they pass both directions
  • Treat every rename as add, copy, switch, drop
  • List every reader of a field before removing it
Avoid
  • Rename or retype a field in a single release
  • Drop the old field because the producer stopped writing it
  • Trust ignore mode during a rename: it hides the break

A team renaming city to city_name finishes the add and copy steps, then removes city from the producer and the table the next morning. What happens?

About this lesson

This section is part of the Schema Drift: Beginner lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.