Backfilling

Concepts: paBackfill

What They Want to Hear 'Backfilling means reprocessing historical data, usually because a bug corrupted it or a pipeline was down. I backfill by re-running the pipeline for a specific date range. The pipeline must be idempotent so that re-running produces the same result as running once. I process one partition at a time to avoid overloading the system.' That is the answer. Backfill = re-run, idempotency = safety, partition-by-partition = control.