A batch ETL processes 10,000 rows
A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.
- Domain
- Pipeline Design
- Difficulty
- medium
Problem
A batch ETL processes 10,000 rows. One row fails. Two extreme answers are both wrong: failing the entire batch loses progress on every good row, and silently dropping the bad row hides what may be a symptom of a larger issue. The section names three strategies: all-or-nothing (depends on idempotency), skip-and-quarantine (a percentage threshold guards the failure), and partial commit with checkpoint (resume from the last committed unit). Pick the strategy that fits this analytics workload by replacing the ingest transform's name with one that states the chosen strategy and its bounding parameter. If skip-and-quarantine is chosen, add a quarantine destination.
Practice This Problem
Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.