An hourly_subscription_metrics pipeline costs $48,000/month
A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.
- Domain
- Pipeline Design
- Difficulty
- medium
Problem
An hourly_subscription_metrics pipeline costs $48,000/month. The structured cost-reduction pass brings it to $4,700/month without breaking SLAs. The audit identifies four wins: full table rebuild every hour (fix with incremental + partition overwrite), wrong partition column (predicate filters on event_time but partition is on ingest_time), full pull from Stripe API every hour (fix with delta pull and rate-limit respect), and an over-frequent run cadence (consumer SLA is 2 hours, not 1). Sequence the fixes by adding a cost-pass transform on the canvas whose name states the four levers in execution order and the verification at each step (lineage check, pillar check, cost rollup).
Practice This Problem
Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.