A daily_finance_close DAG started missing its 6am SLA after the upstream stripe_to_raw DAG migrated
A medium Pipeline Design mock interview question on DataDriven. Practice with AI-powered feedback, real code execution, and a hire/no-hire decision.
- Domain
- Pipeline Design
- Difficulty
- medium
Interview Prompt
A daily_finance_close DAG started missing its 6am SLA after the upstream stripe_to_raw DAG migrated from a 30-minute cadence to a 5-minute cadence. The cross-DAG seam was time-offset and the cadence change broke the implicit contract. Apply the postmortem framing this section just taught and redesign the seam structurally. (1) Replace the plain Snowflake raw.payments with a lakehouse format (Iceberg, Delta, or Hudi) so the asset has snapshot semantics; the daily_finance_close DAG fires on the asset's snapshot freshness rather than the clock. (2) Add an observability_tool monitoring the lakehouse asset's freshness with an alert_destination routing misses to on-call. (3) Split into two orchestrators so finance-close runs on its own priority path, isolated from the backfill workloads that compete for the same warehouse. The four pieces together replace a fragile time-offset with an explicit contract.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write your pipeline design solution with real code execution
- Get instant feedback and a hire/no-hire decision