A daily_finance_close DAG started missing its 6am SLA after the upstream stripe_to_raw DAG migrated

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

A daily_finance_close DAG started missing its 6am SLA after the upstream stripe_to_raw DAG migrated from a 30-minute cadence to a 5-minute cadence. The cross-DAG seam was time-offset and the cadence change broke the implicit contract. Apply the postmortem framing this section just taught and redesign the seam structurally. (1) Replace the plain Snowflake raw.payments with a lakehouse format (Iceberg, Delta, or Hudi) so the asset has snapshot semantics; the daily_finance_close DAG fires on the asset's snapshot freshness rather than the clock. (2) Add an observability_tool monitoring the lakehouse asset's freshness with an alert_destination routing misses to on-call. (3) Split into two orchestrators so finance-close runs on its own priority path, isolated from the backfill workloads that compete for the same warehouse. The four pieces together replace a fragile time-offset with an explicit contract.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.