A logistics company runs three pipelines (shipments hourly, inventory every 6 hours, analytics daily

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

A logistics company runs three pipelines (shipments hourly, inventory every 6 hours, analytics daily) glued by time offsets. On Black Friday a slow upstream broke the analytics output. Apply the entire L4 intermediate tier: (1) two orchestrators (one for the upstream cadence DAGs, one for the daily analytics) so each cadence runs separately; (2) replace any plain Snowflake mart between the upstream and analytics DAGs with a lakehouse format (Iceberg, Delta, or Hudi) so the analytics DAG fires on the asset's snapshot freshness rather than the clock; (3) add an observability_tool node monitoring the analytics output with an alert_destination so SLA misses route to on-call.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.