Three Airflow DAGs all hit the same Snowflake XL warehouse: a customer-facing daily dashboard DAG, a

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

Three Airflow DAGs all hit the same Snowflake XL warehouse: a customer-facing daily dashboard DAG, a 365-day backfill DAG, and an experimental ML feature build DAG. On Mondays they collide and the daily dashboard misses its 6am SLA. Apply the pools-and-priority framing this section just taught and add structural isolation: split into two orchestrators (one running the customer-facing daily dashboard at high priority on a dedicated path, another running the backfill and ML experimental DAGs at lower priority on isolated compute). The two orchestration nodes together with an observability_tool monitoring the customer-facing path's SLA replace the implicit shared-pool contention with explicit priority isolation that the canvas can model.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.