One Airflow instance owns a 60-task mega-DAG spanning three teams: ingestion, analytics-engineering

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

One Airflow instance owns a 60-task mega-DAG spanning three teams: ingestion, analytics-engineering curation, and ML feature extraction. When the analytics dbt step fails, the unrelated ML feature extraction halts and pages cross team lines all night. Apply the section's split-DAG framing and split the orchestrator along ownership boundaries so a failure in one team's branch does not halt unrelated work in another.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.