A task-based Airflow deployment runs two tasks in sequence: compute_fct_orders writes fct_orders, co

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

A task-based Airflow deployment runs two tasks in sequence: compute_fct_orders writes fct_orders, compute_mart_revenue reads fct_orders and writes mart_revenue. The data dependency is in the engineer's head; the orchestrator does not know which tables each task touches. Apply the asset-vs-task framing this section just taught and add the asset-aware lineage layer. The structural footprint is a catalog node (DataHub, Atlan, Collibra, Marquez, or OpenLineage) that records which task produces which asset; the catalog turns implicit code-level lineage into orchestrator-queryable metadata. Replace the plain Snowflake mart with a lakehouse format (Iceberg, Delta, or Hudi) so the assets carry snapshot identity that the catalog can track. The deployment becomes hybrid: tasks still execute the work, but the catalog + lakehouse seam expose lineage at the asset level.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.