Three teams each have a small daily pipeline (Postgres source, an orchestrator-slot transform in the

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

Three teams each have a small daily pipeline (Postgres source, an orchestrator-slot transform in the middle, a Snowflake mart, a Looker dashboard). Every row is wired correctly except the middle orchestrator slot, which carries the placeholder tech_label TBD. Each team's slot name encodes the section's selection signal for that team. Team A is a 60-engineer enterprise that already runs Cloud Composer with eighty stable DAGs. Team B is a Series A new build that wants software-defined assets and queryable lineage from the orchestrator metadata. Team C is a midsize fintech that needs workers in its own VPC with the orchestrator running as managed SaaS (a hybrid cloud-control plane). Apply the section's three selection rules and tag each orchestrator slot with the brand the section names for that signal: Airflow when an existing Airflow / Cloud Composer / MWAA deployment is in the stack; Dagster when asset lineage and software-defined assets matter; Prefect when a hybrid cloud-control plane with workers in own infrastructure is preferred. Set each orchestrator slot's tech_label to one of Airflow, Dagster, or Prefect (or for Team A, Cloud Composer or MWAA also accepted as managed-Airflow flavors named in the section). Do not add new orchestrator nodes; tag the three existing slots.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.