A daily aggregation task on the canvas hardcodes CURRENT_DATE in its SQL and is bypassed by a custom

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

A daily aggregation task on the canvas hardcodes CURRENT_DATE in its SQL and is bypassed by a custom Python backfill loop that submits 365 queries at once. The destination is plain Snowflake INSERT INTO append. Apply the backfill framing this section just taught and make the pipeline backfillable structurally. (1) Add an orchestrator (Airflow, Dagster, or Prefect) that owns the backfill operation as a first-class scheduled job, replacing the custom Python loop. (2) Replace the plain Snowflake destination with a lakehouse format (Iceberg, Delta, or Hudi) whose ACID transactions support partition overwrite during backfill, so each historical day's run replaces only its own partition.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.