A daily aggregation task on the canvas hardcodes CURRENT_DATE in its SQL and is bypassed by a custom
A medium Pipeline Design mock interview question on DataDriven. Practice with AI-powered feedback, real code execution, and a hire/no-hire decision.
- Domain
- Pipeline Design
- Difficulty
- medium
Interview Prompt
A daily aggregation task on the canvas hardcodes CURRENT_DATE in its SQL and is bypassed by a custom Python backfill loop that submits 365 queries at once. The destination is plain Snowflake INSERT INTO append. Apply the backfill framing this section just taught and make the pipeline backfillable structurally. (1) Add an orchestrator (Airflow, Dagster, or Prefect) that owns the backfill operation as a first-class scheduled job, replacing the custom Python loop. (2) Replace the plain Snowflake destination with a lakehouse format (Iceberg, Delta, or Hudi) whose ACID transactions support partition overwrite during backfill, so each historical day's run replaces only its own partition.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write your pipeline design solution with real code execution
- Get instant feedback and a hire/no-hire decision