A regional bank closes the books at midnight and needs the leadership meeting at 8am to read the day

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

A regional bank closes the books at midnight and needs the leadership meeting at 8am to read the day's transactions on a dashboard. The canvas already has the endpoints of the nightly batch shape: a MySQL transactions source on the left and a Looker executive dashboard on the right. Apply the canonical nightly-batch run this section just walked through and add the missing middle: a raw-zone storage node (S3, GCS, or ADLS) where the nightly extract lands as a partitioned file, a curated transform (dbt is the standard) that reads raw and writes the daily_balances table, a warehouse (Snowflake, BigQuery, Redshift, or Databricks) that holds the curated table, and an orchestrator (Airflow, Dagster, or Prefect) that wakes the job once a day at 2am Pacific. Tag the curated warehouse table with a daily slaFreshness (< 24h) so the contract to the morning consumer is named. The pipeline runs once per day; nothing on this canvas runs continuously.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.