A capital-markets data platform delivers per-symbol minute bars to algorithmic trading desks under s

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

A capital-markets data platform delivers per-symbol minute bars to algorithmic trading desks under sub-minute SLA, while a regulatory consumer needs the same bars within audit tolerance. Trade fills sometimes settle late; cross-border trades can land back-dated by a full business day. The section's worked configuration composes everything: a CDC pipeline with allow-additive policy from the settlement system, a streaming pipeline with bounded out-of-orderness watermark plus 60-minute allowed lateness on 1-minute tumbling event_time windows, a daily reconciliation pass that atomically overwrites yesterday's bars from the settlement system after it finalizes, and a delta-metric log. Compose the system: build the streaming path, the CDC path with its policy named, the reconciliation path with its cadence and window, and route both consumers to the shared Iceberg minute_bars target so the daily reconciliation is canonical for closed days while streaming owns the current day.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.