A streaming aggregation tumbles in 5-minute event-time windows with bounded out-of-orderness waterma

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

A streaming aggregation tumbles in 5-minute event-time windows with bounded out-of-orderness watermark using 60-second lag. The product team needs to catch slightly-late events (mobile retries up to an hour) but cannot hold engine state forever. The section's worked configuration is allowed lateness of 60 minutes plus a dead-letter destination for events past the budget; events on time fire once, slightly late events fire again with updated totals, very late events route to the dead-letter sink for the next-day reconciliation pass. Walk the timeline by adding two pieces to the canvas: an annotation on the aggregation transform stating the allowed-lateness budget, and a dead-letter storage destination wired from the aggregation for events past the budget.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.