A 1-minute tumbling window over event_time on a streaming aggregation needs to close

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

A 1-minute tumbling window over event_time on a streaming aggregation needs to close. The wall clock has reached 09:01:25 and the window covers [09:00, 09:01). One late event with event_time 09:00:46 is still in flight in the broker. If the window closes on the wall clock the late event is dropped silently. If it never closes, no result emits. The section's compromise is the watermark: a timestamp the engine emits declaring no events with event_time earlier than the watermark will be processed against an open window. The strategy that fits this source is bounded out-of-orderness: watermark = max event_time seen minus a fixed lag. Place the watermark by adding a watermark generator node between the source and the aggregation whose name states the strategy and the lag value the engine uses.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.