Loading section...

Allowed Lateness and Side Outputs

Watermarks close windows, but late events still arrive. Allowed lateness keeps windows open for a grace period after the watermark passes. Events arriving within the grace period update the window result. Events arriving after the grace period are routed to a side output. The Three-Tier Late Data Strategy Side Output Architecture Batch Reconciliation Side output events are cold storage. A daily batch job reads the side output, groups by the original window, and merges corrections into the aggregate tables. This gives eventual correctness: streaming provides fast, mostly-correct results; batch provides slow, fully-correct results. This is the Lambda Architecture pattern, modernized. The strong-hire framing: 'I would serve streaming results for real-time use cases and batch-reconciled result