# A consumer-app metrics streaming aggregation must serve a real-time dashboard with sub-minute freshn Canonical URL: Domain: Pipeline Design ยท Difficulty: medium ## Problem A consumer-app metrics streaming aggregation must serve a real-time dashboard with sub-minute freshness. The team profiled lateness for a full week: 50th percentile is 2 seconds, 95th is 30 seconds, 99th is 8 minutes, 99.9th is 4 hours. Engine state cost grows linearly with allowed lateness; a 4-hour budget is unaffordable on the current cluster. The section's framing: pick a percentile target (95th for most consumer dashboards, 99th for important reports), compute state cost at that percentile, and plan a reconciliation pass for everything past the budget. Pick the budget by setting the aggregation's allowed lateness in its name to a value that matches the 95th-or-99th percentile for this workload, and add a reconciliation pass node downstream that catches events past the budget. ## Related - [All practice problems](https://datadriven.io/problems) - [Mock interview mode](https://datadriven.io/interview/a-consumer-app-metrics-streaming-aggregation-must-serve-a-re-775b633a) - [System Design Interview Questions](https://datadriven.io/data-engineering-system-design) - [Data Engineering Interview Prep Guide](https://datadriven.io/data-engineer-interview-prep) - [Daily Challenge](https://datadriven.io/daily) --- Source: DataDriven (https://datadriven.io). 100% free data engineering interview prep. Live code execution against Postgres 16, Python 3.11, and Spark sandboxes. No paywall, no premium tier, no signup gate.