The Leaderboard That Costs $25K a Month

A hard Pipeline Design interview practice problem on DataDriven. Design the pipeline on the canvas: sources, transforms, storage and checks, then get feedback on it.

Domain

Pipeline Design

Difficulty

hard

Seniority

L5

Problem

Design the analytics pipeline for a gaming platform with tens of millions of active players, feeding a leaderboard of top players by score, a matchmaking dashboard, and post-game session analytics for the data science team. Today every game event is batch-processed once a day. Justify your architecture decisions.

Product wants it live. Engineering has a price tag.

Business requirements

• Product wants the leaderboard and matchmaking to feel live to players; analytics for data science can wait until tomorrow.
• Some players have manipulated their device clock to climb the leaderboard; rankings have to use server-side time, not what the device claims.
• Players under thirteen can't have their data used for personalized analytics under COPPA; their events have to be filtered before reaching personalization.

Worked solution and explanation

This is a trust-boundary problem dressed up as a latency problem. Everyone draws a streaming leaderboard and a daily batch. What separates candidates is where the clock and the compliance filter sit. If the aggregator ranks on the device's timestamp, a player who edits their clock climbs the board. If the under-13 filter is a view downstream of personalization, those events were already inside it. Put both controls at the front of the stream and every later path inherits them.

Build it in this order

Step 1: Stamp server time before anything reads the event

server_time_stamp assigns the authoritative event time as events leave game_events. The device's claimed time is kept as a field but never used for ranking. Stamping first means the aggregator and the archive both see the same trusted clock.

Step 2: Filter under-13 events before the data lands

coppa_filter sits ahead of both event_archive and live_aggregator. It is a quality gate, so under-13 events never reach a store that personalization could read. A filter placed after the archive is the version where the data was already there.

Step 3: Pay for streaming only where people watch it live

live_aggregator keeps scores and matchmaking state current in live_store, which serves real-time reads to both the players' leaderboard and the live-ops matchmaking dashboard. Session analytics reads the archive through analytics_batch on a daily cadence, because data science can wait until tomorrow.

One streaming aggregator

Ranks on device time, so clock edits climb the board. Under-13 events reach personalization unless a downstream view hides them. Analytics pays streaming compute for a question asked once a day.

Fork after stamp and filter

Ranking uses server_time_stamp, the filter runs before any store, and only the live path is always on. The archive also lets you replay history without asking game servers to resend.

Common follow-up questions

• A player turns thirteen. What lets their new events reach personalization, and what stays excluded? (Tests whether the age band is read per event at the `coppa_filter` boundary, so history stays filtered and new events pass.)
• Two players tie on score. Where does the tie-break rule live, and which timestamp decides it? (Tests that ties break on server time inside `live_aggregator`, never on the device's claimed time.)