Every Deal Is a Financial Transaction

A hard Pipeline Design interview practice problem on DataDriven. Design the pipeline on the canvas: sources, transforms, storage and checks, then get feedback on it.

Domain

Pipeline Design

Difficulty

hard

Seniority

L7

Problem

We host millions of real-money rummy and poker games a day in India, and every deal, bet and fold arrives as a raw event stream with no session context. A player who has sent nothing for two minutes has left the table. Design the pipeline that reconstructs complete game sessions and feeds our fraud and analytics systems.

Real money on the table. Reconstruct every hand.

Business requirements

• Chips clear within seconds of game end; fraud has to flag suspicious sessions before that happens, not after the money has moved.
• Indian gaming law treats each state's residents under different rules; data has to be partitioned by player state, not pooled globally.
• Each game produces hundreds of micro-events and analysts need them assembled into one session record; disconnected players still need their session to close.
• State regulators may request the complete record of any specific game within days of asking; missing records have cost the platform compliance fines.

Worked solution and explanation

Stateful streaming dressed as compliance

This is keyed stateful stream reconstruction wearing a gambling-law costume. The skill being probed: can you hold per-game state, close it on a timer, and hand the closed session straight to fraud? Anyone can log events and stitch sessions in a nightly job. The trap is that chips clear in seconds, so a batch flag arrives after the money moved. A player who disconnects never sends a final event, so a session with no timer never closes. And state law has to be applied where data lands, not filtered at query time.

Step 1: Key by game and close on silence

The session assembler keys its state by game_id and emits a closed session on the final event or after two minutes with no event from that game, marked as disconnected. Without the timer, state for abandoned games piles up until someone restarts the job.

Step 2: Tag the player state at ingestion

The player state router stamps every event with the player's state code before anything is stored or assembled. Pool globally and filter later, and one state's retention rule leaks into another state's records.

Step 3: Feed fraud from the assembler, not a table

The assembler streams each closed session directly to the fraud scorer. Fraud sees it within seconds of the last deal, while chips are still held. A fraud job polling a sessions table adds a delay nobody budgeted.

Step 4: Write raw events once, laid out per game

The router also lands every raw event in object storage, partitioned by state and game_id, written once and never edited. A regulator asking for one game becomes a lookup, and the archive is the only place the full record lives.

Nightly batch rebuild

Sessions exist only after the job runs. Fraud flags land after chips clear, disconnected games close whenever the batch sweeps, and state rules are applied at query time.

Keyed streaming assembly

Sessions close within seconds on the final event or the timer. Fraud reads them off the stream, and state is part of every record from the first hop.

Common follow-up questions

• An event arrives an hour late, after the timer closed its session. What happens? (Tests whether they append it to the archive so the record stays complete, and re-emit a corrected session only if the change matters downstream.)
• The same event is delivered twice after a restart. How do you keep the session correct? (Tests idempotent assembly: deduplicate on an event id within `game_id` state so replays do not inflate a session.)
• One state asks for longer retention than the rest. What changes? (Tests whether they see the state partition as the boundary: only that state's archive rule changes, not the assembler.)