Score It Before It Clears

A hard Pipeline Design interview practice problem on DataDriven. Design the pipeline on the canvas: sources, transforms, storage and checks, then get feedback on it.

Domain

Pipeline Design

Difficulty

hard

Seniority

L5

Problem

Card transactions hit the payment gateway at roughly 10,000 per second, and each one is scored for fraud by a scorer the gateway must never wait on. Design the pipeline that scores them and keeps the fraud model learning.

The fraudsters move fast. Your pipeline has to move faster.

Business requirements

• Card transactions clear quickly and the fraud decision has to come back before the transaction clears, not after.
• If the scoring service is unavailable, the business chose to approve and review later rather than block all payments; that policy has to be implemented.
• Chargebacks confirm whether transactions were actually fraud weeks later; without that loop the model never learns from real outcomes.

Worked solution and explanation

What this really is

This is a deadline contract between two services that must never share a failure, dressed up as fraud detection. Anyone can draw a model behind an API. What separates candidates is who owns the clock. If the gateway waits on the scorer, the scorer's worst day becomes the payment platform's worst day. At 10,000 transactions per second, a two-second stall leaves 20,000 hung authorizations. The second trap is quieter: no chargeback loop. The model then retrains on its own guesses and stops improving.

Walk the requirements

Step 1: Put a log between gateway and scorer

Kafka absorbs the 10k/s firehose, and Flink reads from it, scores each transaction and returns a decision inside the authorization budget. The log is what lets the two sides fail independently: the gateway only ever writes to it and never calls the scorer directly.

Step 2: Draw approve-on-timeout as its own path

Two error paths leave the gateway, not the scorer. One goes straight to the authorization decision: no score in time means approve. The other goes to the review queue so an analyst looks later. The business chose approve-and-review, so on the canvas it is visible edges with destinations, not a timeout someone added during an incident.

Step 3: Join predictions to chargebacks before retraining

The scorer writes every decision to the labels store keyed by transaction id. Chargebacks land weeks later against the same key. Retraining reads that join, so it learns from confirmed fraud, and the new model ships back to the scorer.

Synchronous call

Gateway blocks on the scorer. A slow model means slow payments, and an outage means either declines or an improvised approve-on-error.

Queue plus deadline

Gateway publishes and waits only up to its budget, then approves itself. A scorer outage costs fraud coverage for a while and never costs availability.

Common follow-up questions

• A chargeback is reversed after dispute. How does the labels store reflect it? (Labels as latest-state per transaction id, so retraining never reads stale fraud labels.)
• Kafka redelivers a transaction after a scorer restart. What stops a double decision? (Idempotent writes keyed by transaction id and Flink exactly-once checkpoints.)