An ingestion pipeline hit a 10x flash-sale spike

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

An ingestion pipeline hit a 10x flash-sale spike. The producer writes 50k events/sec; the consumer drains at 5k; there is no durable queue between them. Memory grows on the consumer host until it crashes; the producer keeps writing into a closed socket; events are silently lost. The section names three backpressure strategies: buffer the spike (Kafka, Kinesis, SQS, Pub/Sub), slow the producer (rate-limit at source), or drop selectively (sample or age out). The pick depends on whether data is per-event optional or must not be lost. Place the buffer by adding a durable queue between producer and consumer whose name states the depth and the strategy for events past that depth.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.