A streaming pipeline consumes order events from Kafka, calls a tax API, and writes to Snowflake

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

A streaming pipeline consumes order events from Kafka, calls a tax API, and writes to Snowflake. Three failure classes coexist: transient (a 503 once, retry succeeds), sustained (503 for 15 minutes, breaker opens), and permanent (validation failure, routed aside). The section composes the patterns: a worker that classifies errors, a retry budget around the API call, a circuit breaker around the API call (retry sits inside the breaker), and a DLQ for permanent and budget-exhausted messages. Trace the failure path by composing all four pieces on the canvas.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.