# A streaming pipeline consumes order events from Kafka, calls a tax API, and writes to Snowflake

Canonical URL: <https://datadriven.io/problems/a-streaming-pipeline-consumes-order-events-from-kafka-calls-657b3647>

Domain: Pipeline Design · Difficulty: medium

## Problem

A streaming pipeline consumes order events from Kafka, calls a tax API, and writes to Snowflake. Three failure classes coexist: transient (a 503 once, retry succeeds), sustained (503 for 15 minutes, breaker opens), and permanent (validation failure, routed aside). The section composes the patterns: a worker that classifies errors, a retry budget around the API call, a circuit breaker around the API call (retry sits inside the breaker), and a DLQ for permanent and budget-exhausted messages. Trace the failure path by composing all four pieces on the canvas.

## Related

- [All practice problems](https://datadriven.io/problems)
- [Mock interview mode](https://datadriven.io/interview/a-streaming-pipeline-consumes-order-events-from-kafka-calls-657b3647)
- [System Design Interview Questions](https://datadriven.io/data-engineering-system-design)
- [Data Engineering Interview Prep Guide](https://datadriven.io/data-engineer-interview-prep)
- [Daily Challenge](https://datadriven.io/daily)

---

Source: DataDriven (https://datadriven.io). 100% free data engineering interview prep. Live code execution against Postgres 16, Python 3.11, and Spark sandboxes. No paywall, no premium tier, no signup gate.