The Bad Row That Broke the Dashboard

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant feedback.

Domain

Pipeline Design

Difficulty

medium

Seniority

L6

Problem

We run a high-volume application event stream that has to reach Snowflake for analytics within about a minute of arrival, and past releases have let bad records into production tables and broken dashboards, so quality has to be enforced in-flight before anything lands. Records that fail validation can't be silently dropped: they need to be held aside with enough context for engineering to trace the root cause and replay them once it's fixed. Product teams reshape their event fields most weeks, so an incompatible producer change has to be rejected against a registered schema contract right at the source, before it enters the pipeline, rather than allowed to crash it or corrupt downstream tables. Design the streaming pipeline.

Summary

Bad records cannot reach the warehouse.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and checks it automatically.