Barr Moses and the Monte Carlo team named the five pillars of data observability: freshness, distrib

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

Barr Moses and the Monte Carlo team named the five pillars of data observability: freshness, distribution, volume, schema, and lineage. The pillars are not a checklist; they are a diagnostic framework. When a consumer reports a wrong number, a senior engineer walks the pillars in order and uses each to narrow or rule out a class of cause. Most quality programs cover four pillars well and lineage poorly because lineage at column granularity is expensive to keep current. Audit pillar coverage by adding five monitor nodes on the curated table, one per pillar, plus a catalog node that captures the lineage pillar (which upstream column produced which output column).

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.