Every Format Imaginable

A hard Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant feedback.

Domain

Pipeline Design

Difficulty

hard

Seniority

L5

Problem

We run a health data platform that aggregates patient records from hospitals, clinics, and labs, and they arrive in every format imaginable: structured insurance claims, semi-structured HL7 hospital messages, PDF lab reports, and free-text clinical notes. Analysts need to query all of it as one canonical patient-event shape, but the raw records carry PHI, so population-health researchers must work from a de-identified view while only the clinical team can reach the raw identifiers. Hospital events feed clinical decision support and must reach it within minutes on their own low-latency serving path, separate from the shared analytical store where claims, lab PDFs, and notes can be up to a day stale. Design the data lake pipeline.

Summary

PDFs, HL7, JSON. All of it lands in the same lake.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and checks it automatically.