The Early Warning
A medium Pipeline Design interview practice problem on DataDriven. Design the pipeline on the canvas: sources, transforms, storage and checks, then get feedback on it.
- Domain
- Pipeline Design
- Difficulty
- medium
- Seniority
- mid
- Asked in
- Epic
Problem
A hospital network ingests millions of vital-sign and clinical events a day from bedside monitors and EHR systems. Clinicians need patient-deterioration alerts at the nursing station within seconds of a reading crossing a threshold, while the compliance and analytics teams need every event landed exactly once for the regulatory reporting that runs the next morning. Design the pipeline that serves both.
Business requirements
- When a patient's vitals cross a deterioration threshold, the charge nurse needs to know within seconds, not at the next shift change.
- Compliance reporting has to account for every clinical event exactly once; a duplicated or dropped event is an audit finding.
- The feed runs into the millions of events a day and spikes during rounds; the pipeline can't drop events under load.
Worked solution and explanation
Why this problem exists in real interviews
This looks like one pipeline but it is two consumers with opposite correctness budgets sharing a feed. Clinicians want a deterioration alert in seconds and will accept an approximate, best-effort signal; compliance wants every event counted exactly once for a report that runs tomorrow morning and does not care about latency at all. The trap is the single path that serves neither well: too slow to page a nurse, or too loose to survive an audit. The interviewer is watching whether you split the path by latency budget instead of defaulting to stream-everything.
The whiteboard reflex is to stream the whole feed into one table that both the alerting UI and the compliance report read. Alerts lag because the job is also doing the heavy exactly-once bookkeeping the report needs, and the report double-counts when the stream replays after a restart because nothing deduplicates on a stable id. Both stakeholders are unhappy in different directions, and one of them is a patient-safety problem.
Walk the requirements
Step 1: Buffer the ingest before you process anything
Monitors and EHRs emit in bursts, heaviest during morning rounds and code events. A durable queue decouples those producers from the processing tiers: ingest spikes turn into queue depth, not dropped events or backpressure on a bedside device. It also becomes your replay log when a downstream tier needs to be rebuilt.
Step 2: Put alerting on a streaming, sub-minute path
The deterioration detector reads the queue, evaluates each reading against thresholds, and emits to an alert destination the charge nurse actually sees, inside 10-15 seconds. This path is allowed to be approximate (a reading briefly in flight is fine) because the cost of slowness here is measured in patient outcomes, not in a reconciliation ticket. Nothing about exactly-once belongs on this path.
Step 3: Make the warehouse load exactly-once on a stable id
Compliance reads next-day batch from the warehouse and must reconcile to the source event count. Each event carries a stable id from the device or EHR; the load upserts on that id so a stream restart that replays from the last checkpoint cannot double-count, and gap-free checkpointing means it cannot under-count either. This is where the exactly-once complexity lives, isolated from the latency-sensitive path.
The shape that fits
- clinical_events (source, Kafka); parallelism: 24 partitions
- deterioration_detector (transform, Flink); freshness: real-time
- nurse_alerts (consumer, Alert service); freshness: < 1min
- warehouse_loader (transform, Spark); idempotency: upsert
- clinical_warehouse (storage, Snowflake); freshness: < 24h
- compliance_reporting (consumer, Tableau); freshness: < 24h
- dead_letter (storage, S3); alerting: DLQ depth above baseline
- clinical_events -> deterioration_detector
- deterioration_detector -> nurse_alerts
- deterioration_detector -> dead_letter
- clinical_events -> warehouse_loader
- warehouse_loader -> clinical_warehouse
- clinical_warehouse -> compliance_reporting
- Ingest: clinical_events
- Two paths: deterioration_detector, warehouse_loader
- Serving: nurse_alerts, dead_letter, clinical_warehouse, compliance_reporting
Common follow-up questions
- The detector goes down for ten minutes during a busy shift. What did clinicians miss, and how do you make sure it never happens silently? (Tests whether the candidate pages on detector health and treats an alerting gap as a safety incident, not just a metrics blip.)
- A device integration starts sending occasional duplicate events with the same id but a corrected value. What does the warehouse load do? (Tests the limits of dedup-on-id: an exact duplicate is absorbed, but a same-id-different-value correction needs an explicit last-write or versioning rule.)