A global trading platform consolidates the lesson into one architecture
A medium Pipeline Design mock interview question on DataDriven. Practice with AI-powered feedback, real code execution, and a hire/no-hire decision.
- Domain
- Pipeline Design
- Difficulty
- medium
Interview Prompt
A global trading platform consolidates the lesson into one architecture. Three event sources (mobile retail trades, institutional order flow, market-data feed) write to a shared Iceberg target. Schema changes flow through a registry on the producer side and a CDC allow-additive policy on the database side. The streaming pipeline serves real-time desks; a daily reconciliation pass serves audit consumers; a delta log quantifies the gap. Each source has a profile-matched watermark strategy. The schema-quarantine topic catches blocked destructive changes for human decision. Compose the system: schema registry between producers and topics with compatibility named; per-source watermark transforms named for their strategy and parameters; aggregation with event_time, lateness budget, dead-letter; CDC allow-additive policy with quarantine destination; reconciliation pass with daily cadence and atomic partition overwrite; delta-metric log; both real-time and audit consumers reading the shared Iceberg target.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write your pipeline design solution with real code execution
- Get instant feedback and a hire/no-hire decision