Nightly Exports Are Too Slow
A medium Pipeline Design mock interview question on DataDriven. Practice with AI-powered feedback, real code execution, and a hire/no-hire decision.
- Domain
- Pipeline Design
- Difficulty
- medium
- Seniority
- L5
Interview Prompt
Our healthcare analytics platform needs near-real-time access to claims and member data that lives in several operational databases. We have been using nightly full exports, but this is too slow for utilization management teams and creating performance problems on the source systems. Design a CDC-based replication pipeline using PySpark that keeps the warehouse current without impacting production.
Summary
Healthcare claims change constantly. The warehouse cannot fall behind.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write your pipeline design solution with real code execution
- Get instant feedback and a hire/no-hire decision