Two tables sit on the same source database
A medium Pipeline Design mock interview question on DataDriven. Practice with AI-powered feedback, real code execution, and a hire/no-hire decision.
- Domain
- Pipeline Design
- Difficulty
- medium
Interview Prompt
Two tables sit on the same source database. customers has 2 million rows that update slowly. orders has 500 million rows growing by 10 million per day. The current pipeline does a full load on both, which is fine for customers and unworkable for orders (the nightly extract has hit 4 hours). The section's spectrum: full load for small reference tables, incremental with a bookmark for large ones. Pick the load strategy by replacing each extract transform with one whose name states the strategy (full reload or bookmark-driven incremental), and add a shared bookmark-store node the incremental extract reads and writes only after a successful run.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write your pipeline design solution with real code execution
- Get instant feedback and a hire/no-hire decision