The Box That Won't Fit the Data
A hard Pipeline Design mock interview question on DataDriven. Practice with AI-powered feedback, real code execution, and a hire/no-hire decision.
- Domain
- Pipeline Design
- Difficulty
- hard
- Seniority
- senior
Interview Prompt
Your nightly Spark job rolls a 100GB event export up to per-account daily totals, but the only box it runs on has 5GB of RAM and no cluster to fall back on. Land those totals durably in the local data lake without the job dying when the 100GB refuses to fit in memory.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write your pipeline design solution with real code execution
- Get instant feedback and a hire/no-hire decision