Staff Data Engineer Elite Architecture
AmdFive Times the Traffic, Five Times the Bill
Our platform's data volumes are unpredictable - we see 5x swings between our quietest and busiest hours, with sudden spikes during product launches. We've been running a fixed-size Spark cluster that's over-provisioned 80% of the time and still falls behind during spikes. Design a data pipeline on AWS that handles variable volume efficiently, auto-scales without intervention, and keeps costs predictable.
Ask the interviewer clarifying questions to understand the requirements and constraints before designing.
When you're ready, click Ready to Design to start building.
Five Times the Traffic, Five Times the Bill
A hard Pipeline Design mock interview question on DataDriven. Practice with AI-powered feedback, real code execution, and a hire/no-hire decision.
- Domain
- Pipeline Design
- Difficulty
- hard
- Seniority
- staff
Interview Prompt
Our platform's data volumes are unpredictable - we see 5x swings between our quietest and busiest hours, with sudden spikes during product launches. We've been running a fixed-size Spark cluster that's over-provisioned 80% of the time and still falls behind during spikes. Design a data pipeline on AWS that handles variable volume efficiently, auto-scales without intervention, and keeps costs predictable.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write your pipeline design solution with real code execution
- Get instant feedback and a hire/no-hire decision