Let AQE Handle It
A medium Spark mock interview question on DataDriven, with an AI interviewer who asks follow-up questions and a hire or no-hire decision at the end.
- Domain
- Spark
- Difficulty
- medium
- Seniority
- L5
Interview Prompt
The nightly match of `transactions` against `users` on `user_id` has crept to 90 minutes against a 30-minute SLA, and the on-call engineer traced it to one merge task reading 8x the median rows on a Spark 3.4 cluster whose job config switched adaptive execution off. She wants a configuration-only fix that splits the hot slice at runtime, with the match still returning each matched transaction's `user_id`, `transaction_date` and `total_amount`.
Summary
Five tasks take 35 minutes. The other 195 take 30 seconds.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write and run your Spark code against the tables
- Answer the interviewer's follow-up questions, then get a hire or no-hire decision