The Cache That Ate the Cluster
A medium Spark mock interview question on DataDriven, with an AI interviewer who asks follow-up questions and a hire or no-hire decision at the end.
- Domain
- Spark
- Difficulty
- medium
- Seniority
- L5
Interview Prompt
For each user in the `transactions` table, what is their total spend, how many transactions did they make, and what percentage of everyone's combined spend do they account for? Every one of those answers reads that same table, which in production runs to hundreds of millions of rows, so we want it pulled from storage only once.
Summary
You cached 200 GB and forgot to let go.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write and run your Spark code against the tables
- Answer the interviewer's follow-up questions, then get a hire or no-hire decision