Salt the Hot Merchant
A hard Spark mock interview question on DataDriven, with an AI interviewer who asks follow-up questions and a hire or no-hire decision at the end.
- Domain
- Spark
- Difficulty
- hard
- Seniority
- L4
Interview Prompt
`transactions` holds one payment per row keyed by `user_id`, and a single internal payout account owns 38% of them, so matching them to `users` on that key parks the whole account on one task for 3 days while the other 199 finish in seconds. Without broadcasting `users` (a downstream 2 GB enrichment step rules that out), salt the key into `SALT_BUCKETS` buckets to spread that account's rows across many tasks, and return each account's payment count and total amount.
Summary
One merchant owns 38% of your rows. Salt or suffer.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write and run your Spark code against the tables
- Answer the interviewer's follow-up questions, then get a hire or no-hire decision