Size the Executors
A medium Spark mock interview question on DataDriven, with an AI interviewer who asks follow-up questions and a hire or no-hire decision at the end.
- Domain
- Spark
- Difficulty
- medium
- Seniority
- L5
Interview Prompt
The recommendations team's feature job needs new executor sizing on a 50-node cluster of 128 GB, 32-core machines, each split into as many executors as its cores allow, after its 2.1 GB broadcast table sank 4 GB executors. Only 1 to 5 cores per executor are in play, and with no cluster table to read, those core counts are positions 1 through 5 of the numbered `users` rows. List the sizings whose heap can safely hold that broadcast, with cores per executor, executors per node, heap in GB, and the executor count across the whole cluster.
Summary
Too big: GC kills you. Too small: broadcast kills you.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write and run your Spark code against the tables
- Answer the interviewer's follow-up questions, then get a hire or no-hire decision