Fix Skewed Viewing Events Pipeline
A hard Spark mock interview question on DataDriven, with an AI interviewer who asks follow-up questions and a hire or no-hire decision at the end.
- Domain
- Spark
- Difficulty
- hard
- Seniority
- L5
Interview Prompt
A streaming company's nightly `viewing_engagement` Spark job, which combines a large `event_data` table with a small `users` dimension on `user_id` to produce daily engagement counts by event type and account status, has blown its 60-minute SLA and is still running after 2 hours. Read the Spark UI evidence to find the root cause and fix the job so it meets SLA.
Summary
Your nightly Spark job just paged you. One task has 40% of the data.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write and run your Spark code against the tables
- Answer the interviewer's follow-up questions, then get a hire or no-hire decision