Too Good To Be True
A hard AI Coding mock interview question on DataDriven, with an AI interviewer who asks follow-up questions and a hire or no-hire decision at the end.
- Domain
- AI Coding
- Difficulty
- hard
- Seniority
- L6
Interview Prompt
Last week's RAG eval reported recall@5 = 0.97 (up from 0.62) with no documented index or model change. Finance wants to expand the agent rollout to all enterprise tenants based on this number. The data team is suspicious. Reproduce by running tests/test_eval_sanity.py. Fix what the test surfaces and whatever else you find while tracing the data flow. Be ready to defend why the new recall number is trustworthy when the interviewer asks 'what would you check next?'
Summary
Suspiciously flawless.
How This Interview Works
- Read the task and the repository: the files, the failing check and the constraints
- Work the fix with the AI agent at your side, and run the checks until they pass
- Submit, then answer the interviewer's questions about the change you made
- Get a hire or no-hire decision with feedback on how you drove the agent