The Skew
A medium AI Coding mock interview question on DataDriven, with an AI interviewer who asks follow-up questions and a hire or no-hire decision at the end.
- Domain
- AI Coding
- Difficulty
- medium
- Seniority
- L5
Interview Prompt
ML team's fraud model trains at 0.84 AUC offline and tanks online. We isolated it to feature `user_orders_last_24h` , offline backfills compute it correctly, but the online value at scoring time is consistently higher. Run tests/test_feature_parity.py to see the divergence on a known prediction. Find the time-leak in the offline pipeline (features/orders_24h.py) and fix it. Defend your fix: what's the upper bound on staleness you've now committed the team to?
Summary
Lab-perfect. Production-broken.
How This Interview Works
- Read the task and the repository: the files, the failing check and the constraints
- Work the fix with the AI agent at your side, and run the checks until they pass
- Submit, then answer the interviewer's questions about the change you made
- Get a hire or no-hire decision with feedback on how you drove the agent