The API Drip Feed
A medium Pipeline Design mock interview question on DataDriven. Practice with AI-powered feedback, real code execution, and a hire/no-hire decision.
- Domain
- Pipeline Design
- Difficulty
- medium
- Seniority
- L5
Interview Prompt
We pull tasks, users, and projects from a third-party project management SaaS API into our analytics warehouse, but the API only serves small rate-limited pages and bursts unevenly, so ingestion has to absorb those bursts without dropping records. The rate limit won't cover pulling every table fresh each hour, so only the one tracked dashboard's task updates can ride a within-the-hour path while the rest of the tables stay on a once-a-day refresh. The vendor reshapes the API's fields every few weeks, so schema changes have to be caught before they reach the warehouse and surfaced to the team instead of silently corrupting the tables.
Summary
The API gives you 100 records at a time. You need millions.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write your pipeline design solution with real code execution
- Get instant feedback and a hire/no-hire decision