The API Drip Feed

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Seniority

L5

Problem

We need to pull data from a third-party project management SaaS API - tasks, users, and projects - and land it in our data warehouse for analytics. The API doesn't push events; we have to poll it. Design a connector that keeps our warehouse in sync incrementally, handles schema changes as the vendor evolves their API, and stays within rate limits.

Summary

The API gives you 100 records at a time. You need millions.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.