A growth team is launching a new analytics platform that needs to ingest from all four shapes at onc
A medium Pipeline Design mock interview question on DataDriven. Practice with AI-powered feedback, real code execution, and a hire/no-hire decision.
- Domain
- Pipeline Design
- Difficulty
- medium
Interview Prompt
A growth team is launching a new analytics platform that needs to ingest from all four shapes at once: a Postgres orders database (pull), a Kafka clickstream topic (push), an SFTP partner directory (file drop), and the Stripe API (API). Every source must land in a single Iceberg raw zone, each with the right resume state and the right defensive checks. Assemble the multi-source ingestion: incremental pull with bookmark on Postgres, write-first-commit-after Kafka consumer with webhook buffer, manifest plus validation plus processed-prefix on SFTP, and cursor-based paginated API ingest with rate limiter, OAuth refresh, and 429/5xx backoff.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write your pipeline design solution with real code execution
- Get instant feedback and a hire/no-hire decision