A growth team owns four new sources to ingest: an internal Postgres orders database, a Kafka clickst

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

A growth team owns four new sources to ingest: an internal Postgres orders database, a Kafka clickstream topic, an SFTP directory where a partner drops daily CSVs, and a Stripe REST API. The section names four ingestion shapes, sorted by who initiates the transfer (pipeline-pull or source-push) and what kind of artifact moves (events or bulk row sets/files): pull, push, file drop, and API. Match the source by adding an ingestion-shape annotation to each source on the canvas: each source's name should state which of the four shapes it uses and what state the pipeline must track to resume after a crash (watermark, offset, processed-file manifest, or cursor).

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.