All at Once

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Seniority

L4

Problem

Our product team stores application data as nested, schemaless documents in MongoDB, but the analytics team needs it as flat tables in Redshift they can query with ordinary joins. The nightly full export takes eight hours and refreshes only once a day, while reports run against Redshift throughout the day, so design a batch pipeline that pulls just the documents that changed, reshapes them, and stages them before loading into the warehouse.

Summary

Two databases. One direction. No data left behind.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.