Greenfield Build for Six Sources

A hard Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

hard

Seniority

L5

Problem

Our company is starting fresh with Databricks as the core data platform. We have six data sources that need to be ingested, transformed, and exposed through a consistent semantic layer for business analysts. Design the end-to-end platform architecture - including infrastructure-as-code configuration for each source, the orchestration DAG, and how the semantic layer sits on top.

Summary

Infrastructure as code. Meaning as a service.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.