Loading...
Streaming and Batch Unified Pipeline on Azure Databricks
A hard Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.
- Domain
- Pipeline Design
- Difficulty
- hard
- Seniority
- staff
Problem
Our digital marketing platform generates a continuous stream of ad impression and conversion events that need to feed both a real-time campaign performance dashboard and a daily attribution model. We have been running separate streaming and batch pipelines that have drifted out of sync, causing discrepancies between the live dashboard and the daily report. Design a unified architecture on Azure Databricks that eliminates the discrepancy.
Practice This Problem
Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it instantly.