Dynamic Schema File Ingestion Pipeline

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Seniority

staff

Problem

We receive data files from a government agency every week. The files contain regulatory reporting data that we need in our warehouse, but the agency changes the file format without warning. Sometimes columns are added, sometimes renamed, sometimes the whole layout shifts. Design an ETL pipeline that can handle these file structure changes dynamically.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it instantly.