The Materialization Storm

A hard AI Coding interview practice problem on DataDriven. Write and execute real ai coding code with instant grading.

Domain

AI Coding

Difficulty

hard

Seniority

L6

Problem

We added a new dataset to our Dagster repo last week. It declared 50 dynamic-partition assets that depend on shared upstream assets. Dagster scheduler CPU pegged at 100% within an hour and we're now seeing 12,000 materialization runs per hour for what should be ~50. Run tests/test_no_runaway_runs.py to reproduce. Find the cycle/fanout pattern in dagster_repo/assets.py and fix it so the partition fan-out is bounded and the run count is sane. Be ready to defend the trade-off your fix makes.

Summary

The graph started eating itself.

Practice This Problem

Solve this AI Coding problem with real code execution. DataDriven runs your solution and grades it automatically.