AI Coding Practice Problems

AI Coding Practice Problems

Agentic IDE practice problems for the AI-assisted data engineer interview round.

AI coding practice problems for the agent-assisted data engineer interview round. Each problem is a real repository with a failing test: reproduce the bug, diagnose it, ship a fix that keeps the other tests green. You get a file tree, a terminal, and an AI agent in the loop, which is how the round actually runs in 2026. Grading is server-side pytest against hidden tests, so a plausible-looking patch that does not fix the grain, the ordering, or the watermark still fails.

AI coding practice problems are the newest shape in the data engineer loop and the least like the others. There is no blank editor and no "write a query that returns the top 10". You are dropped into a repository that already runs, told which test fails, and given an agent you are expected to use. The 13 scenarios in this catalog span 11 companies and follow one arc: reproduce, diagnose, fix, explain. The explain step is scored as heavily as the patch, because a candidate who cannot say why the bug happened cannot be trusted to have fixed it rather than silenced it.

What the round is actually testing is judgment under agent assistance. The agent will happily produce a patch that makes the named test pass. Roughly half of these scenarios are engineered so that the obvious agent-suggested fix is wrong: it mutates the test, it special-cases the failing row, or it fixes the symptom one layer below where the defect lives. The hidden tests catch exactly that. The signal the interviewer is reading is whether you audit what the agent hands you, notice the fix is at the wrong altitude, and redirect it, or whether you paste and hope.

The bug classes are the ones that actually page data engineers at 3am. Schedule time versus event time: an Airflow DAG aggregates whatever arrived by the run and 3 percent of orders land after the window closes, so every on-schedule aggregate is provisional and the fix is a bounded lookback. Nondeterminism: a dbt uniqueness test passes locally and fails 20 percent of CI runs because the dedup ordering ties on identical timestamps and picks a different survivor each run, fixed with a complete tiebreaker in the model SQL. Grain violation: a join to a dropoffs table counts one order once per dropoff, so multi-stop deliveries inflate partner revenue 12 percent. Vacuous validation: a Great Expectations suite passes every check on an empty table, because absence of rows is not evidence of validity.

The streaming and lakehouse scenarios go deeper into engine semantics. Kafka consumer rebalance with mismanaged offset commits replays four million messages. A Flink watermark stalls forever because one quiet partition never advances and the watermark is the minimum across partitions, so window emission freezes until idleness is configured. Iceberg time travel after a rollback still returns corrupt rows, because rollback appends a new snapshot rather than erasing history and an AS OF query resolved by wall-clock timestamp lands back inside the corrupt lineage. A Snowflake MERGE collapses SCD Type 2 history because the matched condition omits the is_current predicate. A Spark job sits at 199 of 200 tasks complete with one executor grinding on a hot key that needs salting.

The ML and retrieval scenarios are increasingly common at AI-first companies. A RAG evaluation reports 0.94 while the answers are visibly wrong, because the eval set leaked into the retrieval corpus. Embedding drift degrades retrieval quality silently over months with no alarm, because nothing monitors the distribution. A feature store serves different values online than offline, because the training transform and the serving transform diverged. A Dagster asset graph fans out into a materialization storm from a dependency cycle. Companies represented across the catalog include Anthropic, OpenAI, Netflix, Stripe, Uber, Airbnb, DoorDash, Shopify, Snowflake, Confluent, and LinkedIn.

What is an AI coding interview round for data engineers?

You are given a real repository with a failing test and an AI agent in the workspace. The task is to reproduce the failure, diagnose the root cause, and ship a fix that keeps the currently-passing tests green. It replaces the blank-editor coding round at a growing number of companies because it matches how the job is actually done now: with an agent, in an existing codebase, under a test suite.

Am I allowed to use the AI agent during the round?

Yes, and you are expected to. The agent is part of the environment, not a loophole. What is being measured is your judgment in using it: whether you can spot that a suggested patch fixes the symptom instead of the cause, whether you verify against the tests rather than trusting the agent's summary, and whether you can explain the resulting diff as your own work.

How are these AI coding problems graded?

Server-side pytest against hidden tests, not just the named failing test. Fixes that special-case the failing row, mutate the test, or address the symptom one layer below the defect will pass the visible test and fail the hidden ones. Several scenarios are engineered specifically so the obvious agent-suggested patch fails this way.

What kinds of bugs appear in the catalog?

Production data-engineering failures: schedule-time versus event-time aggregation, nondeterministic dedup ordering, grain violations that double-count on fan-out joins, validation suites that pass vacuously on empty tables, Kafka offset mismanagement causing replay, Flink watermarks stalled by an idle partition, Iceberg time travel after a rollback, MERGE statements that flatten SCD Type 2 history, Spark skew, RAG eval leakage, embedding drift, and feature-store training-serving skew.

How is this different from the standard coding round?

The standard round starts from a blank editor and tests whether you can produce code. This round starts from working code and tests whether you can navigate it, localize a defect, and reason about blast radius. Reading unfamiliar code, deciding which layer owns the bug, and keeping the rest of the suite green matter more here than syntax fluency.

How long is each AI coding problem?

25 to 60 minutes, matching the real round. The four easy scenarios run 25 to 30 minutes, the medium ones 35 to 45, and the hard ones 45 to 60. The hard scenarios (RAG eval leakage, Dagster fan-out cycles, Flink watermarks, embedding drift) take longer because localizing the defect is most of the work.

Which companies run this interview format?

The catalog spans 11: Anthropic and OpenAI (RAG evaluation and agent-assisted debugging), Netflix (Iceberg), Stripe (dbt), Uber (feature stores and Spark), Airbnb (data quality), DoorDash (delivery fan-out), Shopify (Airflow), Snowflake (MERGE and SCD), Confluent (Kafka), and LinkedIn (Flink). AI-first companies adopted the format earliest; it is spreading fastest through teams that already ship with agents day to day.

How should a data engineer prepare for the AI coding round?

Practice reading unfamiliar repositories under time pressure rather than grinding algorithm problems. Get fluent at running a test suite, reading a stack trace to the right layer, and using git history to date a regression. Build the habit of asking of every agent patch: does this fix the cause or hide the symptom, and what else does it touch. That question is the round.

13 practice problems matching this filter. Difficulty: easy (4), medium (5), hard (4).