Pipeline Architecture Mock Interview for Data Engineers

Almost every design round touches this, sometimes explicitly and sometimes through tool-choice questions. The expected answer for new pipelines is ELT, because warehouse compute became cheap and elastic and dbt made in-warehouse transformations reproducible. Designs that still benefit from in-flight transformation tend to involve PII that cannot be persisted in raw form for compliance reasons. Some mock prompts include those constraints to surface the trade-off.

Related topics: ETL vs ELT deep dive (/etl-vs-elt), Pipeline architecture patterns (/pipeline/architecture), dbt interview questions (/tools/dbt-interview-questions), Snowflake interview questions (/tools/snowflake-interview-questions)

Streaming costs more across operations, on-call burden, debugging, and infrastructure. Part of the question in this section of the interview is whether the candidate examines the latency requirement instead of accepting it on its face. A dashboard that refreshes every five minutes does not require streaming; a micro-batch trigger or a scheduled dbt run is sufficient. True streaming (Kafka with a stream processor like Flink, using event-time processing and watermarks) is justified when sub-second latency directly affects revenue or safety, such as fraud scoring or ad serving.

How much Spark appears in any given interview is determined by the company. Lakehouse teams tend to spend a significant portion of the round on Spark internals, while Snowflake-with-dbt teams rarely touch it. When Spark questions appear, the focus is usually on the execution model: whether the candidate can explain what their code is actually doing at the stage and task level, not whether they can write a DataFrame chain. Databricks rounds layer Delta Lake, Unity Catalog, and the Photon engine on top of those fundamentals.

Most modern pipeline interviews touch all three. Kafka is justified when durable replay across multiple consumer groups is genuinely required; for occasional batched event delivery there are cheaper options. Airflow is justified when the number of interrelated jobs and the complexity of their dependencies have outgrown a cron file. dbt is the dominant choice for SQL-based transformation in the warehouse for the same reasons that ELT became dominant. The evaluation reflects the reasoning behind each tool choice rather than the choice itself.

A small set of scenarios recurs across companies. Real-time fraud detection focuses the discussion on latency budgets and model serving. Clickstream analytics raises sessionization and event retention. ML feature pipelines surface training-serving skew. CDC replication into a lakehouse leads to questions about handling upstream schema changes. The mock samples from these patterns and also generates novel prompts so candidates do not end up memorizing specific scenarios.

Reading about pipeline architecture provides a useful baseline but does not transfer directly to interview performance. The session tests the ability to defend a design twenty minutes into the conversation, when fatigue and prior commitments to specific tools start to constrain reasoning. That skill primarily develops through practice in a similar format.

The diagram is the easier part. Producing a reasonable initial design is within reach for most candidates with some experience. The discussion phase is where outcomes diverge, particularly on questions like sustained Kafka consumer lag or an unexpected upstream schema change. Those questions require reasoning during the session rather than recall.

Cost discussion separates levels. Junior responses tend to select tools without elaboration. Senior responses include explanations of why a less expensive alternative would have worked. Staff responses add an approximate cost comparison and an opinion on whether the operational overhead is justified. The follow-up depth in the mock matches the seniority level selected at the start.