Data Engineering System Design Mock Interview

Every DE system design interview follows a similar structure. Having a framework keeps you organized and ensures you cover what interviewers evaluate. Here's the framework DataDriven teaches, with time allocation for a 60-minute round.

Step 1: Requirements (10 min). Don't start designing. Start asking. What's the data volume? What's the freshness SLA? Who consumes the data and how? What are the access patterns (ad-hoc queries, scheduled reports, ML model serving)? This step is where senior candidates differentiate themselves. L4 candidates skip it and jump to architecture. L5+ candidates spend 10 minutes asking pointed questions that shape every subsequent decision.

Step 2: Data Model (15 min). Before you draw boxes and arrows, define what the data looks like. What are the entities? What are the relationships? What's the grain of your fact tables? How do dimensions change over time? This step catches the candidates who can architect infrastructure but can't model data. At Meta, data modeling is often the deciding factor.

Step 3: Processing Layer (15 min). Now choose how data moves and transforms. Batch, streaming, or hybrid? Which framework (Spark, Flink, dbt, custom Python)? How do you handle late-arriving data? How do you ensure idempotency? The processing layer is where your experience shows. Generic answers like "we'll use Spark" score poorly. Specific answers like "Spark with daily batch jobs, 128MB partitions, repartitioned by date, triggered by Airflow at 03:00 UTC after upstream sources confirm data readiness" score well.

Step 4: Serving Layer (10 min). How does the data reach consumers? If it's analysts running ad-hoc queries, you need a warehouse with a good query engine. If it's a dashboard with sub-second response time, you might need a materialized view or an OLAP engine. If it's an ML model, you need a feature serving layer with p99 latency constraints. The serving layer connects your data pipeline to the business value.

Step 5: Monitoring and Failure Handling (10 min). What breaks? How do you know it broke? How do you fix it? Every system design that doesn't address failure modes is incomplete. Cover: data quality checks (row counts, NULL rates, distribution anomalies), pipeline monitoring (job duration, success/failure alerts), and recovery strategy (backfill process, dead letter queues, manual reconciliation for edge cases).

System design can't be practiced by reading solutions. You need to practice the conversation. In a real interview, the interviewer doesn't sit quietly while you monologue for 60 minutes. They interrupt, ask clarifying questions, challenge your assumptions, and introduce new constraints mid-conversation.

DataDriven's discuss mode replicates this experience. You receive a design prompt. You start by typing your clarifying questions. The AI responds with specific answers about data volume, SLA requirements, team size, and existing infrastructure. As you propose architecture components, the AI pushes back.

"You chose Kafka for event ingestion. The prompt says the system handles 500 events/second. Is Kafka the right choice for that volume, or is it over-engineering? What's the simplest solution that meets the requirements?"

"You mentioned using Snowflake as the serving layer. The prompt requires sub-second dashboard queries. Snowflake's cold query latency is 2-5 seconds. How do you handle that?"

"Your pipeline has no error handling. What happens when the Spark job fails at step 3 of 5? Do you reprocess everything from scratch? How long does that take?"

These aren't generic questions. The AI generates follow-ups based on your specific design choices. If you propose a streaming architecture, it asks about exactly-once semantics and late event handling. If you propose batch, it asks about freshness trade-offs and backfill strategy. This targeted questioning is what makes discuss mode better than practicing with a generic study partner who doesn't know the domain deeply.

1. Tool-first thinking. "Let's use Kafka, Spark, and Snowflake." Why? Because they're popular tools. This is the most common L4 mistake. Senior interviewers want to see requirements-first thinking. Start with the constraints (volume, freshness, access patterns), then choose tools that meet those constraints. If 500 events/second need to be ingested and daily batch freshness is acceptable, Kafka is over-engineering. A simple S3 file drop with hourly processing does the job at 5% of the operational complexity.

2. Skipping the data model. You draw a beautiful architecture diagram with boxes for ingestion, processing, and storage. The interviewer asks: "What does the data look like?" Silence. The data model is the foundation. Without it, your architecture is a collection of tools with no clear purpose. Define your entities, relationships, grain, and key dimensions before you discuss how to process them.

3. No failure handling. Your design works perfectly when everything goes right. But what happens when the source system sends duplicate records? When a Spark job fails halfway through a 4-hour run? When upstream data arrives 6 hours late? Every production pipeline fails. Interviewers want to see that your design accounts for failure, not just success. Discuss idempotency, dead letter queues, reconciliation processes, and alerting.

4. Over-engineering. You propose a real-time streaming architecture with Kafka, Flink, and Redis for a use case that requires daily batch reports. The interviewer asks: "The SLA is T+1. Why not a daily Spark job?" Over-engineering signals inexperience, not expertise. Senior engineers choose the simplest architecture that meets the requirements and articulate why complexity is (or isn't) justified.

5. Monologue instead of conversation. You talk for 20 minutes without checking in with the interviewer. System design is a collaborative exercise. After each major decision, pause: "Does this direction make sense, or should I go deeper on any part?" This shows communication skills and gives the interviewer opportunities to redirect you toward what they want to evaluate.

Every DE system design interview includes this decision, either explicitly or implicitly. The interviewer is testing whether you can match the processing model to the requirements instead of defaulting to the one you're most comfortable with.

Choose batch when: Freshness SLA is measured in hours or days. Data volume is large but bounded (you know when today's data is "done"). Consumers are reports, dashboards with T+1 freshness, or ML training pipelines. Cost sensitivity is high (batch compute is 60-80% cheaper than always-on streaming clusters). The data sources deliver data in files or bulk exports.

Choose streaming when: Freshness SLA is measured in seconds or minutes. Data arrives continuously with no natural "end of day." Consumers need real-time alerts, live dashboards, or online ML predictions. The business value degrades rapidly with staleness (fraud detection, dynamic pricing, live personalization).

Choose hybrid when: Some consumers need real-time and others need batch (this is the most common real-world case). The lambda architecture (parallel batch and streaming paths) has fallen out of favor due to maintenance burden. The kappa architecture (streaming-only with reprocessing capability) is simpler but harder to debug. Most modern teams use a practical hybrid: streaming for the critical path and batch for backfills, historical aggregations, and ML training.

DataDriven's design problems explicitly state freshness and volume requirements. The AI evaluates whether your batch/streaming choice matches those requirements and flags over-engineering (streaming when batch would work) and under-engineering (batch when the SLA demands real-time).