Data Engineering System Design Interview (2026)

The first fork in any system design answer. Interviewers test whether you can reason about this trade-off from first principles, not just recite definitions.

• Batch is right when data freshness requirements are hours, not seconds. Daily revenue reports, weekly cohort analysis, and monthly aggregations are all batch workloads. Choosing batch when it suffices shows maturity.
• Streaming is right when business decisions depend on seconds-to-minutes freshness: fraud detection, real-time recommendations, live dashboards for operations teams.
• The Lambda architecture (batch + streaming in parallel) sounds elegant but doubles your maintenance burden. Prefer one or the other unless the business requirement genuinely demands both. State this trade-off explicitly in your answer.
• Cost is part of the architecture. Streaming infrastructure runs 24/7 and costs 3-10x more than equivalent batch processing. The interviewer wants to hear that you weigh economics alongside latency requirements.

Data systems force you to choose. Can your dashboard show slightly stale data? Can your pipeline tolerate duplicate records temporarily? Your answer to these questions shapes the entire architecture.

• Strong consistency (every reader sees the latest write) is expensive. It requires synchronous replication, distributed locks, or serializable transactions. Know when the business actually needs it vs when eventual consistency is fine.
• Eventual consistency is cheaper and faster but means downstream consumers may see stale or temporarily inconsistent data. For analytical workloads, this is usually acceptable. Say so in your answer.
• Exactly-once delivery is a spectrum, not a binary. At-least-once with idempotent consumers is the practical pattern for most data pipelines. If you claim exactly-once, the interviewer will push back.
• When drawing your architecture, label each connection with its consistency guarantee. This demonstrates that you think about data correctness at every boundary, not just at the endpoints.

Where your data lives determines how fast you can query it, how much it costs, and how flexible your schema can be. This is where many candidates go shallow. Go deep.

• Data lake vs data warehouse is not either/or. Most modern architectures use both: raw data in object storage (S3, GCS), curated data in a warehouse (Snowflake, BigQuery). Explain the layering in your answer.
• File format matters. Parquet for analytical queries (columnar, compressed, schema-embedded). Avro for streaming (row-based, schema evolution). JSON for flexibility at the cost of performance. Pick the right format for each layer and explain why.
• Partitioning strategy drives query performance and cost. Partition by date for time-series workloads. Cluster by high-cardinality filter columns. Always state your partitioning choice when designing a table.
• Materialized views and pre-aggregation tables trade storage and refresh cost for query speed. In your design, identify which queries are latency-sensitive and pre-compute those results.

System design interviews expect back-of-envelope math. How many events per second? How much storage per day? What compute do you need? These numbers drive your architecture choices.

• Start with the input rate. If the prompt says '1M events per minute,' convert that to roughly 17K events/second and estimate record size. This gives you throughput in MB/s, which determines whether you need a message queue, what tier of compute, and how much storage per day.
• Storage growth compounds. 1M events/minute at 500 bytes each is 720 GB/day raw. With compression (3-5x for Parquet), that is 150-240 GB/day. Over a year, that is 50-85 TB. State these numbers to show you think about operational cost.
• Scaling bottlenecks differ by layer. Ingestion is usually network-bound. Transformation is CPU-bound. Storage is IOPS-bound for random access, throughput-bound for scans. Identify which layer is your bottleneck and design around it.
• Right-size your compute. A daily batch job that runs for 10 minutes does not need a cluster running 24/7. Mention auto-scaling and ephemeral compute to show cost awareness.

Naming Kafka or Airflow is not enough. Interviewers want to hear why you chose that component over the alternatives, and under what conditions you would choose differently.

• Lead with the requirement, then justify the tool. 'We need a durable message queue with replay capability for consumer recovery, so I would use Kafka here. If we only needed simple task queueing, SQS would be simpler and cheaper.'
• For orchestration, explain the trade-off between managed services (Step Functions, Cloud Composer) and self-hosted (Airflow, Dagster). Managed reduces ops burden but limits customization.
• For compute, distinguish between the SQL engine (warehouse queries), the processing framework (Spark, Flink for heavy transforms), and lightweight scripts (Python for small transforms). Not every stage needs the same engine.
• Draw clear boundaries between components. Each component should have a single responsibility. The orchestrator schedules. The queue buffers. The warehouse stores and queries. When you couple responsibilities, explain why.

Every architecture decision has a cost dimension. Senior candidates discuss cost trade-offs as naturally as they discuss latency trade-offs.

• Streaming costs 3-10x more than batch for the same data volume because the infrastructure runs continuously. Quantify this in your answer when possible. If the business needs sub-minute freshness, the premium is justified. If daily is fine, batch wins.
• Storage tiering saves money. Hot data (last 30 days) in the warehouse for fast queries. Cold data (older than 90 days) in object storage for cheap archival. Define your retention policy and tiering strategy.
• Compute costs scale with query complexity and data volume. Pre-aggregating common query patterns into materialized views costs refresh compute but saves 10-100x on downstream queries. Identify which queries justify this investment.