Streaming data engineer roles became their own discipline in 2020-2024 as Flink, Kafka Streams, and Spark Structured Streaming matured. The role owns the real-time data substrate: ingestion, stateful stream processing, exactly-once delivery, backfill from historical events. The interview is technically demanding because streaming systems require reasoning about event ordering, late data, watermarks, and stateful transformations that batch engineers rarely face. Loops run 4 to 5 weeks. This page is part of the data engineer interview prep guide.
Concept frequency from 124 reported streaming data engineer loops in 2024-2026. The L4+ bar adds depth on watermarks, exactly-once, and state management.
| Concept | Test Frequency | Common In |
|---|---|---|
| Exactly-once semantics | 94% | Every L4+ streaming loop |
| Event-time vs processing-time | 89% | Every loop |
| Watermarks and late data | 82% | Every L4+ loop |
| Stateful processing (RocksDB, etc.) | 78% | L4+, deep at L5 |
| Kafka partitioning and ordering | 76% | Every loop |
| Backpressure handling | 67% | L5+ |
| Checkpointing and recovery | 71% | L4+ |
| Schema evolution in streams | 62% | Every L4+ loop |
| Sliding vs tumbling vs session windows | 58% | L4+ |
| Hot key handling | 54% | L5+ |
| Lambda vs Kappa architecture | 47% | L5+ |
| Backfill from historical events | 63% | L5+ |
| Cost optimization for streaming compute | 39% | L5+ |
Exactly-once is not a property of a single component; it is a property of the entire pipeline from producer to consumer. A pipeline is exactly-once if every event has its effect applied exactly once at the consumer, even under retry, replay, or partial failure.
Three common implementations: (1) Idempotent consumer + at- least-once delivery: producer sends each event possibly multiple times; consumer deduplicates by event_id with TTL. Cheap and works for most cases. (2) Transactional sink with exactly-once delivery: Kafka transactions or Flink two-phase commit ensure that the consumer's output and its offset commit are atomic. Expensive but truly exactly-once. (3) Event sourcing with deterministic replay: store the full event log, derive state by deterministic fold; on failure, replay from snapshot + delta. Expensive in storage but trivially exactly-once.
In an interview, when exactly-once comes up, name which of the three patterns you would use and why. Vague mentions of "exactly-once" without naming the implementation signal junior. Naming the trade-off (cost, latency, operational complexity) signals senior.
Event-time: the timestamp embedded in the event itself (when the click happened on the user's device). Processing-time: the timestamp when the event arrives at the stream processor. The two diverge because of network latency, mobile-app retries, batch upload delays.
Most analytical questions need event-time (revenue per day means revenue per day in the user's timezone, not per day in the processor's clock). Event-time processing requires watermarks: a per-stream signal of "we believe all events with event_ts <= T have arrived". Aggregations close when the watermark passes their window's end.
The honest answer about watermarks is that they are heuristics, not guarantees. A watermark of 5 minutes after event_ts means you tolerate up to 5 minutes of late data; anything later is late and must be handled separately (dropped, side-output, dead-letter). Stronger candidates describe the watermark choice as a freshness-vs-correctness trade-off: a tighter watermark closes windows faster but drops more late events; a looser watermark is more correct but adds latency to downstream consumers.
Real prompts from streaming data engineer loops in 2024-2026. Each architecture below is what got the candidate the L5 offer.
Producer -> Kafka (200K/sec, 100 partitions, key=user_id)
-> Flink stateful job:
EXACTLY_ONCE checkpointing, RocksDB state, 5-min interval
Window: 5-min tumbling, watermark 60 sec late allowed
Output: aggregated session metrics
-> S3 Iceberg (event-time partitioned, parquet)
-> Materialize (real-time view for dashboards)
Hourly Spark batch:
S3 raw -> Spark -> Snowflake fact_session_summary (source of truth)
Failure modes:
1. Flink TaskManager crash: checkpoint recovery, no data loss
2. Late events (> 60 sec): dead-letter, daily reprocess
3. Hot user_id (whale): mod-N salt, recombine in agg step
SLA tiers:
Tier 1 (real-time dashboards): p95 < 60 sec end to end
Tier 2 (hourly batch): completed within 90 min of hour-end
Tier 3 (daily): completed by 06:00 UTC dailyTotal comp ranges. US-based. Streaming roles pay roughly 5-10% above standard data engineer roles at the same level due to specialized skill requirement.
| Company tier | Senior streaming DE range | Notes |
|---|---|---|
| FAANG | $340K - $510K | All have substantial streaming infra |
| Stripe / Airbnb / Netflix | $320K - $470K | Streaming central to product |
| Uber / Lyft / DoorDash | $280K - $410K | Marketplace pricing requires streaming |
| Pinterest / Twitter / Snap | $300K - $440K | Real-time recommendations and timeline |
| Confluent / Striim / data-streaming vendors | $280K - $420K | Vendor-side streaming roles |
| Mid-size SaaS | $210K - $320K | Often analytics-event streaming |
Streaming overlaps with the ML data engineer interview guide on the real-time feature pipeline patterns and with the system design round prep guide on the system design framework. The Kafka vs Kinesis decision page covers the message broker trade-off relevant to streaming roles.
Companies most likely to hire streaming-specialized data engineer roles: Netflix has heavy streaming infra investment, Uber's marketplace pricing runs on streaming, Lyft uses streaming for surge pricing, Twitter (X) timeline generation is streaming-first.
Drill Kafka, Flink, exactly-once, and stateful streaming patterns in our practice sandbox.
Start PracticingSenior Data Engineer interview process, scope-of-impact framing, technical leadership signals.
Staff Data Engineer interview process, cross-org scope, architectural decision rounds.
Principal Data Engineer interview process, multi-year vision rounds, executive influence signals.
Junior Data Engineer interview prep, fundamentals to drill, what gets cut from the loop.
Entry-level Data Engineer interview, what new-grad loops look like, projects that beat experience.
Analytics engineer interview, dbt and SQL focus, modeling-heavy take-homes.
Continue your prep
50+ guides covering every round, company, role, and technology in the data engineer interview loop. Grounded in 2,817 verified interview reports across 929 companies, collected from real candidates.