Batch or Stream?

The architecture decision that defines your pipeline

Challenges

0 hands-on challenges

Lesson Sections

1. When Would You Use Streaming?

   Proving Streaming ROI The real question isn't "batch or stream?" The interviewer wants to hear: "Can you justify the operational cost of streaming to a VP who's staring at a $400K/year infrastructure bill?" Streaming pipelines cost 3-10x more than equivalent batch jobs. Your answer needs a business case, not a technical preference. Show the math or lose the room. The ROI calculation has three components: the value of reduced latency, the cost of streaming infrastructure, and the hidden cost of o

2. Design for Both Batch and Stream

   Kappa Architecture and Reprocessing at Scale Lambda's fatal flaw is dual maintenance. Every business logic change ships to two codepaths. Every bug exists in two places. The interviewer wants to hear you name this problem first, then present Kappa as the solution: eliminate the batch layer entirely. Everything flows through a single streaming pipeline, and reprocessing is done by replaying the event log from a historical offset. The follow-up will be: "What's the prerequisite for Kappa?" Your an

3. What About Late Data?

   Multi-Watermark Strategies and Result Retraction At the intermediate level, you learn that watermarks handle late data. The next level up, the interviewer tests whether you know that a single watermark is often insufficient. Real systems have multiple input streams with different lateness characteristics, and a single global watermark is bottlenecked by the slowest source. If you mention per-source watermarks unprompted, you've signaled serious depth. Result retraction is the advanced mechanism

4. Micro-batch or True Streaming?

   Trigger Intervals, State TTL, and Cost Modeling At the advanced level, the interviewer is testing whether you frame micro-batch vs true streaming as a cost optimization problem, not a technology preference. The variables are: per-event processing cost, state storage cost, checkpoint overhead, and the business value of each millisecond of reduced latency. Walk through all four to score highest. Checkpoint tuning is where the interviewer will go deep if you claim Flink production experience. The c

5. How Do You Handle Failures?

   Transactional Sinks, Barrier Snapshotting, and Failure as Architecture The interviewer is testing a specific mindset: failure handling isn't a recovery strategy, it's an architectural input. You design the system assuming failures happen constantly, and the architecture's job is to make failures invisible to downstream consumers. If you say "we handle failures with try-catch and retries," you've capped your score. The right answer is: "Failures are part of the steady-state design." Barrier snaps