Batch vs Streaming

Concepts: paBatchVsStreaming

The #1 Pipeline Interview Question This question is designed to test your judgment, not your knowledge. The interviewer describes a scenario and wants to see you reason through the decision, not recite definitions. Here is the framework that works every time: Step 1: Ask 'If this data is 1 hour old, does anyone lose money or make a bad decision?' If no, batch. Step 2: If yes, ask 'Does a 5-minute delay cause the same problem?' If 5 minutes is fine, micro-batch. If sub-minute matters, true streaming. Step 3: State the tradeoff. 'Streaming costs 3-5x more in compute and engineering time. The business value of freshness needs to justify that.' Practice Scenarios