Delivery Semantics

At-least-once, at-most-once, exactly-once: know what your pipeline actually guarantees

At-least-once, at-most-once, exactly-once: know what your pipeline actually guarantees

Category

Pipeline Architecture

Difficulty

advanced

Duration

25 minutes

Challenges

0 hands-on challenges

Topics covered: "What Guarantee Does Your Pipeline Provide?", At-Least-Once: The Default and Its Duplicates, At-Most-Once: Fire and Forget, Exactly-Once: The Holy Grail and Its Cost, Choosing Semantics for a Business Requirement

Lesson Sections

1. "What Guarantee Does Your Pipeline Provide?"

   What They're Really Testing The Three Guarantees The Unlock The 60-Second Framework Why Companies Care At Stripe, payment events use exactly-once because a double-processed payment charge means a customer is billed twice. At Netflix, playback telemetry uses at-least-once because a duplicate play event is corrected by downstream dedup at negligible cost. At Uber, ride fare calculations moved from at-least-once to exactly-once after duplicate fare charges caused a customer trust incident that made

2. At-Least-Once: The Default and Its Duplicates

   At-least-once is the default in almost every messaging system. Kafka, SQS, Pub/Sub, RabbitMQ all default to at-least-once. The reason: it is the only semantic that guarantees no data loss without the complexity of transactions. The tradeoff is duplicates, which are handled downstream. How Duplicates Happen Downstream Dedup Strategies

3. At-Most-Once: Fire and Forget

   At-most-once is the least discussed semantic but it appears in interviews as a trap. The interviewer describes a use case (monitoring metrics, debug logs) and waits to see if you over-engineer it with exactly-once. The strong candidate says 'this is at-most-once territory; losing a few metrics samples is acceptable and the simplicity is worth it.' How It Works Commit the offset BEFORE processing. If the consumer crashes after committing but before finishing, the message is lost. No retry, no rep

4. Exactly-Once: The Holy Grail and Its Cost

   Exactly-once semantics (EOS) is the most misunderstood concept in streaming. Candidates either dismiss it as impossible or claim it is the only acceptable guarantee. Neither is correct. EOS is achievable within Kafka, with specific mechanisms, at a specific cost. Kafka's Three-Layer EOS The Cost of EOS The Follow-Up Trap

5. Choosing Semantics for a Business Requirement

   The interview signal is not knowing all three semantics. It is mapping a business requirement to the right one and defending the choice. This is the 'trade-offs and bottlenecks' rubric item that separates L5 from L6. Decision Matrix Vocabulary That Signals Seniority The Bridge Move