When a Send Counts

Concepts covered: Producer acknowledgements

acks takes 3 values

Step 6 of the path, the reply, happens when the leader decides the record is safe enough, and the producer sets that threshold with 1 setting, acks. It takes 3 values. With acks=0 the producer asks for no reply at all, and treats the send as done the instant the request is written to the socket; the leader may append it, or may never receive it, and the producer will not know. With acks=1 the leader replies as soon as the record is in its own log, before any follower has fetched it. With acks=all the leader waits until every replica currently in the in-sync set has copied the record, then replies. Kafka 4.4 defaults to acks=all.

What the cluster has agreed to

The word all means the replicas in the in-sync set at the moment of the write, the set from the replication lesson. A follower that has dropped out of the ISR is skipped: with 3 replicas and 1 of them lagging, acks=all returns after the leader and the remaining follower hold the record. The acknowledgement travels with the replication: when the leader replies, the record already sits on every broker that could be elected leader next, so the leader's death a millisecond later changes nothing about that record. With acks=1 the same death loses the record if the followers had not yet fetched it, and the producer holds an offset for a record that no surviving log contains.

A later replication lesson adds a floor under how small the in-sync set may shrink before the leader refuses writes; without it, acks=all with an ISR of 1 is the leader alone.

The animation plays 6 payments through a 3-broker cluster with the followers deliberately slow. Switch between acks=1 and acks=all and watch when each record is marked acknowledged relative to the followers' copies; then strike the leader down after a couple of sends and compare what the surviving brokers hold against what the producer believed was done.

The wait each acks level adds

The price of acks is time per send. acks=0 costs 1 one-way trip, and the producer never waits. acks=1 costs a round trip to the leader, which inside 1 datacenter is typically 1 to 3 ms. acks=all adds the followers' fetch on top: the leader cannot answer until each in-sync follower has asked for the record and received it, which adds around 1 more round trip, so a send that took 2 ms at acks=1 takes 3 to 5 ms at acks=all. Because the producer keeps many records in flight at once, the longer wait rarely reduces how many records per second a producer can push; it lengthens how long each individual record waits for its callback.

# acks=all is the default in Kafka 4.4
acks=all
client.id=checkout-payments

The command-line producer takes the same setting on its command line: kafka-console-producer.sh --bootstrap-server broker-0:9092 --topic payments --producer-property acks=1 sends every typed line at the weaker level, which is handy for watching the difference and for nothing else.

The same crash under acks=1 and acks=all

Under acks=1 the producer's callbacks reported 5 landed records, and the new leader's log holds 3; the application has already written done for pay-4 and pay-5 somewhere else. Under acks=all the callbacks for pay-4 and pay-5 had not fired, so the producer still holds both records, and once a new leader is elected it sends them there, which section 3 walks through.

About this lesson

This section is part of the Producers: Beginner lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.