Why Rebalancing Stalls
Whenever a consumer joins or leaves, the partition assignment has to be recalculated. The older protocol (eager) has every consumer give up everything first and then redivide, so partitions that never needed to move stop too. The cooperative protocol revokes only what actually has to move and lets the rest keep reading.
01Why Rebalancing Stalls
Concept at a Glanceeager · everyone revokes, then redivides
Two consumers are splitting three lanes. A third wants to join.
The eager protocol first has everyone give up what they hold. The assignment empties out completely.
Meanwhile all three lanes stop being consumed. The problem is that lanes 0 and 2 stopped even though nothing about them had to change.
Once the recalculation finishes, everyone takes a new assignment and starts reading again — one lane each.
cooperative · revoke only what moves
Same situation. Two consumers are reading and a third wants to join.
The cooperative protocol works out the plan first — only lane 1 actually needs to change hands.
So only lane 1 is revoked. Lanes 0 and 2 keep the same owner and keep reading.
The revoked lane 1 is handed to consumer-3 and it's done. The end state matches eager exactly.
02 Understand It Simply
For EveryonePartition assignment is recomputed whenever consumers join or leave. Eager has everyone give up their partitions first, stopping all consumption meanwhile; cooperative revokes only the partitions changing hands, so the rest keep reading.
Eager rebalancing is stop-the-world.
One consumer joins and everybody revokes their assignment and waits for the recalculation, so the whole group stops consuming meanwhile.
Cooperative (incremental) rebalancing reclaims only the partitions that genuinely change owner and leaves the rest untouched, minimising the pause.
The more often processes restart, the more that difference shows up as lag.
- –Choosing an assignment strategy for environments with frequent deploys or autoscaling
- –diagnosing rebalance-induced lag
- –and tuning session.timeout and max.poll.interval
03 Frequently Asked Questions
FAQWhat is Why Rebalancing Stalls?+
Whenever a consumer joins or leaves, the partition assignment has to be recalculated. The older protocol (eager) has every consumer give up everything first and then redivide, so partitions that never needed to move stop too. The cooperative protocol revokes only what actually has to move and lets the rest keep reading.
Where is Why Rebalancing Stalls used?+
Choosing an assignment strategy for environments with frequent deploys or autoscaling, diagnosing rebalance-induced lag, and tuning session.timeout and max.poll.interval.
What's a simple analogy for Why Rebalancing Stalls?+
Partition assignment is recomputed whenever consumers join or leave. Eager has everyone give up their partitions first, stopping all consumption meanwhile; cooperative revokes only the partitions changing hands, so the rest keep reading.