Why Rebalancing Stalls
Whenever a consumer joins or leaves, the partition assignment has to be recalculated. The older protocol (eager) has every consumer give up everything first and then redivide, so partitions that never needed to move stop too. The cooperative protocol revokes only what actually has to move and lets the rest keep reading.
01 Concept at a Glance
Interactive Step-by-StepTwo consumers are splitting three lanes. A third wants to join.
Same situation. Two consumers are reading and a third wants to join.
02 Understand It Simply
For EveryoneLike redividing warehouse aisles: everyone walks out and files back in, versus only the people who need a different aisle moving. The first way creates a stretch where nobody works.
Eager rebalancing is stop-the-world.
One consumer joins and everybody revokes their assignment and waits for the recalculation, so the whole group stops consuming meanwhile.
Cooperative (incremental) rebalancing reclaims only the partitions that genuinely change owner and leaves the rest untouched, minimising the pause.
The more often processes restart, the more that difference shows up as lag.
- –Choosing an assignment strategy for environments with frequent deploys or autoscaling
- –diagnosing rebalance-induced lag
- –and tuning session.timeout and max.poll.interval
03 Frequently Asked Questions
FAQWhat is Why Rebalancing Stalls?+
Whenever a consumer joins or leaves, the partition assignment has to be recalculated. The older protocol (eager) has every consumer give up everything first and then redivide, so partitions that never needed to move stop too. The cooperative protocol revokes only what actually has to move and lets the rest keep reading.
Where is Why Rebalancing Stalls used?+
Choosing an assignment strategy for environments with frequent deploys or autoscaling, diagnosing rebalance-induced lag, and tuning session.timeout and max.poll.interval.
What's a simple analogy for Why Rebalancing Stalls?+
Like redividing warehouse aisles: everyone walks out and files back in, versus only the people who need a different aisle moving. The first way creates a stretch where nobody works.