Consumer Group
A consumer group is how several consumers split one topic between them. Within a group each partition is assigned to exactly one consumer, so adding consumers raises throughput — until you pass the partition count, at which point the extras sit idle. Each consumer commits offsets for the partitions it owns, and a different group reads the same data on its own independent schedule.
01Consumer Group
Concept at a Glance① Who reads? · Partition Assignment
Events are stacked across three lanes. First things first: who reads them?
Start with one reader and it takes all three lanes. It works, but the pace is tied to that single one.
Add a second to the same group and the assignment is recalculated. Lane 1 passes to consumer-2 and the two share the work.
Add a third and it's one lane each — 1:1. This is the fastest this topic can go.
What about a fourth? A lane is assigned to only one member within a group, so consumer-4 has no lane to take and sits idle.
② How far? · Offset Commits
Starting from the 1:1 assignment. None of them has read anything yet, so every ▲ sits at the front.
Each reads its own lane from the front. ▲ marks where it reads *next*, so it slides right as work gets done.
The position processed so far is recorded back into Kafka (an offset commit). Restart the process and it resumes right here.
consumer-2 just died. Lane 1 has lost its owner, so it passes to a remaining consumer — a rebalance.
The new owner resumes from the committed position. ▲ hasn't moved, so nothing is lost and nothing is read twice.
③ Another team? · Group Independence
Everything so far was the order-processing team (group A). This is how far each of them has read.
Now an analytics team (group B) wants the same records. A different group name makes it entirely separate.
B keeps its own ▲ and reads from the very front. A's progress doesn't shift by a single slot.
02 Understand It Simply
For EveryoneWithin one group a partition is assigned to exactly one consumer, so the partition count is that group's ceiling on parallelism. A different group reads the same log independently, at its own offsets.
Inside a group, partitions are assigned exclusively, which makes the partition count the ceiling on parallelism.
Each consumer commits the offset it has processed, so restarting resumes from there.
When a consumer joins or leaves, a rebalance recalculates the assignment — and consumption pauses briefly while it happens.
Groups with different IDs never interfere; each reads the same log at its own offsets.
- –Matching consumer count to partition count
- –scaling out throughput
- –choosing an offset-commit strategy (at-least-once vs at-most-once)
- –and fanning the same data out to multiple uses
03 Frequently Asked Questions
FAQWhat is Consumer Group?+
A consumer group is how several consumers split one topic between them. Within a group each partition is assigned to exactly one consumer, so adding consumers raises throughput — until you pass the partition count, at which point the extras sit idle. Each consumer commits offsets for the partitions it owns, and a different group reads the same data on its own independent schedule.
Where is Consumer Group used?+
Matching consumer count to partition count, scaling out throughput, choosing an offset-commit strategy (at-least-once vs at-most-once), and fanning the same data out to multiple uses.
What's a simple analogy for Consumer Group?+
Within one group a partition is assigned to exactly one consumer, so the partition count is that group's ceiling on parallelism. A different group reads the same log independently, at its own offsets.