Introduction
A group can divide a topic's partitions among several consumers. Each partition has one active owner within a stable group, while another group can independently read the same retained records. This enables shared work without making every application compete for a single global reading position.
You will start two long-running console clients in one group, inspect their actual assignments before publishing, and compare a separate group's observations. After one member leaves, you will wait for the remaining member to own both partitions before sending another small batch. The prepared publisher explicitly targets both partitions so each has work during the experiment; its actual broker delivery is checked.
This independent lab supplies a two-partition topic, public event fixtures, and bounded read-only waiting tools. Basic terminal use is enough; the fixture tools require no Python editing. The console clients run in the background until you stop them in the final steps. Keep the same Terminal shell so their process variables remain available.
Start Two Members and Wait for Stable Ownership
In this step, you will observe assignment before giving the group work. The prepared topic has two partitions and no records. Inspect its metadata:
cd /home/labex/project
kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic orders-shared
The following clients use the same group, orders-team. --command-property passes each configuration value to the consumer client. client.id labels them reader-a and reader-b in reports. The familiar automatic commit settings let their reading progress settle while they continue polling.
The trailing & launches a client in the shell's background so you can run inspection commands in the same terminal. $! is the newly launched background process ID; each immediately following assignment saves it for the specific cleanup later.
The shell keeps two output streams: standard output for records and standard error for diagnostics. > writes the first stream to the JSONL file; 2> writes the error stream to the log. The assignment consumer_a_pid=$! stores a shell variable without spaces around =; "$consumer_a_pid" later retrieves that specific process ID. Ordinary records and diagnostics go to separate files so they do not interrupt your prompt. These clients keep running until the scoped kill commands near the end:
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders-shared --group orders-team --from-beginning --command-property client.id=reader-a --command-property enable.auto.commit=true --command-property auto.commit.interval.ms=1000 --command-property max.poll.records=1 > /home/labex/project/output/reader-a.jsonl 2> /home/labex/project/output/reader-a.log &
consumer_a_pid=$!
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders-shared --group orders-team --from-beginning --command-property client.id=reader-b --command-property enable.auto.commit=true --command-property auto.commit.interval.ms=1000 --command-property max.poll.records=1 > /home/labex/project/output/reader-b.jsonl 2> /home/labex/project/output/reader-b.log &
consumer_b_pid=$!
Wait for the actual two-member assignment and save its report:
/home/labex/project/tools/wait-assignment 2 > /home/labex/project/output/two-members.txt
cat /home/labex/project/output/two-members.txt
The read-only helper repeatedly uses the consumer-group members report until two labeled clients own nonempty, nonoverlapping assignments whose union is partitions 0 and 1. It has a deadline and does not manufacture assignments or change progress.
In the report, CLIENT-ID identifies reader-a or reader-b, and CURRENT-ASSIGNMENT lists actual current ownership. For example, orders-shared:0 means partition 0 of that topic. A target assignment can still be pending, so inspect the current column. Expect each member to own one partition, without requiring reader-a to receive a particular partition number. A rebalance redistributes partition ownership as group members join or leave. The group needs time to reach its new assignment; a process starting does not mean ownership has already settled. A stable assignment is the evidence of work sharing; two process IDs alone do not prove which partitions they own.
Publish Work and Compare an Independent Group
In this step, you will give both assigned partitions records and observe a second group reading the same history. Inspect the public batches. The initial batch contains share1 through share6; followup contains share7 and share8:
cat /home/labex/project/data/group-batches.json
The provided publisher sends the selected fixture to the actual broker, waits for delivery callbacks, and fails on incomplete delivery. Publish the six initial events only after the stable assignment:
/home/labex/project/tools/publish-shared-batch initial
/home/labex/project/tools/wait-caught-up orders-team 3 > /home/labex/project/output/team-initial-position.txt
cat /home/labex/project/output/team-initial-position.txt
The read-only caught-up helper waits until both partitions have committed next offset 3, end offset 3, and lag 0. Each partition received three records. Inspect what the two clients observed:
cat /home/labex/project/output/reader-a.jsonl
cat /home/labex/project/output/reader-b.jsonl
Together their files should include all six initial IDs. The live assignment report is the evidence of stable ownership; a file can also contain earlier observations from before an assignment change.
Now start a separate bounded group, orders-observer, from the retained beginning:
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders-shared --group orders-observer --from-beginning --max-messages 6 --timeout-ms 30000 --command-property enable.auto.commit=true --command-property auto.commit.interval.ms=1000 --command-property max.poll.records=1 > /home/labex/project/output/observer-initial.jsonl
cat /home/labex/project/output/observer-initial.jsonl
/home/labex/project/tools/wait-caught-up orders-observer 3 > /home/labex/project/output/observer-initial-position.txt
cat /home/labex/project/output/observer-initial-position.txt
It should receive all six records independently. It does not take partitions away from orders-team because it belongs to a different group. Both groups can be caught up at the same time while retaining separate committed positions.
Stop One Member and Observe the Remaining Assignment
In this step, you will leave reader-a running while stopping only reader-b. kill sends the saved process a termination signal; it does not stop the broker. wait lets the shell finish collecting that child process. A signaled process can report a nonzero exit status, so the final || true keeps this intentional cleanup command from being treated as an application failure:
kill "$consumer_b_pid"
wait "$consumer_b_pid" || true
/home/labex/project/tools/wait-assignment 1 > /home/labex/project/output/one-member.txt
cat /home/labex/project/output/one-member.txt
Expect reader-a to own both partitions after the group settles. The waiting tool checks the real report before the next publication. With two partitions, adding more than two members would leave some members without a partition in a stable assignment; this exercise does not require a third client.
Publish one followup event to each partition and wait for the remaining member to catch up:
/home/labex/project/tools/publish-shared-batch followup
/home/labex/project/tools/wait-caught-up orders-team 4 > /home/labex/project/output/team-followup-position.txt
cat /home/labex/project/output/team-followup-position.txt
cat /home/labex/project/output/reader-a.jsonl
reader-a should now include both share7 and share8. reader-b has stopped, so it cannot process these new records. The observer group still has its earlier committed position; resume it for the two new events:
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders-shared --group orders-observer --from-beginning --max-messages 2 --timeout-ms 30000 --command-property enable.auto.commit=true --command-property auto.commit.interval.ms=1000 --command-property max.poll.records=1 > /home/labex/project/output/observer-followup.jsonl
cat /home/labex/project/output/observer-followup.jsonl
/home/labex/project/tools/wait-caught-up orders-observer 4 > /home/labex/project/output/observer-followup-position.txt
cat /home/labex/project/output/observer-followup-position.txt
Expect share7 and share8 and each group's own next offset 4 in both partitions. The beginning option does not override the observer group's valid saved offset 3.
A rebalance can cause repeated delivery if a departing member handled records before its progress was committed. Event observations may therefore include repeats; stable assignment still gives a partition one owner at a time. This lab checks actual assignments and complete event content without claiming end-to-end exactly-once delivery.
Close the Remaining Client and Retain the Evidence
In this step, you will finish the long-running experiment by stopping the one remaining client. Its final positions have already been observed at the end, so no new work is pending. Use its own saved PID:
kill "$consumer_a_pid"
wait "$consumer_a_pid" || true
/home/labex/project/tools/wait-assignment 0 > /home/labex/project/output/no-members.txt
cat /home/labex/project/output/no-members.txt
The group should now have no active members. Saved offsets and broker events remain; client lifetime and stored progress are separate. The bounded observer clients have already exited automatically.
Inspect the retained member reports from before and after takeover:
cat /home/labex/project/output/two-members.txt /home/labex/project/output/one-member.txt
The first report demonstrates two nonoverlapping owners and the second one owner of both partitions. Those are actual reports with run-specific member identifiers, not fixed assignments to memorize. Reader files and observer files retain event evidence for both groups. You have closed only your two console clients while preserving the topic and committed positions for review.
Summary
You observed two consumers share real partition assignments, a separate group read the same records independently, and one remaining member take over both partitions before new work arrived. Bounded state checks replaced timing guesses, and scoped cleanup closed the clients while preserving progress and events. Next, you will replace the prepared publishing tool with a Python producer whose serialization and delivery handling you can read and modify.
