Introduction
Several events can describe the same order: created, paid, and shipped. If related records are routed consistently, a consumer can follow that order's sequence within one partition. A record key supplies a stable routing input; it is different from an event_id, which identifies one event occurrence.
You will publish interleaved events for three orders to a three-partition topic, print their actual keys and positions, and add one more event for an existing order. An offset is a record's position inside one partition. Offsets are not global topic row numbers, and records from different partitions have no single guaranteed global ordering.
The broker and input fixture are independent. Basic terminal use is enough. The event values use readable JSON text; no Python, JSON programming, cluster configuration, or earlier topic state is required. Keep the partition count and client routing strategy fixed throughout the experiment. Use the Terminal.
Publish Interleaved Events with an Order Key
In this step, you will send a stable order identifier as the key while keeping the complete event as the value. Inspect the supplied five-record batch:
cd /home/labex/project
cat /home/labex/project/data/keyed-orders.txt
Each line has key|value. The key is an order identifier. The JSON value uses named fields in braces: event_id is unique, order_id matches the key, sequence is that order's business sequence, and event_type describes the occurrence. JSON text is the value's format; Kafka does not automatically interpret it as a table schema.
Create a topic with three partitions and one replica:
kafka-topics.sh --bootstrap-server localhost:9092 --create --topic orders-keyed --partitions 3 --replication-factor 1
The producer's --reader-property options configure how it interprets input lines. parse.key=true and key.separator=| split each input line at the separator. The separator itself does not become part of the key or value. The shell's < supplies the file as standard input, and the producer exits after sending its finite input:
kafka-console-producer.sh --bootstrap-server localhost:9092 --topic orders-keyed --reader-property parse.key=true --reader-property 'key.separator=|' < /home/labex/project/data/keyed-orders.txt
The file interleaves orders, so events for order-2001 are not adjacent in the input. With the same key-routing strategy and unchanged partition count, equal keys route to the same partition. Different keys can share a partition; three orders do not guarantee three different partition numbers.
Publish this supplied five-record batch only once. Repeating a consumer read is an inspection; repeating this producer command appends five more records. If you are resuming after publication, continue with the consumer inspection instead of resending the file.
You supplied the routing input, rather than selecting a partition number manually. Next, you will read the actual routing decisions instead of guessing them.
Inspect Keys, Partitions, and Offsets Together
In this step, you will display record metadata with each value. The console consumer's --formatter-property options control the display, adding the key, partition, and offset to its ordinary output. They do not change stored event content. Its fresh-consumer beginning option, five-message limit, and idle timeout make this finite inspection repeatable. Save the output locally:
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders-keyed --from-beginning --max-messages 5 --timeout-ms 30000 --formatter-property print.key=true --formatter-property print.partition=true --formatter-property print.offset=true > /home/labex/project/output/keyed-first.txt
cat /home/labex/project/output/keyed-first.txt
Each data line should show fields with this general shape, using actual numbers and a complete JSON value:
Partition:... Offset:... order-... {"event_id":"...",...}
The separators are tabs. Find k1 and k3: both have key order-2001 and should share a partition, with k3 at a larger offset. k2 and k4 should likewise share the order-2002 partition. Do not require those two orders to use different partitions.
Offsets belong to the partition, not the key. If other keys share that partition, an order's successive offsets can have gaps. The payload's sequence field is an application value; Kafka did not generate it as an offset.
The consumer can interleave records from different partitions, so its global display order need not match the producer file. Check related events through their key, partition, and increasing offsets rather than treating the output as one globally ordered list.
Add Another Event for an Existing Order
In this step, you will test the same routing rule with another producer invocation. Predict the outcome for order-2001's shipped event: it should go to that order's existing partition and follow its earlier records there. Its exact offset depends on any other records sharing that partition.
The printf pattern from the first Kafka lab sends one complete keyed line to the same console producer settings:
printf '%s\n' 'order-2001|{"event_id":"k6","order_id":"order-2001","sequence":3,"event_type":"shipped"}' | kafka-console-producer.sh --bootstrap-server localhost:9092 --topic orders-keyed --reader-property parse.key=true --reader-property 'key.separator=|'
Read all six retained records into a new observation:
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders-keyed --from-beginning --max-messages 6 --timeout-ms 30000 --formatter-property print.key=true --formatter-property print.partition=true --formatter-property print.offset=true > /home/labex/project/output/keyed-with-shipped.txt
cat /home/labex/project/output/keyed-with-shipped.txt
Find k6 with order-2001 and sequence 3. Compare its partition with k1 and k3 and its offset with their offsets. The order's sequence should remain created, paid, shipped when viewed in partition-offset order. No fixed partition number is required.
The original five-record observation remains in keyed-first.txt. The new producer appended an event rather than replacing the earlier ones. Topic partition counts and routing strategy can affect routing, so this experiment deliberately changes neither. Later clients must choose a compatible routing strategy if they need the same mapping across implementations.
You now have evidence for stable per-key routing and partition-local ordering, without assuming a global event order or one partition per order. The next challenge applies these same topic, retention, and keyed-publication skills to a new batch.
Summary
You supplied order keys, inspected actual partitions and offsets, and verified an appended event followed its order's earlier records under the same routing rule. You distinguished event IDs, order keys, payload sequence, and partition offsets, while avoiding a global-order assumption. Next, a new batch challenge combines these skills with topic retention configuration.
