Introduction
An order is created, another order is created, and then the first order is paid. A table of current status might show only the latest state of each order. An event describes something that happened, so all three facts can remain useful as a history. Systems that react to new orders need a way to pass those facts between applications.
Apache Kafka stores and delivers event records. A producer sends records, a broker is the server that receives and stores them, a topic names a stream of records, and a consumer reads them. Kafka can connect applications that operate at different times or speeds. Other processing tools can calculate reports from those events; this course first focuses on reliable delivery and reading progress.
You will create a topic, observe a producer and consumer in separate terminals, and read the same events again after the first consumer exits. The prepared single-broker environment uses one replica; cluster deployment is a separate topic. The first six activities require basic terminal use only. The later Python-client activities require functions, loops, dictionaries, JSON, and running scripts; the Python path can prepare you for those. Java, Hadoop, Hive, Spark, and external accounts are not prerequisites. Use the Terminal service.
Create a Place for the Order Events
In this step, you will confirm that the prepared Kafka service responds and create a topic. Keep this first shell as Terminal A. Start in your workspace:
## Terminal A
cd /home/labex/project
The command-line clients are already installed. --bootstrap-server supplies an initial broker address used to discover the cluster. Here localhost:9092 is the prepared local service, not an external account. List its topics to confirm a real broker response:
## Terminal A
kafka-topics.sh --bootstrap-server localhost:9092 --list
A clean teaching environment may have no application topics yet. Create orders-first. A topic contains one or more partitions, ordered logs that store its records. Start with one partition so this first sequence is easy to follow. --replication-factor 1 fits the single prepared broker:
## Terminal A
kafka-topics.sh --bootstrap-server localhost:9092 --create --topic orders-first --partitions 1 --replication-factor 1
Inspect the new topic:
## Terminal A
kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic orders-first
Find the topic name, PartitionCount: 1, and ReplicationFactor: 1. IDs, leader identifiers, and formatting may vary. Creating the topic established the storage destination; it did not yet send an order or install another server.
The broker will remain running while short-lived producer and consumer clients start and exit. Next, you will use that separation to watch events arrive.
Watch One Client Receive What Another Sends
In this step, you will keep a consumer waiting in one terminal and send three records from another. The Terminal service preserves its shell with tmux, a terminal-session manager. A second browser connection initially shows the same persistent session. Create a separate named shell session for Terminal B so the clients have independent prompts.
In Terminal A, create the detached session. -d leaves your current terminal selected, -s names the new session, and -c chooses its starting directory. This creates a shell, not another Kafka broker:
## Terminal A
tmux new-session -d -s kafka-b -c /home/labex/project
Open the same Terminal service in a second browser tab, keeping Terminal A open. Call the new tab Terminal B and switch that connection to the prepared session:
## Terminal B
tmux switch-client -t kafka-b
After the switch, press Ctrl-C once to clear any unsubmitted terminal input while only the shell is active. Confirm the session name:
## Terminal B
tmux display-message -p '#S'
Expect kafka-b. Terminal A retains its original session; both shells access the same VM files. Keep these roles when switching browser tabs. If the named session was already created during this exercise, reuse it instead of creating it again.
In Terminal B, start the consumer below. It waits until it reads three records, then exits automatically. --from-beginning requests the retained beginning for this fresh consumer; --max-messages 3 bounds its work, and --timeout-ms 30000 bounds an idle wait. The pipe sends ordinary output through tee, which displays it and saves a local copy:
## Terminal B
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders-first --from-beginning --max-messages 3 --timeout-ms 30000 | tee /home/labex/project/output/received-events.txt
Leave Terminal B waiting. Switch to Terminal A and send the batch within its 30-second idle limit. If that limit expires before you send anything, start the same consumer command in Terminal B again, then switch back to send the batch. If you already sent the events, rerun only the consumer to inspect the retained three records; sending the producer batch again would append duplicates:
## Terminal A
printf '%s\n' 'order-1001 created' 'order-1002 created' 'order-1001 paid' | kafka-console-producer.sh --bootstrap-server localhost:9092 --topic orders-first
printf prints each quoted text value followed by a newline. The pipe supplies those lines to the producer; this console producer treats each line as one event record. When printf finishes, its input stream closes, so the producer completes the pending sends and exits. You do not need to type into an interactive producer prompt.
Return to Terminal B. Its three data lines should be:
order-1001 created
order-1002 created
order-1001 paid
A processed-message count or client diagnostics may also appear in the terminal. The saved file should contain the three event values. The two facts about order-1001 are separate records; the paid event did not overwrite its earlier created event.
The consumer normally returns to its prompt after three messages. If you accidentally launch an unbounded client, Ctrl-C stops that client in its own terminal; it does not stop the broker or delete the topic. Here the limits make the normal path finish without manual interruption.
Read the Retained Events with a New Consumer
In this step, you will check whether reading removed the events. The first consumer has exited. Predict what a new independent consumer starting at the retained beginning should see: the same three records.
In Terminal B, run a new bounded consumer and save this second read separately:
## Terminal B
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders-first --from-beginning --max-messages 3 --timeout-ms 30000 | tee /home/labex/project/output/replayed-events.txt
Expect the same three values in the same order within this single partition. You did not send another batch. Inspect both local observations from Terminal A:
## Terminal A
cat /home/labex/project/output/received-events.txt
## Terminal A
cat /home/labex/project/output/replayed-events.txt
Both should contain all three original events. Consumption and storage retention are separate: reading does not remove a record from the topic. This first experiment uses fresh console consumers. Later you will learn how a named consumer group saves its reading progress, and why a beginning option does not override an already committed valid position.
Kafka kept the event history available after both short-lived clients exited. Next, you will inspect topic properties and see how retention configuration affects the storage policy independently of consumer progress.
Summary
You identified Kafka's event-delivery role, created a one-partition topic, and observed separate producer and consumer clients exchange three order events. A new consumer read the same retained history after the first exited, showing that reading and deletion are different operations. Next, you will inspect topic partition, replica, and retention properties.
