Apache Kafka for Beginners introduces event delivery through a retailer's order history. You will publish and read real Kafka records, inspect how keys and partitions organize them, and track consumer progress before building Python clients that save events to a file.
The course combines 8 Guided Labs with 2 Challenges. The first six activities use terminal tools; the final four introduce Python clients. Guided examples explain success and controlled failure before Challenges combine familiar operations with new inputs.
What You Will Learn
After completing this course, you will be able to:
- create topics, inspect their metadata, and configure retention;
- publish keyed records and inspect ordering and offsets within partitions;
- track committed next positions, resume reads, and replay retained events;
- inspect consumer-group assignments and changes when a member leaves;
- serialize JSON in a Python producer and check delivery callbacks and queued work;
- write a bounded consumer that validates events, flushes file output before explicit commits, and exposes malformed-record and redelivery behavior.
Who This Course Is For
This course is for learners beginning event-driven data pipelines who want to understand Kafka records and client behavior through small, observable examples.
Prerequisites: Basic terminal use for the first six activities. The final four also require basic Python functions, loops, dictionaries, JSON, and running scripts. No prior Kafka, Java, Hadoop, Hive, or Spark knowledge is required.
Learning environment: LabEx provides a browser-based Linux terminal, one Kafka broker, and installed Python dependencies. The course covers client behavior rather than production cluster deployment or end-to-end exactly-once guarantees.
Frequently Asked Questions
Do I need Python or experience with Hadoop, Hive, or Spark?
Basic terminal use is enough for the first six activities. The final four require basic Python functions, loops, dictionaries, JSON, and running scripts. The other Big Data courses are not prerequisites.
How does Kafka fit alongside file-based batch processing?
Kafka stores event records that producers publish and consumers read. This course follows order events through topics, partitions, and consumer progress, then saves them to a file. It does not build a Spark streaming application or promise end-to-end exactly-once delivery.
Do I need to install Kafka or use a managed cloud service?
No. Each activity provides a prepared single-broker Kafka environment and the required clients in LabEx. You practice in the browser without provisioning an external Kafka service.
