Analyzing Shopping Sessions with Spark is a hands-on project for learners who know basic PySpark batch processing. Turn a closed shop-visit log into a reusable analysis that distinguishes repeated events from viewed session/product journeys and matches carts to the right product at the right time.
Two Guided Labs build the relationship decisions and a complete product comparison. One independent Challenge applies the same rules to a different shop batch. You will deliver real Parquet datasets and a small matching CSV, with counts that explain every ratio. The estimated 125–155 minutes is an author estimate including reading and ordinary mistakes.
What You Will Learn
After completing this course, you will be able to:
- retain each viewed session/product pair and its earliest view after removing identical event retransmissions;
- recognize valid cart existence without borrowing another product's cart or multiplying a journey when several carts match;
- calculate viewed-session denominators, completed and incomplete counts, and incomplete ratios;
- preserve zero-cart products and every tied comparison;
- write and consume actual Parquet and CSV results, rerun a derived snapshot and preserve its raw input;
- explain why closed-session patterns are observation clues rather than counts of unique people or causal proof.
Who This Course Is For
This project is for aspiring data analysts and data engineers who want to make Spark relationships and summary measures trustworthy.
Prerequisites: Basic terminal file navigation and Apache Spark for Beginners skills: explicit schemas, filtering, exact deduplication, DataFrame joins, grouping and Parquet. The project introduces the session/product observation unit, earliest-view timing, semi/anti joins and complete comparison denominators.
Learning environment: Each lab provides an independent browser-based Terminal with Spark, Python, local batch files and thin output drivers prepared. Spark runs with two local worker threads on one machine. No installation, Hive, Kafka or remote cluster is required.
Frequently Asked Questions
Do I write Python?
Yes. You complete small reusable functions returning Spark DataFrames. Guided Labs explain the function parameters and relationship expressions before you use them; the supplied driver handles session creation and file serialization.
Is this a real-time funnel or unique-customer report?
No. The inputs are finite batches of closed sessions with valid identifiers and UTC timestamps. A session is a visit identifier, and one session can view several products. Streaming, checkout and cross-device identity are outside the course.
What changes in the Challenge?
The batch identifiers and event relationships change. The Spark tools, time rule, output formats and recovery methods remain the same. Your result must retain the full comparison, including zero-cart products and ties.
