Introduction
A community workshop has two signup-page versions, A and B. Before comparing them, its analyst must decide what counts as one observation. One visitor can visit three times or click signup twice, while another never generates an event. Counting those event rows as separate people would distort the comparison.
You will first read a six-person worked example, then build one seven-day binary outcome for every assigned visitor in a larger, fixed experiment. The records are original synthetic teaching data representing a randomized allocation, not live visitor tracking or evidence about a real business. Python, pandas, NumPy, SciPy and Matplotlib are prepared in an independent WebIDE environment. You need basic terminal navigation, Python functions and introductory pandas table selection; the required joins and time operations are explained here. Statistical inference is not a prerequisite. The 55-minute duration is an author estimate including reading and ordinary corrections.
Compare People Instead of Clicks
In this step, you will read six visitor cards and calculate signup rates with the assigned people as the denominators.
Start in WebIDE, then open Terminal → New Terminal. The terminal and file explorer share the same files. cd changes this terminal's working directory; repeat it when opening another terminal.
cd /home/labex/project/signup-experiment
cat displays a text file without changing it. Read the small worked example:
cat warmup-visitors.csv
Each row is one assigned visitor. The first three belong to A and the last three to B. For this worked example, qualifying_signup_events has already been classified inside each person's seven-day window. signed_up is the corresponding binary outcome: 1 if there was at least one qualifying signup, otherwise 0.
P-8 has three visits and two signup events, but contributes one person and one positive outcome. Q-3 has no visit or signup event, but remains an assigned person with outcome 0. Read the six cards before calculating: A has one signup among three assigned people; B has two among three.
A's signup rate is 1 / 3, or about 33.33%. B's is 2 / 3, or about 66.67%. Their difference is B minus A, about +33.33 percentage points. Percentage points subtract percentages directly; this is not a relative percentage increase. More visits do not enlarge either denominator.
Write a reusable comparison function. The runner passes a pandas DataFrame named people, with visitor_id, group and signed_up columns. A Boolean mask such as people["group"] == "A" selects A's rows. len(a) counts assigned people; summing a binary column counts people who signed up. int and float produce ordinary Python numbers for JSON files.
The shell redirection > replaces the named file. A quoted here-document, <<'PY', passes everything up to a line containing only PY into that file without shell expansion. Paste the whole block including its final PY. Alternatively, edit the same file in WebIDE and use File → Save.
cat > compare.py <<'PY'
def summarize(people):
result = {}
for group in ["A", "B"]:
selected = people[people["group"] == group]
assigned = len(selected)
signed_up = int(selected["signed_up"].sum())
result[group] = {
"assigned": assigned,
"signed_up": signed_up,
"rate_pct": 100 * signed_up / assigned,
}
result["difference_pp"] = result["B"]["rate_pct"] - result["A"]["rate_pct"]
return result
PY
python3 runs Python. The prepared run.py calls your function and writes its actual return value; its warmup argument selects this input. It does not fill in missing answers.
python3 run.py warmup
The command saves output/warmup.json. JSON stores named objects and numbers. The prepared reader only displays files that already exist; it never reruns your program or repairs a missing result.
python3 show_results.py warmup
Confirm assigned counts 3 and 3, positive counts 1 and 2, and a difference near +33.33 percentage points. Keep unrounded numbers in saved results; rounded numbers in this explanation are for reading only. A larger observed rate by itself does not establish that B improved the page.
Only compare.py and, next, outcomes.py need changes in this lab. You may add regular Python helper files beside them. Preserve assignments.csv, events.csv, warmup-visitors.csv, config.json, data-dictionary.md, run.py and show_results.py. Other units have independent copies; they will not depend on files from this VM.
If a filename or directory is wrong, use pwd to display your current directory and ls to list its files, then return with the cd command above. Recopy the complete function block to replace an ordinary editing mistake; there is no need to rerun setup or reset the environment.
Build Complete Seven-Day Outcomes
In this step, you will turn raw assignment and event records into the complete person-level table that the comparison requires.
Continue in the same WebIDE terminal at /home/labex/project/signup-experiment. Read the field definitions, then the original tables:
cat data-dictionary.md
cat assignments.csv
cat events.csv
The full table has 12 assigned people, six in each group. visitor_id is a stable text identity, not a row position. The event table has repeated people and an unrelated OUTSIDE visitor, so the two tables cannot be zipped by row number. Its log is complete for this teaching example: no qualifying event means no signup, rather than an unknown outcome.
Every person has a timezone-aware allocation instant, assigned_at. The window is [assignment, assignment + seven days): the start counts and the exact upper boundary does not. UTC is a common time reference; 12:00:00+02:00 and 10:00:00Z describe the same instant. pd.to_datetime(..., utc=True, format="ISO8601") converts these explicitly formatted strings to comparable UTC timestamps. A Timedelta(days=7) represents seven days of elapsed time.
All follow-up must have finished by the supplied analysis instant:
cat config.json
For this unit, every window ends before analysis_at. The other settings will be used in later labs. Do not change the analysis instant to make an unfinished person appear complete. All provided course tables contain 4–14 people with at least two per group. Duplicate assignment IDs, including conflicting groups, are invalid; choosing whichever row produces a nicer result is not a repair.
A left merge starts from every assignment and attaches matching events. how="left" retains a person with no event. validate="one_to_many" requires each ID to occur at most once in the assignment table while allowing several event rows. An unrelated event ID cannot add a person to the assigned population.
The comparisons create Boolean masks. In pandas, & combines conditions and | means either condition; put each comparison in parentheses. groupby(...).any() then asks whether any qualifying event exists for each person. Mapping those flags back by ID retains one row per assignment, including legitimate zeros. ValueError explicitly refuses invalid allocation or unfinished follow-up instead of pretending the data can be compared.
cat > outcomes.py <<'PY'
import pandas as pd
def build_people(assignments, events, analysis_at):
a = assignments.copy()
if (a["visitor_id"].isna().any()
or (a["visitor_id"] == "").any()
or a["visitor_id"].duplicated().any()):
raise ValueError("Each visitor needs one unique, nonempty assignment ID.")
if not a["group"].isin(["A", "B"]).all():
raise ValueError("Only groups A and B are valid.")
a["assigned_at"] = pd.to_datetime(a["assigned_at"], utc=True, format="ISO8601")
a["window_end"] = a["assigned_at"] + pd.Timedelta(days=7)
if not (a["window_end"] <= pd.Timestamp(analysis_at)).all():
raise ValueError("Every seven-day follow-up must be complete.")
e = events.copy()
e["occurred_at"] = pd.to_datetime(e["occurred_at"], utc=True, format="ISO8601")
joined = a.merge(e, on="visitor_id", how="left", validate="one_to_many")
eligible = ((joined["kind"] == "signup")
& (joined["occurred_at"] >= joined["assigned_at"])
& (joined["occurred_at"] < joined["window_end"]))
flags = eligible.groupby(joined["visitor_id"]).any()
a["signed_up"] = a["visitor_id"].map(flags).astype(int)
return a[["visitor_id", "group", "signed_up"]]
PY
An unmatched left-merge event has missing event fields. Its eligibility comparisons are false; that person therefore gets outcome 0. This does not fill an unknown assignment or an incomplete follow-up with zero.
The next command deliberately misspells a stage. It should print an invalid choice message and exit with status 2. This is a command-argument error; it has not invalidated or changed your data.
python3 run.py pepole
Correct the stage name and save the person-level table:
python3 run.py people
python3 show_results.py people
There should be 12 distinct IDs. Check P-8 (two qualifying events but outcome 1), Q-3 (no events but outcome 0), M-90 (signup exactly at the excluded upper boundary, outcome 0), and K-4 (one second before that boundary, outcome 1). X-02 signed up before allocation and remains 0. OUTSIDE must not appear.
Now reuse your comparison function on this real saved table:
python3 run.py summary
python3 show_results.py summary
The full experiment has 1 of 6 people signed up in A and 5 of 6 in B, a difference near +66.67 percentage points. You now have a defensible denominator and window, not yet a claim that a new page caused an improvement. The next labs examine how random allocation can vary the observed difference.
The checks read your saved work and call your unchanged functions only on isolated copies within the same public contract. They cover independent input row orders, other IDs, no-event inputs and invalid allocation/follow-up. Equivalent pandas or Python implementations are welcome if build_people returns the same named DataFrame columns and summarize returns the documented objects. Repeated checks do not rebuild the live results or change your progress.
Summary
You compared six visitor cards, wrote a reusable rate comparison and built complete seven-day outcomes from raw events. Stable IDs, an explicit UTC window and assigned-person denominators kept repeated clicks and absent events from distorting the result. Your saved person table and summary can now support an honest examination of random variation.
