Introduction
A community exhibition needs a useful comparison of its opening slots. More visitors in total can simply mean more opportunities to record visitors. You will first compare two equally observed slots, then build a reusable analysis that preserves what was open, what was recorded, and what remains unknown.
This lab assumes Python basics, pandas reading, filtering and grouping, and Matplotlib bar charts. Python and the libraries are ready in the WebIDE environment. All counts are synthetic; a row represents one hour, not a visitor identity. The time is an author estimate including reading and ordinary errors. You will create real CSV and PNG files, with every original input preserved.
Compare Two Equally Observed Slots
In this step, you will calculate a rate by hand before drawing the first chart. In the WebIDE Tab, use the terminal at the bottom. If it is hidden, open the top-left menu and choose Terminal → New Terminal. Commands run in this terminal; files appear in the Explorer at the left.
Enter the prepared workspace. cd changes this terminal's working directory; it does not change other terminals. pwd prints the current directory. If a later command says a file is missing, return here and check the spelling with ls.
cd /home/labex/project/opening-hours
pwd
/home/labex/project/opening-hours
cat prints a text file without changing it. The CSV has two one-hour observations per slot.
cat data/equal.csv
slot,visitors
morning,6
morning,10
evening,8
evening,12
Morning has 16 visitors across two recorded hours: 16 / 2 = 8 visitors per hour. Evening has 20 across two hours: 20 / 2 = 10. These denominators are equal, so both total and hourly comparisons favor evening. A rate retains its unit: visitors per recorded hour.
Create the small script below. The shell's > replaces the named file. <<'PY' begins a here-document: every line until a line containing only PY is saved literally. Paste the complete block, including its last line. This creates the file immediately; there is no editor save action. agg names the result columns, while size counts the observed hours. index=False keeps pandas' row index out of the CSV. Agg lets Matplotlib save a PNG without opening a separate window.
cat > equal.py <<'PY'
import pandas as pd
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
data = pd.read_csv('data/equal.csv')
summary = data.groupby('slot').agg(visitors=('visitors', 'sum'), hours=('visitors', 'size'))
summary['rate'] = summary['visitors'] / summary['hours']
summary.to_csv('results/equal-summary.csv')
summary['rate'].plot.bar(color=['#256b5d', '#b35828'], rot=0)
plt.ylabel('Visitors per recorded hour')
plt.title('Equal recording opportunities: two hours each')
plt.tight_layout()
plt.savefig('results/equal.png', dpi=150)
plt.close()
print(summary)
PY
python3 equal.py
visitors hours rate
slot
evening 20 2 10.0
morning 16 2 8.0
The rows are sorted by slot name; leading whitespace in the display can vary. Open opening-hours/results/equal.png in the Explorer to view the saved chart. Its two vertical bars should show 10 and 8 visitors per recorded hour. Keep each slot name on its horizontal tick and the vertical axis label Visitors per recorded hour; colors, fonts, figure size and slot ordering may differ. The PNG is your real output, not a prepared target image. The chart must be deterministic: rerunning your current plotting script with unchanged input must produce the same image pixels. Avoid random styles or timestamps. PNG metadata may differ; your own colors, fonts and size remain allowed.
For a normal filename error, first note that Python reports a missing file and stops; it has not reset your work. The following failure is intentional:
python3 missing.py
You should see can't open file and No such file or directory. Recover by listing the workspace and running the real filename again.
ls
python3 equal.py
Self-check: both slots use two observed hours. You have not yet assumed that all future data will have equal recording opportunities.
Preserve Every Planned Hour
In this step, you will match visitor counts to their planned hours. Continue in the WebIDE terminal at /home/labex/project/opening-hours. Each plan row is one hour on one date in one slot. open=1 means planned open; open=0 means closed. The pair date and slot is a composite key: neither column alone identifies a row. Dates use YYYY-MM-DD; all dates in these files are valid calendar dates.
cat data/plan.csv
cat data/counts.csv
The count file is deliberately in a different order. A count of 0 is a real observation of no visitors. An absent count for an open planned hour means missing recording, not zero visitors. Closed hours have no count row. Every count must match an open plan row; both files must have complete, unique composite keys. Do not deduplicate or invent counts to hide an invalid key.
A left merge keeps every row from the left table: the plan. It finds counts using both key columns, regardless of row positions. Missing matches get pandas' missing value, displayed as NaN. Check key completeness before merging: pandas can match missing keys to each other, which would not identify a real hour. duplicated(keys) detects repeated key pairs, and validate='one_to_one' asks merge to reject duplicate relationships.
The script uses .loc[condition, column] to assign only matching rows. .eq(0) and .eq(1) compare each value with zero or one; & combines conditions, and notna() distinguishes a present count from a missing value. The closed mask runs first, then the open-and-recorded mask, so a real zero is observed.
Replace the prepared observe.py with the complete script below. sys.argv[1] and [2] read the two path arguments after the script name. A Path joined with / builds a file path. mkdir(parents=True, exist_ok=True) creates the output directory if necessary, so the same program can analyze another input directory. The two input CSV files remain read-only.
cat > observe.py <<'PY'
import sys
from pathlib import Path
import pandas as pd
source = Path(sys.argv[1])
output = Path(sys.argv[2])
plan = pd.read_csv(source / 'plan.csv')
counts = pd.read_csv(source / 'counts.csv')
keys = ['date', 'slot']
if plan[keys].isna().any().any() or counts[keys].isna().any().any():
raise ValueError('Every date and slot must be present')
if plan.duplicated(keys).any() or counts.duplicated(keys).any():
raise ValueError('Every date/slot key must be unique')
if not plan['open'].isin([0, 1]).all():
raise ValueError('open must be 0 or 1')
if counts['visitors'].isna().any() or (counts['visitors'] < 0).any():
raise ValueError('Recorded counts must be nonnegative')
known = counts.merge(plan, on=keys, how='left', validate='one_to_one')
if known['open'].isna().any() or (known['open'] != 1).any():
raise ValueError('Counts must match an open planned hour')
hours = plan.merge(counts, on=keys, how='left', validate='one_to_one')
hours['status'] = 'missing'
hours.loc[hours['open'].eq(0), 'status'] = 'closed'
hours.loc[hours['open'].eq(1) & hours['visitors'].notna(), 'status'] = 'observed'
hours['valid_hour'] = hours['status'].eq('observed').astype(int)
output.mkdir(parents=True, exist_ok=True)
hours.sort_values(keys).to_csv(output / 'hours.csv', index=False)
print(hours[['date', 'slot', 'visitors', 'status']].head())
PY
The first mask marks closed hours. The second marks open hours with a count as observed, including real zero. & combines two elementwise conditions, and notna() asks whether a value is present. valid_hour converts the observed condition to 1 or 0; summing it will count usable observation hours. No missing count is filled with zero.
python3 observe.py data results
head -n 10 results/hours.csv
head -n 10 prints the header and first nine data rows. The saved CSV includes date,slot,open,visitors,status,valid_hour. Its order is sorted by the composite key. Inspect it in the Explorer too: quiet has recorded zeros with observed and valid_hour=1; unlogged has blank visitor counts with missing and valid_hour=0; closed has blank counts with closed. Blank means unknown or inapplicable according to its status, never a guessed zero.
Self-check: every plan row appears exactly once in hours.csv, and every recorded count remains attached to its own date and slot. If a ValueError occurs, read which relationship is invalid. Check your script against this block and the untouched source files; rerun after correcting the script. Reopening the terminal is unnecessary.
Calculate Rates and Recording Coverage
In this step, you will summarize each slot without changing what an unknown count means. Continue in the same WebIDE terminal and workspace.
For evening, the plan contains four open hours, and three have recordings of 12 visitors each. The total is 36, the observed hours are 3, the hourly rate is 36 / 3 = 12, and coverage is 3 / 4 = 0.75, or 75% of planned open hours recorded. Missing hours do not enter the rate denominator; they still reduce coverage. morning has 40 visitors in four observed hours: its higher total has a lower rate, 10.
For quiet, four recorded zeros give total 0 and rate 0 with full coverage. For unlogged, no hour is observed: coverage is 0 but total and rate remain blank. For closed, planned hours are 0: both coverage and rate are undefined and remain blank. An undefined value must not become a zero-height rate bar later.
Append the following block once to observe.py. Here >> appends rather than replacing. It uses the hours and output variables created above. sum(min_count=1) returns a missing total when every value is missing, instead of the default empty sum of zero. replace(0, float('nan')) makes zero denominators explicitly missing before division. These operations preserve a real observed zero numerator.
cat >> observe.py <<'PY'
grouped = hours.groupby('slot')
summary = grouped.agg(planned_hours=('open', 'sum'), observed_hours=('valid_hour', 'sum'))
summary['visitors'] = grouped['visitors'].sum(min_count=1)
summary['coverage'] = summary['observed_hours'] / summary['planned_hours'].replace(0, float('nan'))
summary['rate'] = summary['visitors'] / summary['observed_hours'].replace(0, float('nan'))
summary.to_csv(output / 'summary.csv')
print(summary)
PY
python3 observe.py data results
cat results/summary.csv
slot,planned_hours,observed_hours,visitors,coverage,rate
afternoon,3,3,36.0,1.0,12.0
closed,0,0,,,
evening,4,3,36.0,0.75,12.0
morning,4,4,40.0,1.0,10.0
quiet,4,4,0.0,1.0,0.0
short,4,1,20.0,0.25,20.0
sparse,5,3,27.0,0.6,9.0
unlogged,4,0,,0.0,
CSV numeric formatting such as 36 versus 36.0 can vary without changing the result. The required columns and their meanings stay the same. short has the highest observed rate but only one observed hour. sparse has three observations but covers just 60% of its planned hours. These are reasons to ask about evidence quality before making a recommendation; this lab has not declared either slot a winner.
Open results/summary.csv in the Explorer. Keep data/plan.csv, data/counts.csv, data/equal.csv, and your scripts. Your reusable contract is python3 observe.py INPUT_DIRECTORY OUTPUT_DIRECTORY; both CSV files are read from the input, and both hours.csv and summary.csv are written to the output. Equivalent calculations and output row ordering are allowed when all keys and numeric relationships are retained. If you accidentally replaced the script while appending, recreate the step 2 script and append this step's block once, then rerun it.
Self-check: an unobserved slot and a truly empty observed slot have different totals, denominators and rates. The next lab will use those distinctions to decide when evidence is sufficient to compare slots.
Summary
You created an equal-opportunity chart, matched counts to complete composite keys, and produced hourly and slot-level CSV files. Closed hours, missing recordings and real zeros remain distinct. Rates use recorded open hours; coverage reveals how much of the planned opportunity was actually recorded.
Keep the reusable analysis and original data. These descriptive historical rates do not prove what would happen after extending opening hours. In the next independent environment, you will apply an explicit evidence rule before making a recommendation.
