Replace Only Verified Downloads

Introduction

Field Notes knows which resources are damaged or missing. Now the organizer needs a repair routine: fetch only those resources, compare their bytes with an independent reference, and publish each replacement only after it passes. A source can return an HTTP error or a perfectly successful HTTP response containing the wrong document.

This lab starts with its own prepared pack. Bring the previous lab's knowledge of SHA256, line-safe reads, command substitution and full filenames; no previous VM files are needed. Workspace and Download Desk share one container. The original synthetic PDF and curl are ready. The time is an author estimate including reading and recovery.

Download One Candidate Before Replacing Anything

In this step, you will save and verify a real handout in a separate practice directory.

Start in Workspace, prepared for UTF-8 text. Use this absolute path whenever a relative filename cannot be found. pwd and ls pack help distinguish the wrong directory from a mistyped name.

cd /home/labex/project/download-desk

The trusted manifest is /opt/labex/download-pack-state/expected.sha256. Its format is the one taught in the first lab: 64 lowercase hexadecimal characters, two spaces, and a complete flat filename. It is independently supplied, readable, nonempty and newline terminated. Names use printable ASCII and may contain spaces; slash, backslash, CR, LF, tabs, . and .. names are outside the scope. Use actual directories without linked parents. An expected member must be absent or a regular local file; links and directory objects are refused. Do not edit the reference or source. A matching SHA256 proves agreement with this trusted reference, not publisher identity.

cat /opt/labex/download-pack-state/expected.sha256

The prepared trial directory is for experiments, separate from the organizer's pack. mktemp creates a unique empty file from a template ending in X characters and prints its path. Capturing it in candidate keeps that path available in this Workspace session. A new connection may lose shell variables; if so, start this practice sequence again rather than guessing the old variable's value.

candidate=$(mktemp trial/.incoming.XXXXXX)

curl makes an HTTP request. Here its options have separate jobs:

• --get sends a GET request, and --data-urlencode 'name=Workshop card.txt' safely encodes the filename as a query parameter. Quoting alone protects shell arguments; URL encoding handles spaces inside the request.
• --fail makes HTTP errors such as 503 produce a nonzero exit status. It cannot detect a wrong body returned with HTTP 200.
• --silent --show-error hides progress while keeping error messages. --max-time 8 bounds the whole transfer to eight seconds.
• --output "$candidate" writes only into the temporary candidate. 127.0.0.1:8080 is the prepared local source in this same container; /good serves the correct bytes.

curl --fail --silent --show-error --get --data-urlencode 'name=Workshop card.txt' --max-time 8 --output "$candidate" http://127.0.0.1:8080/good

printf 'Download exit: %s\n' "$?"

Expect Download exit: 0. If you mistyped the URL, curl may report a connection failure or HTTP rejection. Correct the address above; remove only your current candidate with rm -- "$candidate", then repeat from mktemp. -- ends options, and this exact variable identifies your own experiment.

Copy the expected handout digest from its trusted manifest line. It is the value below, independent of the candidate:

expected=a9986fb03d5ca72e42753d0ddd11dce7794b0e5a8b253dd4cd356825ca6d0583

A pipe, |, connects one command's standard output to the next command's standard input. The following printf builds a checksum record naming your candidate. sha256sum --check reads that record, and && runs mv only if the check succeeds. mv -- renames the verified candidate to the final practice filename. This practice file may be replaced when you repeat this experiment; the organizer's selective-repair rules apply to pack.

printf '%s  %s\n' "$expected" "$candidate" | sha256sum --check && mv -- "$candidate" "trial/Workshop card.txt"

Expect a line ending in : OK. If it reports FAILED, do not move that candidate: remove only that file and repeat the corrected download. The verified handout can now be read:

cat "trial/Workshop card.txt"

REPAIR DESK: EAST HALL

You have kept an unverified download away from a final filename. Next you will apply that order to the whole pack.

Repair Selectively and Recover from Failure

In this step, you will give the routine three arguments and test both its useful result and its failure boundary.

Start in Workspace, in /home/labex/project/download-desk. The script contract is bash repair-pack.sh MANIFEST SOURCE_URL PACK_DIRECTORY. It is a standalone Bash program using those arguments and installed commands, without neighboring helper files. It must classify the actual member bytes first. A matching member makes no download request and preserves its bytes, mode, owner and mtime. The unrelated pack/notes.txt has the same protection. Only missing or mismatched members may be downloaded, once each per invocation. A matching empty member is correct. A complete second run must make zero download requests and preserve the whole pack state.

Each candidate lives beside its destination and is removed after normal success or a reported failure. Publication is per file: a verified replacement may remain if a later member fails. Failed or wrong candidates never replace their destination. This is not a whole-pack transaction or a guarantee against concurrent writers, power loss or every process-termination condition. Use only the course-owned paths, with no concurrent edits.

The parser and digest extraction below use the syntax taught in the previous lab. These additional pieces protect the replacement boundary:

• [ -r "$manifest" ] checks readability; [ -s "$manifest" ] requires a nonempty manifest. ! negates a test. || means OR between tests, or runs a failure branch when a preceding command fails. && means AND here. Braces group related tests without creating a subshell.
• [ -d "$pack" ] checks for a directory. [ -L "$target" ] detects even a dangling link; -e tests an existing target and -f a regular file. Refusing links and other object kinds before mv avoids following a source link or accidentally moving a candidate inside a directory.
• temporary= starts an empty variable. trap '...' EXIT registers cleanup when this script exits. Single quotes delay expansion until cleanup runs. -n means the variable is nonempty; cleanup removes only the file the script itself created. rm -f quietly accepts an already-absent candidate; -- ends options.
• continue advances to the next manifest line. exit 1 reports failure. >&2 sends an explanation to standard error rather than ordinary output. Clearing temporary after a successful rename prevents cleanup from touching a published file.

Save this complete script using the literal here-document method from the first lab:

cat > repair-pack.sh <<'BASH'
#!/usr/bin/env bash
manifest=$1
base=$2
pack=$3
if [ ! -r "$manifest" ] || [ ! -s "$manifest" ]; then
    printf 'A readable nonempty trusted manifest is required.\n' >&2
    exit 1
fi
if [ ! -d "$pack" ] || [ -L "$pack" ]; then
    printf 'A real pack directory is required.\n' >&2
    exit 1
fi
temporary=
trap 'if [ -n "$temporary" ]; then rm -f -- "$temporary"; fi' EXIT
while IFS= read -r line; do
    expected=${line:0:64}
    name=${line:66}
    target=$pack/$name
    if [ -L "$target" ] || { [ -e "$target" ] && [ ! -f "$target" ]; }; then
        printf 'Expected a regular file or absent member: %s\n' "$name" >&2
        exit 1
    fi
    if [ -f "$target" ]; then
        actual=$(sha256sum -- "$target")
        actual=${actual%% *}
        if [ "$actual" = "$expected" ]; then
            printf 'Keep: %s\n' "$name"
            continue
        fi
    fi
    temporary=$(mktemp "$pack/.repair.XXXXXX") || exit 1
    if ! curl --fail --silent --show-error --get --data-urlencode "name=$name" --max-time 8 --output "$temporary" "$base"; then
        printf 'Download failed: %s\n' "$name" >&2
        exit 1
    fi
    actual=$(sha256sum -- "$temporary")
    actual=${actual%% *}
    if [ "$actual" != "$expected" ]; then
        printf 'Checksum mismatch: %s\n' "$name" >&2
        exit 1
    fi
    mv -- "$temporary" "$target" || exit 1
    temporary=
    printf 'Repaired: %s\n' "$name"
done < "$manifest"
BASH

Cleanup receives only your exact temporary path; it does not remove other people's files. The same contract permits another safe temporary name or equivalent loop. The checker tests behavior, not these command spellings or the Keep: output text.

Try a successful HTTP response with the wrong body

The controlled /html endpoint returns an HTML error page with HTTP 200. This is a real response, not a simulated curl status. The next command is expected to fail, without replacing the existing damaged handout:

bash repair-pack.sh /opt/labex/download-pack-state/expected.sha256 http://127.0.0.1:8080/html pack

printf 'Repair exit: %s\n' "$?"

Expect Repair exit: 1 and Checksum mismatch: Workshop card.txt. The PDF was already correct and was skipped. The wrong candidate was removed; HTTP success alone could not authorize publication. ls -a includes hidden names, so you can see that no .repair.* candidate remains:

ls -a pack

Now use the correct source. The script repairs the handout and missing map while keeping the PDF, empty credits file and personal note:

bash repair-pack.sh /opt/labex/download-pack-state/expected.sha256 http://127.0.0.1:8080/good pack

(cd pack && sha256sum --check /opt/labex/download-pack-state/expected.sha256)

Every listed member should say OK. Switch to Download Desk, reload it, and open the PDF and map. The map names the east entrance. Select Download beside the PDF and open that saved file in your browser's reader. Both operations consume your local pack; there is no fallback to the source. Use native zoom controls for fixed PDF pages. Return to Workspace.

Repeat without downloading again

The prepared source writes one line per actual resource request to requests.jsonl, including the source process identity. It flushes each event before observation. wc -l < FILE counts lines while redirecting the file to the command; it does not request a resource. Read the count, rerun the routine, and read it again:

wc -l < /opt/labex/download-pack-state/requests.jsonl

bash repair-pack.sh /opt/labex/download-pack-state/expected.sha256 http://127.0.0.1:8080/good pack

wc -l < /opt/labex/download-pack-state/requests.jsonl

The two numbers must match. All four members should be kept. Downloading everything again would waste requests and could change correct files' modes or timestamps even if the final digests matched.

Practice transport failures away from the finished pack

mkdir -p creates an experiment directory if needed. It does not reset it. Keep the finished organizer's pack intact:

mkdir -p trial/fault-test

The controlled /error endpoint returns HTTP 503. Expect a curl error mentioning 503 and a nonzero script status:

bash repair-pack.sh /opt/labex/download-pack-state/expected.sha256 http://127.0.0.1:8080/error trial/fault-test

printf 'HTTP-error exit: %s\n' "$?"

Expect HTTP-error exit: 1. The controlled /partial endpoint closes the connection after only part of its declared body. Expect curl error 18 about bytes remaining, and again no published candidate:

bash repair-pack.sh /opt/labex/download-pack-state/expected.sha256 http://127.0.0.1:8080/partial trial/fault-test

printf 'Partial-transfer exit: %s\n' "$?"

Expect Partial-transfer exit: 1. A connection failure, HTTP rejection and wrong HTTP-200 body are distinct observations; all prevent an unverified replacement. Recover the experiment using the same correct endpoint and arguments:

bash repair-pack.sh /opt/labex/download-pack-state/expected.sha256 http://127.0.0.1:8080/good trial/fault-test

If a manifest or directory path is misspelled, the script reports the required object and exits nonzero. Check pwd, ls and the absolute reference path before retrying. If a saved script has a syntax error, replace only repair-pack.sh with the complete here-document above. Do not delete the organizer's pack to recover a typo.

The final check reads your real pack and preserved objects. It also runs your unmodified script on isolated packs with different names and controlled HTTP faults, including an existing bad target and a missing legitimate empty file. Those fixture requests come from your script, not a grading download that fills in missing evidence. Repeating the check does not repair or reset your live work.

Summary

You downloaded a candidate, checked its SHA256 before publishing it, and built a reusable selective repair script. Real HTTP errors, wrong successful responses and interrupted bodies stayed away from final filenames. Correct files and personal notes survived, and a complete second run needed no downloads.