Introduction
Community Radio has an incoming clip for its evening programme, but the small media disk is full. A maintainer already removed the large access log's name. The station still serves its archive, yet the space has not returned. Deleting valuable recordings would solve the wrong problem.
You will find the file still held by Nginx, use its documented log-reopen signal, and save and play the complete new clip while the original services keep running. Only basic terminal familiarity is assumed. No C, device formatting or storage configuration is required. The prepared audio uses a synthesized English voice for this rehearsal.
Use Workstation for commands and Radio Player to listen. Each lab has its own prepared incident. Work only inside /home/labex/project/radio and signal only its identified station master; unrelated processes belong to the learning platform.
Recover Space without Stopping the Station
In this step, you will connect a full filesystem to a specific deleted file and recover its blocks while keeping the station running.
Hear what must survive
Open Radio Player. Play Archive recording, then Incoming clip. The archive is already on the media disk; the incoming clip is waiting outside it. The third player has no complete saved recording yet. These are synthesized rehearsal voices, not a live microphone or a real interview.
In Workstation, pwd prints your location. ls lists directory entries. It starts in the prepared station directory; if necessary, return with cd:
cd /home/labex/project/radio
pwd
/home/labex/project/radio
ls
media/ is the isolated disk. incoming/ is on the instance's normal filesystem. services/ contains the two course-owned Nginx configurations and master PID files. media.ext4 is the fixed-size backing file prepared by LabEx; leave it unchanged. You will not create filesystems or edit mount settings.
Confirm the failure and the boundary
findmnt -T shows the filesystem containing a path. The target should be the exact media directory, its source a /dev/loop... device, and its type ext4. The root filesystem has a different source. A loop device presents this course's bounded disk image as a block device.
findmnt -T media -o TARGET,SOURCE,FSTYPE
findmnt -T incoming -o TARGET,SOURCE,FSTYPE
cp copies a file. The following copy is deliberately expected to fail with No space left on device. It may leave an incomplete destination. Do not run it with sudo, which is unnecessary for these files.
cp incoming/evening-recording.wav media/recordings/evening-recording.wav
This is a real capacity failure on the small media filesystem. The original archive stays intact. df -h reports filesystem block capacity in readable units. du -xsh totals allocated blocks reachable through names beneath one directory: -x stays on that filesystem, -s gives a summary, and -h uses readable units.
df -h media
du -xsh media
df shows nearly all capacity used while du sees relatively little. These commands measure different things. This difference alone does not identify the cause: filesystem metadata, reserved blocks, permissions, other mount views and other factors can matter too.
Find the actual holder
A directory entry maps a name to an inode, the filesystem's record for a file. A hard-link count tells how many names refer to that inode. A process can also hold an open file descriptor, a numbered reference to the open file. Removing the last name reduces the link count to zero; the file's blocks remain until the last open reference closes.
lsof lists open files. We use sudo for this read-only inspection because Nginx has a root master. -nP avoids hostname and port-name lookups, +L1 selects link counts below one, +f -- media selects the filesystem containing media, and -a intersects those selections. Keep this filesystem filter: a global list can include unrelated deleted memory files and platform processes.
sudo lsof -nP -a +L1 +f -- /home/labex/project/radio/media
Expect two Nginx rows for media/logs/access.log (deleted): the master and its worker. Read PID, FD, DEVICE, SIZE/OFF, NLINK and NODE. The rows share device and inode, with NLINK zero; their sizes describe the same file, so do not add them together. An FD such as 4w means descriptor 4, open for writing. The device/inode identify this filesystem's file, even though its original name is gone.
cat displays text. Read the station's actual configuration and master PID file. The pid and access_log paths must match this station and the deleted media log:
cat services/studio/nginx.conf
cat services/studio/nginx.pid
MASTER below is a shell variable. $(...) captures command output, and double quotes keep its value as one argument. ps displays the selected process; -o chooses columns and -p selects that PID, and --ppid also selects its children. Confirm it is the master from the filtered lsof rows, with this course's configuration path. The other row is a worker whose parent PID is this master.
MASTER=$(cat services/studio/nginx.pid)
ps -ww -o pid,ppid,user,args -p "$MASTER" --ppid "$MASTER"
Do not act if the mount, path or master identity differs. Recheck the directory and configuration rather than guessing a PID or signaling all Nginx processes.
Ask this program to reopen
A signal's meaning depends on the program. Nginx documents USR1 as reopen log files. It closes the old log references and opens the configured name again; TERM would stop it, and HUP has different reload semantics. Read the installed manual with man nginx, find SIGNALS, then press q to return. The same contract is in the Nginx control documentation.
man nginx
kill can send a named signal; it does not always mean termination. sudo is required to signal this root master. Send only the documented signal to the master you just confirmed:
sudo kill -USR1 "$MASTER"
Now inspect space, holders and the master again. Reopening is asynchronous; if a holder is briefly still listed, repeat these read-only observations after a moment.
df -h media
sudo lsof -nP -a +L1 +f -- /home/labex/project/radio/media
ps -ww -o pid,ppid,user,args -p "$MASTER" --ppid "$MASTER"
Space should return, the deleted media log rows disappear, and the same master PID remains. An empty filtered lsof result is expected here. You did not stop the station or free another filesystem's file.
curl -fsS reads a URL, fails on HTTP errors and prints a concise error if needed. This normal station request must still work. tail -n 2 then shows the last two lines of the newly named access log:
curl -fsS http://127.0.0.1:8080/on-air.txt
tail -n 2 media/logs/access.log
Expect Community Radio: archive and live programme remain available. and a new GET /on-air.txt log entry. The original process now writes the new named log. If space did not return, inspect the filtered holders again; do not delete the archive, truncate descriptors, stop services or recreate the mount. If the new log is absent, check the unchanged configuration and repeat the documented reopen for the correct master.
Save and Use the Complete Recording
In this step, you will replace the failed partial copy with the entire incoming clip, verify its bytes, and use it through the running station.
The incoming source and the archive have different roles. Preserve both. Rerunning cp to the same destination replaces the partial destination from the failed attempt; it does not modify the source.
cp incoming/evening-recording.wav media/recordings/evening-recording.wav
cmp compares every byte and is silent when files match. Check immediately after copying:
cmp incoming/evening-recording.wav media/recordings/evening-recording.wav
sha256sum prints a content fingerprint. Matching hashes offer another visible comparison; the grader also checks the original archive and source against the prepared bytes.
sha256sum incoming/evening-recording.wav media/recordings/evening-recording.wav
Both lines must have the same hash. A zero-filled or sparse file with a plausible name or size is not a recording. A file copied onto the wrong filesystem also leaves the station's media delivery incomplete.
findmnt -T media/recordings/evening-recording.wav -o TARGET,SOURCE,FSTYPE
It must still be the same isolated ext4 media mount. Check that the station can serve the actual saved file. -o tells curl to save a downloaded response outside the media disk; compare that response with the incoming clip:
curl -fsS http://127.0.0.1:8080/recordings/evening-recording.wav -o /tmp/radio-delivered.wav
cmp incoming/evening-recording.wav /tmp/radio-delivered.wav
Return to Radio Player and reload. Play Your saved recording, seek near the end, and listen to the closing sentence. Play Archive recording again. Download the saved recording using Download saved recording. Follow Read the on-air message; it should still display the station message. Browser audio needs a user play action; saving bytes alone does not start playback.
If the saved player reports a format or network error, compare the source and destination again and check the URL. A full-disk attempt can leave a truncated WAV header or data; after recovery, recopy the unchanged source. If comparison fails, do not repair audio headers or change the source to match the broken destination.
df -h media
The filesystem should now have ample free space and the complete saved clip. The course's checks only read your final files, mount identity, open descriptors and actual service state. They do not send signals, copy audio, repair files or create a recording for you.
Summary
You recovered real disk capacity by separating names, inodes and open descriptors. A filesystem-filtered investigation connected the deleted log to the correct Nginx master; its documented USR1 signal released the old references without replacing that process. You then saved, compared, downloaded and played the complete clip while keeping the archive and station services intact.
A df/du difference is a clue, not a diagnosis. Reopen behavior is specific to the program: consult its documentation before sending any signal. This rehearsal uses an isolated ext4 disk and synthesized voice clips; it does not teach device formatting, arbitrary-process termination or a complete log-rotation policy.
