Introduction
An archive can contain identical pictures under different names, and different pictures with the same size. You will build the byte-reading part of a reusable duplicate finder and save a complete inventory without changing the originals.
You should know Java classes, methods, lists, exceptions, and basic file reading. This walkthrough introduces binary input, automatic resource closure and SHA-256. The prepared directory adapter and command runner are supporting code; you implement the digest. This is a read-only exercise, not a disk cleanup tool. The author estimates 50 minutes, including reading and ordinary corrections.
Read the Bytes, Not the Filename
In this step, you will compare a normal image example, then implement a digest that reads every byte exactly once. Open WebIDE, choose Terminal > New Terminal, and use that integrated terminal throughout. Each lab starts independently with its own files.
cd changes this terminal's working directory. The prepared run.sh compiles src/*.java in a fresh temporary build directory, then runs the named Java class. Its compiler uses -encoding UTF-8 so source text does not depend on the terminal locale. Only its compiler and Java child processes use C.UTF-8; the Java child also uses UTF-8 text defaults, allowing the supported UTF-8 filenames without changing your shell settings.
cd /home/labex/project/duplicates
./run.sh Sample
The helper decodes the real PNG files and compares their bytes. It prints red = renamed: true and red = blue: false. All three images have the same dimensions and byte length. A name or length can narrow a search, but cannot prove equality. Optionally, in the WebIDE Explorer, expand duplicates/data/gallery and click red.png, blue.png and copies/renamed.png to view the supplied images. The required comparison is the Sample decoding and byte result above. These are original synthetic course images; no external assets are required.
The public Digester interface promises one method: digest(Path) returns a hexadecimal string or throws IOException when reading fails. Path identifies a file; an InputStream reads binary bytes without treating them as text. SHA-256 condenses those bytes into a fixed-length candidate fingerprint. A later unit will still compare actual content before declaring duplicates.
Read the small adapter and starter before replacing the starter:
cat src/Digester.java src/Sha256.java
A 1024-byte buffer is reused. read(buffer) returns the number of valid bytes this time, or -1 at the end. The final read may fill only part of the buffer: its remaining bytes retain earlier values. Therefore sha.update(buffer, 0, count) must use that actual count. An empty file makes no updates and still has a valid digest.
try (InputStream ... ) is try-with-resources: Java closes the stream when the block ends, including when reading throws. Closing releases the file resource; it does not promise immediate garbage collection. MessageDigest.getInstance can throw a checked algorithm exception, so the outer catch converts it into the same file-operation error contract. Hexadecimal uses two characters per byte; & 255 treats a signed Java byte as its unsigned value.
The next command writes the complete replacement. > replaces this source file. The quoted here-document marker <<'JAVA' sends the intervening text literally to cat; the final JAVA must stand alone with no indentation. You may instead edit the same file in WebIDE and save it. If the terminal remains at an unfinished here-document prompt after pasting, press Ctrl+C to cancel, then paste the entire replacement command again, including its final marker.
cat > src/Sha256.java <<'JAVA'
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
public class Sha256 implements Digester {
public String digest(Path path) throws IOException {
try {
MessageDigest sha = MessageDigest.getInstance("SHA-256");
byte[] buffer = new byte[1024];
try (InputStream input = Files.newInputStream(path)) {
int count;
while ((count = input.read(buffer)) != -1) {
sha.update(buffer, 0, count);
}
}
StringBuilder hex = new StringBuilder();
for (byte value : sha.digest()) {
hex.append(String.format("%02x", value & 255));
}
return hex.toString();
} catch (NoSuchAlgorithmException error) {
throw new IOException("SHA-256 unavailable", error);
}
}
}
JAVA
Compile and run the same sample again. Compiling checks the new class, while the sample's equality decisions stay unchanged.
./run.sh Sample
If Java reports a line number, reopen src/Sha256.java, compare the braces and spelling with the replacement, save, and rerun. If a path is wrong, use pwd to see your current directory, then repeat the absolute cd above. No VM reset is needed.
Save Every Path and Its Fingerprint
In this step, you will use your digest to create a complete UTF-8 inventory. Continue in the same WebIDE integrated terminal at /home/labex/project/duplicates.
The prepared Inputs.files adapter walks only the supplied directory and closes its directory stream. It rejects symbolic links, special files and paths containing tabs or newlines rather than silently following or skipping them. Only stable ordinary files and directories are supported. Do not change inputs while scanning. Attribute checks detect some changes, but do not freeze the directory or provide a concurrent snapshot guarantee.
The prepared Inventory saves each relative path, size and digest after all reads succeed. Its TSV format uses a tab between columns. The prepared report adapter writes UTF-8 to a new temporary file, then replaces the report path, so even an existing report hardlink cannot overwrite original file bytes. This is not a crash-durability guarantee. The output must be outside the scanned root. reports already exists alongside data, so the report cannot accidentally become its own input. The command uses Inventory ROOT REPORT: the first argument is the directory to scan, and the second is the output file path; do not reverse them.
./run.sh Inventory data/gallery reports/inventory.tsv
Expect Inventory complete: 11 files. cat displays the saved text:
cat reports/inventory.tsv
The header is PATH, SIZE, SHA256. Compare the rows for red.png, copies/renamed.png and copies/third.png: different paths share a digest. blue.png has the same size but a different digest. Both empty files have size zero and the same valid digest. tail.bin has a partial final buffer; its matching copy must still receive the same digest. Same-name north/readme.txt and south/readme.txt remain separate rows.
Now practise an ordinary path mistake. The next command intentionally fails with exit status 2 and Scan failed; it must not produce a new report. A failure is not an empty, successful inventory. Any older report is not evidence for the failed run.
./run.sh Inventory data/missing reports/missing.tsv
Correct the input path and create a new report:
./run.sh Inventory data/gallery reports/recovered.tsv
Read failures have the same rule: keep the error visible and return failure, never silently omit an unreadable file. The checks run your current source on isolated empty, partial-buffer and unreadable fixtures, and read the real inventory. All originals must retain their paths, bytes, permissions, ownership and modification times; do not delete or replace them. Ordinary reads may update access times. You now have candidate facts; equal fingerprints will become candidates for actual byte comparison in the next lab.
Summary
You read binary files with try-with-resources, updated SHA-256 with the actual read count, and saved a complete UTF-8 inventory. Names, sizes and digests are candidate facts; actual byte comparison will confirm duplicate groups.
