Introduction
Your retailer now receives files from two stores on two dates. The counting program will process the files you give it; it cannot guess which day you intended or recognize that an old result file should be excluded. A correct program can therefore produce a wrong business answer when its input selection is wrong.
You have learned how to preserve, copy, archive, and export individual files. Here you will move from one file to a batch input, a deliberately selected collection of files. You will keep dates separate, compare file counts and contents, and save a manifest: a list of the exact HDFS files selected for one run.
HDFS stores file contents without understanding retail columns, dates, or CSV headers. You will observe that boundary before arranging the input. This lab prepares its own local files and an example prior result independently; it does not depend on earlier lab state. Use the Terminal.
Separate Stored Text from Its Meaning
In this step, you will inspect the local deliveries and observe that uploading a CSV preserves its header as ordinary text.
Start in your local workspace:
cd /home/labex/project
List the first date's delivery. A date in a directory name is a useful convention, not a data type enforced by HDFS:
ls /home/labex/project/data/2026-10-14
You should see store-east.txt and store-west.txt. Read them in that order with local cat:
cat /home/labex/project/data/2026-10-14/store-east.txt /home/labex/project/data/2026-10-14/store-west.txt
Expected text:
notebook pen
notebook
folder pen
Here each word is a recorded product label. These files are suitable for the word-count workflow you will learn later. Another system has supplied a different format. Read the small CSV preview:
cat /home/labex/project/data/orders-preview.csv
Expected text:
product,quantity
notebook,2
pen,1
CSV uses commas to separate fields. Its first line is a header naming the columns; the other two lines are business records. HDFS does not automatically remove that first line or interpret quantity as an integer.
Keep this preview separate from the product-label batch. Create an HDFS directory for it:
hdfs dfs -mkdir -p /user/labex/batch-inputs/preview
Upload the CSV:
hdfs dfs -put /home/labex/project/data/orders-preview.csv /user/labex/batch-inputs/preview/orders.csv
Read it back:
hdfs dfs -cat /user/labex/batch-inputs/preview/orders.csv
All three lines remain, including the header. Whether a header should be skipped or a number parsed is a processing decision. Mixing this file into the word-label input would change the meaning of the calculation, even though every file is readable.
Organize Deliveries by Date
In this step, you will give each date a separate raw-input directory and upload the two matching store files.
Use a directory layout that makes the distinction between inputs and outputs visible:
batch-inputs/
raw/2026-10-14/ first day's product files
raw/2026-10-15/ next day's product files
preview/ CSV format example
results/previous-run/ prepared output from an earlier run
The previous result is only a fixture for learning input selection. It must not become part of the new raw batch. Create the two daily directories:
hdfs dfs -mkdir -p /user/labex/batch-inputs/raw/2026-10-14 /user/labex/batch-inputs/raw/2026-10-15
The shell's *.txt pattern matches local filenames ending in .txt. In this next command, your shell expands the unquoted local pattern into the two source paths before running the HDFS client. The final argument is an existing HDFS directory:
hdfs dfs -put /home/labex/project/data/2026-10-14/*.txt /user/labex/batch-inputs/raw/2026-10-14/
Now change only the date in both locations to upload the second delivery:
hdfs dfs -put /home/labex/project/data/2026-10-15/*.txt /user/labex/batch-inputs/raw/2026-10-15/
If the destination files already exist, inspect them before deciding to overwrite anything. Keeping a raw input is more useful than deleting it merely to make an upload command run again.
Check each batch with -count, learned in the lifecycle lab:
hdfs dfs -count /user/labex/batch-inputs/raw/2026-10-14 /user/labex/batch-inputs/raw/2026-10-15
Each date should contain one directory and two files. The content-size columns need not agree: the dates contain different records. File count alone cannot prove that you selected the right date, so also list the first date:
hdfs dfs -ls /user/labex/batch-inputs/raw/2026-10-14
Both entries should show the expected store names under 2026-10-14. A few small files make this exercise readable. At much larger scale, many tiny files impose metadata overhead on HDFS; that does not mean every small file must always be merged.
Select and Check One Batch
In this step, you will prepare an exact file manifest for October 14 and compare its input text with the expected local delivery.
Before running a command, predict which directory belongs in the manifest: the whole batch-inputs tree, the whole raw tree, or only raw/2026-10-14? The narrow date directory matches the requested day and excludes the next date, CSV preview, and old results.
The HDFS -ls -C option prints paths without the detailed listing columns. Use it on the selected date:
hdfs dfs -ls -C /user/labex/batch-inputs/raw/2026-10-14
The two paths should end in store-east.txt and store-west.txt. To make the saved list easy to compare, a shell pipeline, written |, passes the first command's output into a second command. sort orders the lines; > saves the result locally:
hdfs dfs -ls -C /user/labex/batch-inputs/raw/2026-10-14 | sort > /home/labex/project/output/input-manifest.txt
Read your manifest:
cat /home/labex/project/output/input-manifest.txt
Expected paths:
/user/labex/batch-inputs/raw/2026-10-14/store-east.txt
/user/labex/batch-inputs/raw/2026-10-14/store-west.txt
You can also use a pattern on the HDFS side. Single quotes preserve *.txt so the HDFS client, rather than your local shell, matches the remote paths:
hdfs dfs -cat '/user/labex/batch-inputs/raw/2026-10-14/*.txt' > /home/labex/project/output/selected-input.txt
For these filenames, HDFS reads the east file before the west file. Save the corresponding local text in that same explicit order:
cat /home/labex/project/data/2026-10-14/store-east.txt /home/labex/project/data/2026-10-14/store-west.txt > /home/labex/project/output/expected-input.txt
Compare the two:
diff /home/labex/project/output/expected-input.txt /home/labex/project/output/selected-input.txt
No differences should appear. As another check, wc -l counts newline-terminated lines in each file:
wc -l /home/labex/project/output/selected-input.txt
The result should begin with 3. These are three text lines containing five product labels, not three orders. The meaning of a record comes from the file format and task, not from HDFS. Your manifest, file contents, and line count together make the input choice explicit.
Summary
You separated a CSV preview from plain product-label files, kept each date's raw inputs in its own directory, and selected one batch using a manifest. You checked both filenames and contents, rather than trusting an upload or a file count alone.
In the next challenge, you will organize a new delivery using the file-management and input-selection skills you have already practiced.
