Introduction
Security tools can produce long outputs containing notes, repeated observations, and diagnostics. A useful result is a small set of relevant findings whose origin you understand.
In this lab, you will practice with a supplied text file before running any scanning tool. You will read its contents, filter and deduplicate open-service records, count them, and keep errors separate from useful output.
Read a Prepared Output File
In this step, you will inspect a prepared text file in several ways before extracting findings. This is sample data, not output from a scan you need to run.
Open the prepared Kali shell:
kali
Use an absolute path to choose the shared working directory:
cd /home/labex/project
The file observations.txt contains short lines beginning with NOTE, OPEN, or CLOSED. Here OPEN means that the sample observation lists a service as open; it does not prove a vulnerability. Print the full small file:
cat observations.txt
You will see repeated open-service lines as well as notes and closed-service lines. Reading the source first helps you choose a filter that preserves the intended information.
For a long file, less displays a page at a time without changing the file:
less observations.txt
Use the arrow keys to move if needed, then press q to quit and return to the shell. The sample is small enough to fit on one screen; the same pager works for larger tool outputs.
head reads the beginning of a file; tail reads its end. The -n option sets how many lines to display. Inspect the first three lines:
head -n 3 observations.txt
Expected output:
NOTE supplied training observations
OPEN training-web:8080 http
CLOSED training-db:5432 postgresql
Now inspect the last two lines:
tail -n 2 observations.txt
The last line is NOTE end of sample. Save the three-line preview using the > operator introduced in the first lab:
head -n 3 observations.txt > preview.txt
cat preview.txt
The preview should contain the same three lines shown above. You kept the source intact and saved a useful view of it.
Filter, Sort, and Remove Repeated Findings
In this step, you will build a pipeline that keeps open-service records and removes duplicates.
Stay in Kali and return to the shared workspace:
cd /home/labex/project
grep prints matching lines. Its pattern below is in single quotes so the shell passes it literally. The ^ anchor means the start of a line, and the space after OPEN separates the record type from its data:
grep '^OPEN ' observations.txt
The output includes every open-service observation, including repeated lines. Notes and CLOSED lines are absent.
The pipe symbol | connects one command's standard output to the next command's standard input. sort orders lines so identical ones become adjacent:
grep '^OPEN ' observations.txt | sort
The repeated lines are now grouped. uniq removes adjacent identical lines, so it belongs after sorting when duplicate lines might be separated:
grep '^OPEN ' observations.txt | sort | uniq
Expected output:
OPEN training-dns:5353 dns
OPEN training-ssh:2222 ssh
OPEN training-web:8080 http
Each command has one job: select records, order them, then collapse repeats. Save the resulting stream to a separate output file:
grep '^OPEN ' observations.txt | sort | uniq > open-services.txt
cat open-services.txt
You should see the same three unique records. Redirection applies to the final output of the pipeline; the original observations are not edited.
Count Results and Separate Error Output
In this step, you will count the selected findings and distinguish normal output from an error message.
Start from the shared workspace:
cd /home/labex/project
wc counts text. Its -l option counts newline-terminated lines. Because the saved file has one finding per line, the count is the number of distinct open-service records:
wc -l open-services.txt
The output reports 3 and the filename; spacing can vary. A pipeline supplies text without a filename argument, making it convenient to save only the count:
cat open-services.txt | wc -l > finding-count.txt
cat finding-count.txt
The value is 3, possibly with leading spaces.
Programs normally send useful data to standard output and diagnostics to standard error. The shell's > redirects standard output; 2> redirects standard error instead. To observe the distinction, try to read a deliberately missing file and save its diagnostic. The next command is expected to fail because missing-observations.txt does not exist; this is an intentional demonstration, so continue afterwards:
cat missing-observations.txt 2> read-error.txt
Nothing should appear on screen because the error message went into read-error.txt. Inspect that separate diagnostic:
cat read-error.txt
The message names missing-observations.txt and says it cannot be read because it does not exist. Your actual findings are still separate:
cat open-services.txt
The three valid records remain unchanged. Keeping diagnostics separate prevents an error message from being mistaken for a tool finding.
Summary
You read text with cat, less, head, and tail; selected lines with grep; composed a pipeline with sort and uniq; counted records with wc; and used output and error redirection. These skills help turn verbose tool output into small, inspectable results.
