Data Stream Redirection

Introduction

Linux commands normally read input from the terminal and write results back to it. The shell can reconnect those data streams to files or other commands. This is called redirection, and it is the foundation of logging, pipelines, background jobs, and shell scripts.

In this lab, you will work with standard input, standard output, and standard error. You will learn when > overwrites, when >> appends, how 2> and 2>&1 handle errors, how a pipe passes data, how tee displays and saves the same output, and when /dev/null is useful.

Understand Standard Input, Output, and Error

In this step, you will create a small input file and connect it to commands through standard input.

Every command starts with three standard streams:

Create and enter a workspace:

mkdir -p /home/labex/project/streams-lab

cd /home/labex/project/streams-lab

Create a three-line input file with a here-document. The shell sends the lines between the two EOF markers into cat, while > stores cat output in input.txt:

cat > input.txt <<'EOF'
alpha
beta
gamma
EOF

Normally, cat input.txt receives a filename as an argument. The < operator instead connects the file to descriptor 0, standard input:

cat < input.txt

You should see the three lines. Use the same input redirection with wc -l, which counts lines:

wc -l < input.txt

The output should be 3. Because the filename is not an argument to wc, only the number is printed.

Redirect and Append Standard Output

In this step, you will redirect normal command output and observe the difference between overwrite and append behavior.

Make sure you are in the stream workspace:

cd /home/labex/project/streams-lab

The > operator connects descriptor 1, standard output, to a file. It creates the file or truncates existing content before writing:

echo "Stream report" > report.txt

cat report.txt

Run another overwrite to see that the previous line disappears:

echo "Input lines: 3" > report.txt

cat report.txt

Now use >>, which opens the destination in append mode:

echo "Status: complete" >> report.txt

cat report.txt

The file should contain exactly two lines: Input lines: 3 followed by Status: complete. Redirection is performed by the shell before the command starts.

Separate Standard Output and Standard Error

In this step, you will run one command that produces both normal output and an error, then store each stream in a different file.

The following ls command receives one existing path and one missing path:

cd /home/labex/project/streams-lab

ls -l input.txt missing.txt

The line describing input.txt is stdout. The No such file or directory message is stderr. Both appear in the terminal by default, but they are separate streams.

Redirect descriptor 1 with > and descriptor 2 with 2>:

ls -l input.txt missing.txt > listing.log 2> errors.log

The command still returns a nonzero exit status because one path is missing, but the shell remains ready for the next command. Inspect the destinations:

cat listing.log

cat errors.log

listing.log should mention input.txt without the error. errors.log should mention missing.txt and the error without the successful listing.

Combine Standard Output and Standard Error

In this step, you will send stdout and stderr to one log and learn why redirection order matters.

First redirect stdout to combined.log, then point descriptor 2 at the current destination of descriptor 1:

cd /home/labex/project/streams-lab

ls -l input.txt missing.txt > combined.log 2>&1

The expression 2>&1 means "send descriptor 2 to the same destination currently used by descriptor 1." Because > combined.log appears first, both streams reach the file.

cat combined.log

The file should contain both the input.txt listing and the missing.txt error.

Bash also supports &> as shorthand for redirecting both streams:

ls -l report.txt absent.txt &> shorthand.log

cat shorthand.log

This second file should contain the successful report.txt listing and the error for absent.txt. The portable > file 2>&1 form is especially common in scripts and service commands.

View and Save Pipeline Output with Tee

In this step, you will use a pipeline and tee to display output while saving the same data to a file.

A pipe, written as |, connects stdout from the command on its left to stdin of the command on its right. Sort the input file alphabetically in reverse order:

cd /home/labex/project/streams-lab

sort -r input.txt

With ordinary output redirection, the sorted lines go to a file and no longer appear in the terminal:

sort -r input.txt > sorted.txt

cat sorted.txt

The tee command copies its stdin to both stdout and a named file. Run the same sort through a pipe:

sort -r input.txt | tee sorted.txt

You should see the sorted lines on screen, and sorted.txt receives the same content. By default, tee overwrites its destination just like >.

The -a option makes tee append. Add one more line while displaying it:

echo "delta" | tee -a sorted.txt

cat sorted.txt

The file should now contain gamma, beta, alpha, and delta. tee is useful when you need live terminal feedback and a saved log at the same time.

Discard Unneeded Data with Dev Null

In this step, you will intentionally discard a stream with /dev/null, preserve useful output with tee, and empty an existing log.

/dev/null is a special device that accepts and discards anything written to it. List one real directory and one missing directory while discarding only stderr:

cd /home/labex/project/streams-lab

ls -l . missing-directory 2> /dev/null

You should see the directory listing but no missing-directory error. Combine this with tee so the visible stdout is also saved:

ls -l . missing-directory 2> /dev/null | tee visible-listing.log

The pipe carries only stdout. Stderr was redirected separately to /dev/null before the pipe received data.

Inspect the saved listing:

cat visible-listing.log

Finally, create a disposable scratch log, then use /dev/null as an empty input source to clear its contents. A separate file preserves the combined-stream evidence from Step 4:

echo "outdated scratch data" > scratch.log
cat /dev/null > scratch.log

Check its size:

ls -l scratch.log

The file should still exist with a size of 0 bytes. Use discarding deliberately: hidden errors can make troubleshooting harder when you actually need diagnostic information.

Summary

You practiced the three standard streams and their file descriptors, connected files to stdin with <, and redirected stdout with > and >>. You separated stderr with 2>, combined it with stdout using 2>&1 and &>, and learned why operator order matters.

You also compared ordinary file redirection with pipelines and tee, used tee -a to append while displaying output, and discarded unneeded data with /dev/null. These patterns prepare you to understand logging, background commands, and shell scripts later in the course.