Linux Output Multiplexing

Introduction

The tee command in Linux is a powerful utility that allows you to view command output in the terminal while simultaneously saving it to a file. This functionality is particularly useful for system administrators and developers who need to both inspect output in real-time and preserve it for later analysis.

This lab will introduce you to the fundamentals of output multiplexing using the tee command. You will learn how to display output and save it simultaneously, redirect standard error, and append output to existing files. These skills are essential for effective log management, debugging, and system monitoring in Linux environments.

Understanding the Basics of the tee Command

The tee command in Linux takes its name from the T-shaped pipe fitting in plumbing, which splits water flow in two directions. Similarly, the tee command splits the output of a command in two directions: to the terminal and to a file.

Let's start by navigating to our working directory:

cd /home/labex/project

Creating a Test File

First, let's create a simple text file that we can use for our experiments:

echo "Hello, World" > hello.txt

This command creates a file named hello.txt containing the text "Hello, World".

Using the tee Command

Now, let's use the tee command to display the content of hello.txt on the terminal while also copying it to another file:

cat hello.txt | tee copy.txt

You should see the following output in your terminal:

Hello, World

The cat hello.txt part reads the content of the hello.txt file. The pipe symbol | takes that output and sends it to the tee command. The tee command then displays the text on the terminal and simultaneously writes it to the copy.txt file.

Let's verify that the content was indeed copied to copy.txt:

cat copy.txt

You should see the same output:

Hello, World

Understanding the Command Structure

The basic syntax of the tee command is:

command | tee filename

Where:

command is any command that produces output
| (pipe) sends the output of the command to tee
tee is the command itself
filename is the name of the file where the output will be saved

Redirecting Standard Error with tee

In Linux, programs can output information through two main channels:

Standard output (stdout): Normal program output
Standard error (stderr): Error messages and diagnostics

By default, the tee command only captures standard output. However, in many situations, you may want to capture error messages as well. This is where output redirection comes in.

Understanding Standard Error Redirection

Let's create a script that outputs both standard output and standard error. Navigate to the project directory first (if you're not already there):

cd /home/labex/project

Now, let's run a command that will produce both standard output and standard error:

echo "This is standard output"
ls /nonexistentfile

You should see output similar to this:

This is standard output
ls: cannot access '/nonexistentfile': No such file or directory

Notice that the error message from the ls command goes directly to the terminal.

Redirecting Standard Error to a File

Let's first see how we can redirect just the error output to a file:

echo "This is standard output"
ls /nonexistentfile 2> error.log

This time, you should only see:

This is standard output

The error message has been redirected to the error.log file. The 2> syntax tells the shell to redirect standard error (file descriptor 2) to the specified file.

Let's verify the content of error.log:

cat error.log

You should see:

ls: cannot access '/nonexistentfile': No such file or directory

Capturing Both Output and Error with tee

Now, let's use tee to capture both standard output and standard error in a file while also displaying them on the terminal:

{
  echo "This is standard output"
  ls /nonexistentfile
} 2>&1 | tee both.log

You should see both the output and the error in the terminal:

This is standard output
ls: cannot access '/nonexistentfile': No such file or directory

Let's check the content of both.log:

cat both.log

You should see the same output:

This is standard output
ls: cannot access '/nonexistentfile': No such file or directory

Understanding the Command Structure

In the command 2>&1 | tee both.log:

{ ... } groups multiple commands together
2>&1 redirects standard error (file descriptor 2) to standard output (file descriptor 1)
| pipes the combined output to the tee command
tee both.log displays the output on the terminal and writes it to both.log

This technique is particularly useful for logging command output during system maintenance or troubleshooting.

Appending Output with tee

By default, the tee command overwrites the target file each time it's used. However, in many scenarios, you might want to add new information to an existing file rather than replacing its contents. This is where the -a (append) option comes in handy.

Understanding the Append Option

Navigate to the project directory if you're not already there:

cd /home/labex/project

Let's first create a file with system information:

uname -a | tee system_info.log

This command will display your system information in the terminal and save it to system_info.log. The output will look something like this (your output may differ):

Linux ubuntu 5.15.0-1031-aws #35-Ubuntu SMP Fri Feb 10 02:14:02 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Now, let's check the content of the file:

cat system_info.log

You should see the same system information.

Appending Information to the File

Now, let's append more information to this file using the -a option:

echo "Date and time: $(date)" | tee -a system_info.log

This will display something like:

Date and time: Wed May 3 14:22:34 UTC 2023

The -a flag tells tee to append the output to the file instead of overwriting it.

Let's check the content of the file again:

cat system_info.log

You should now see both the system information and the date:

Linux ubuntu 5.15.0-1031-aws #35-Ubuntu SMP Fri Feb 10 02:14:02 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Date and time: Wed May 3 14:22:34 UTC 2023

Adding More Information

Let's append even more information to our log file:

echo "Disk usage:" | tee -a system_info.log
df -h | tee -a system_info.log

This will display disk usage information and append it to our log file. The output will vary depending on your system, but it might look something like this:

Disk usage:
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        19G  5.8G   13G  32% /
tmpfs           494M     0  494M   0% /dev/shm
tmpfs           198M  1.1M  197M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/sda15      105M  5.2M  100M   5% /boot/efi
tmpfs           100M  4.0K  100M   1% /run/user/1000

Let's verify the content of our log file once more:

cat system_info.log

You should now see all the information we've added:

Linux ubuntu 5.15.0-1031-aws #35-Ubuntu SMP Fri Feb 10 02:14:02 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Date and time: Wed May 3 14:22:34 UTC 2023
Disk usage:
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        19G  5.8G   13G  32% /
tmpfs           494M     0  494M   0% /dev/shm
tmpfs           198M  1.1M  197M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/sda15      105M  5.2M  100M   5% /boot/efi
tmpfs           100M  4.0K  100M   1% /run/user/1000

Understanding the Command Structure

The syntax for appending with tee is:

command | tee -a filename

Where:

command is any command that produces output
| (pipe) sends the output of the command to tee
tee is the command itself
-a is the option to append to the file rather than overwrite it
filename is the name of the file where the output will be appended

This approach is particularly useful for:

Building log files over time
Capturing the output of multiple commands in a single file
Maintaining a record of system states or operations

Summary

In this lab, you have learned how to use the tee command for output multiplexing in Linux. This powerful utility allows you to view command output in the terminal while simultaneously saving it to files, making it an essential tool for system administrators and developers.

Here are the key skills you've acquired:

Basic Usage of tee: You learned how to use tee to display output on the terminal while simultaneously saving it to a file.
Redirecting Standard Error: You mastered how to capture both standard output and standard error using the 2>&1 redirection in combination with tee, which is crucial for comprehensive logging.
Appending Output: You discovered how to use the -a option with tee to append output to existing files rather than overwriting them, enabling you to build logs incrementally.

These techniques are valuable for various tasks in Linux, including:

Creating detailed logs for debugging
Monitoring system activities
Documenting command outputs during system administration
Preserving command results for future reference

Understanding how to effectively manage output in Linux is a fundamental skill that will serve you well as you progress in your journey with Linux systems.