How to use 'tee' to capture standard error

Introduction

This tutorial guides you through working with Linux standard streams, with a focus on understanding and managing the standard error (stderr) stream. You will learn how to use the powerful tee command to capture and process stderr output, a valuable skill for error handling and logging in Linux systems.

Understanding Linux Standard Streams

Linux uses three fundamental streams to handle input and output operations. These streams form the foundation of how programs communicate in a Linux environment.

What are Standard Streams?

Open a terminal in your Linux environment. We'll explore the concept of standard streams through practical examples.

Standard streams are communication channels that connect programs with their environment. Linux has three standard streams:

1. Standard Input (stdin) - File descriptor 0
2. Standard Output (stdout) - File descriptor 1
3. Standard Error (stderr) - File descriptor 2

Let's see these streams in action with some simple commands.

Demonstrating Standard Input (stdin)

Standard input is how programs receive data, typically from keyboard input.

Type the following command in your terminal:

cat

Now type any text and press Enter. The cat command reads from stdin and outputs it to stdout. Type a few more lines of text.

To exit the cat command, press Ctrl+D (which signals end-of-file).

Demonstrating Standard Output (stdout)

Standard output is where programs send their normal output.

Run this command:

echo "This message goes to standard output"

You should see:

This message goes to standard output

The echo command sends the text to stdout, which is displayed on your terminal.

Demonstrating Standard Error (stderr)

Standard error is where programs send error messages and warnings.

Run this command to generate an error:

ls /nonexistent_directory

You should see an error message similar to:

ls: cannot access '/nonexistent_directory': No such file or directory

This error message is sent to stderr, but it appears on your terminal just like stdout.

Distinguishing Between stdout and stderr

To see the difference between stdout and stderr, let's redirect them separately:

ls /home /nonexistent_directory > output.txt 2> error.txt

Now examine the contents of each file:

cat output.txt
cat error.txt

You should see that output.txt contains the listing of the /home directory, while error.txt contains the error message for the nonexistent directory.

Understanding how these streams work is crucial for controlling program input and output in Linux.

Introducing the tee Command

Now that you understand standard streams, let's learn about the tee command, which offers a powerful way to manage these streams.

What is the tee Command?

The tee command in Linux takes input and sends it to both standard output and one or more files simultaneously. It's named after the T-splitter used in plumbing that sends water in two directions.

Let's explore how tee works with a basic example:

echo "Hello, tee command" | tee hello.txt

You should see:

Hello, tee command

This text appears on your terminal and is also saved to hello.txt. Confirm this by checking the file:

cat hello.txt

Combining tee with Standard Output

Let's see how we can use tee with program output:

ls -la ~ | tee home_contents.txt

This command lists the contents of your home directory, displays it on the screen, and saves it to home_contents.txt.

Basic Error Handling with tee

By default, tee only captures standard output. To capture standard error as well, we need to redirect stderr to stdout first.

Try this example:

ls /home /nonexistent_directory 2>&1 | tee mixed_output.txt

The 2>&1 part redirects stderr (file descriptor 2) to stdout (file descriptor 1), combining both streams. The tee command then captures this combined output.

Examine the contents of the file:

cat mixed_output.txt

You should see both the directory listing and the error message in the file.

Appending Output Instead of Overwriting

If you want to append to a file rather than overwrite it, use the -a option:

echo "First line" | tee log.txt
echo "Second line" | tee -a log.txt

Check the contents:

cat log.txt

You should see both lines in the file, as the second command appended to the file rather than overwriting it.

Advanced Error Handling with tee

Now that you understand the basics of tee, let's explore more advanced uses for error handling and logging.

Separating Standard Output and Standard Error

Sometimes you want to capture stdout and stderr separately while still displaying both on the terminal. Let's create a script that generates both types of output:

nano test_script.sh

Add the following content to the script:

#!/bin/bash
echo "This is standard output"
echo "This is standard error" >&2
ls /home
ls /nonexistent_directory

Save the file (press Ctrl+O, then Enter) and exit (press Ctrl+X).

Make the script executable:

chmod +x test_script.sh

Now run the script with both stdout and stderr captured separately:

./test_script.sh > >(tee stdout.log) 2> >(tee stderr.log >&2)

This complex command:

1. Runs your script
2. Redirects stdout to tee stdout.log, which shows it on screen and saves to a file
3. Redirects stderr to tee stderr.log >&2, which shows it on screen and saves to a file

Examine the results:

cat stdout.log
cat stderr.log

You should see regular output in stdout.log and error messages in stderr.log.

Creating a Complete Error Log

For comprehensive logging, you might want to:

1. Send normal output to the screen
2. Send errors to both the screen and a log file
3. Timestamp the errors for easier tracking

Let's create a script to demonstrate this:

nano logging_script.sh

Add the following content:

#!/bin/bash

## Function to generate a timestamp
timestamp() {
  date +"%Y-%m-%d %H:%M:%S"
}

## Echo with timestamp to stderr
echo_error() {
  echo "$(timestamp) - ERROR: $1" >&2
}

## Normal output
echo "Starting the script"

## Error output
echo_error "Something went wrong"

## More normal output
echo "Script continuing despite the error"

## Another error
echo_error "Another issue occurred"

## Final output
echo "Script completed"

Save and make the script executable:

chmod +x logging_script.sh

Now run it with error logging:

./logging_script.sh 2> >(tee -a error_log.txt >&2)

This will:

1. Display all output (both stdout and stderr) on the screen
2. Additionally capture stderr in the error_log.txt file

Check the error log:

cat error_log.txt

You should see only the error messages with timestamps, which makes them easy to track.

Real-World Application: Command with Progress and Error Handling

Let's create a practical example that downloads a file, showing progress on screen while logging errors:

wget https://example.com/nonexistent_file.txt 2> >(tee -a download_errors.log >&2)

This command:

1. Attempts to download a file that doesn't exist
2. Displays all output on the screen, including progress and errors
3. Additionally logs any errors to download_errors.log

Check the error log:

cat download_errors.log

The error log contains only the error messages from the failed download attempt.

These techniques allow you to build sophisticated error handling and logging systems using standard Linux commands.

Summary

In this tutorial, you have learned essential techniques for managing standard streams in Linux, with a focus on error handling using the tee command. You now understand:

• The three standard streams in Linux: stdin, stdout, and stderr
• How to use the tee command to capture output while displaying it on screen
• How to redirect stderr to stdout and capture both with tee
• Advanced techniques for separating and logging different output streams
• How to implement practical error logging in real-world scenarios

These skills are valuable for creating robust scripts, troubleshooting applications, and maintaining comprehensive logs in Linux environments. The ability to manage standard error output effectively will help you develop more professional and maintainable Linux applications.