sed Stream Editing

Introduction

When working with text files in Linux, you often need to perform automated edits without opening an interactive text editor. For example, you might need to remove a header from a CSV file, delete blank lines, or extract a specific range of lines from a log file.

For these tasks, we use sed (stream editor).

sed is a powerful non-interactive text editor. It reads input from a file or pipe, performs operations on each line, and outputs the result. This "read, transform, print" model is a classic Unix approach and works especially well in pipelines.

That model explains the name stream editor. sed is designed to work on flowing text, not only on files opened like a full-screen editor. This is why it fits so naturally into shell pipelines and automation scripts.

In this lab, you will learn how to:

Delete Lines (d): Remove unwanted headers or comments.
Remove Empty Lines: Use regular expressions to filter out blank lines.
Insert Text (i): Add new lines to the file.
Print Lines (p): Extract specific ranges of lines.

Delete Lines

Let's start by inspecting our sample file.

cat /home/labex/project/data.csv

You will notice it has two comment lines at the top starting with #. We want to remove these lines.

To delete lines in sed, we use the d command.

1d deletes line 1.
1,2d deletes lines 1 through 2.

Addressing lines by number is one of sed's basic ideas. An address tells sed which line or range of lines a command should apply to, and the editing command tells it what to do there.

Run the following command to preview the deletion (without modifying the file):

sed '1,2d' data.csv

The output shows the file content without the first two lines. However, check the file content again:

cat data.csv

The lines are still there. By default, sed sends output to the screen (standard output) and does not modify the original file. This is a safety feature.

To apply changes to the file (edit "in-place"), use the -i option:

sed -i '1,2d' data.csv

Verify the changes:

cat data.csv

Now the header lines have been permanently removed.

Remove Empty Lines

Our file still contains empty blank lines that break the structure of the data. We rarely want these in processed data files.

To remove them, we combine the delete command (d) with a Regular Expression (Regex).

^ matches the start of a line.
$ matches the end of a line.
^$ matches a line that starts and immediately ends, i.e., an empty line.

This is a good example of sed combining two layers: an address pattern and an editing action. The pattern /^$/ selects matching lines, and d deletes those selected lines.

The command /^$/d tells sed to "delete any line that matches the empty pattern".

Run the command to remove empty lines in-place:

sed -i '/^$/d' data.csv

Check the result:

cat data.csv

The file should now be a compact list of data without any gaps.

Insert Text

Now that the useless comments are gone, let's say we want to add a proper classification header to the file.

The i (Insert) command adds text before the specified line.

1i Text inserts "Text" before line 1.

The a (Append) command adds text after the specified line.

1a Text inserts "Text" after line 1.

Along with delete and substitute, commands like i and a show that sed is not limited to replacement. It can reshape the stream by removing, adding, and selecting lines as they pass through.

Run the following command to insert "Company Confidential" at the very top of the file:

sed -i '1i Company Confidential' data.csv

Verify the file content:

cat data.csv

The new line has been inserted at the beginning, shifting all other lines down.

Print Specific Lines

Sometimes you don't want to edit the file; you only want to extract specific lines. sed can act like a smarter version of head or tail.

By default, sed prints every line it processes. To print only specific lines, we:

Use the -n option to suppress automatic printing (be silent).
Use the p command to explicitly print specific lines.

Without -n, sed would print every line automatically and then print the selected lines again when p runs, which often confuses beginners. -n turns off the default behavior so your command prints only what you asked for.

To print lines 5 through 10:

sed -n '5,10p' data.csv

To print only the last line, use the $ symbol to represent the end of the file:

sed -n '$p' data.csv

This is extremely useful when analyzing large log files where you only need to see a specific section.

Summary

In this lab, you learned the basics of stream editing with sed:

Deletion (d): Removing specific lines (1,2d) or lines matching a pattern (/^$/d).
Insertion (i): Adding text into the file stream at specific locations.
Printing (-n ... p): suppressing default output and printing only selected lines.
In-Place Editing (-i): Modifying the file directly instead of just printing to the screen.

sed is an essential tool for automated text processing and is widely used in shell scripts for configuration management and log analysis.