How to perform global substitution with sed in Linux?

Introduction

Linux provides a powerful text processing tool called sed (stream editor) that allows you to perform global substitution on text files. This tutorial will guide you through the process of using sed for global substitution, covering both basic and advanced techniques to help you streamline your text processing workflows on Linux systems.

Skills Graph

Introduction to sed and Global Substitution

What is sed?

sed (Stream Editor) is a powerful command-line tool used for text manipulation in Linux and Unix-like operating systems. It allows you to perform a wide range of operations on text files, including searching, replacing, deleting, and inserting text.

Global Substitution with sed

One of the most common use cases for sed is performing global substitution, which involves replacing all occurrences of a pattern in a text file with a new pattern. This is particularly useful when you need to make large-scale changes to your text files, such as updating variable names, removing unnecessary characters, or standardizing formatting.

Advantages of Global Substitution with sed

• Efficiency: sed can process text files quickly and efficiently, making it a great choice for large-scale text manipulation tasks.
• Automation: sed commands can be easily scripted and incorporated into shell scripts, allowing you to automate repetitive text-processing tasks.
• Versatility: sed supports a wide range of regular expression patterns, enabling you to perform complex text substitutions.

Example: Replacing all occurrences of "old_string" with "new_string"

sed 's/old_string/new_string/g' input_file.txt

In this example, the s command is used to perform the substitution, and the g flag ensures that all occurrences of "old_string" are replaced with "new_string" in the input file.

Using sed for Global Substitution

The sed Substitution Command

The basic syntax for using sed for global substitution is:

sed 's/pattern/replacement/g' input_file

• s: Specifies that this is a substitution command.
• pattern: The text or regular expression you want to replace.
• replacement: The text you want to use as the replacement.
• g: The global flag, which ensures that all occurrences of the pattern are replaced, not just the first one.

Examples of Global Substitution with sed

1. Replace all occurrences of "old_word" with "new_word" in a file:

   sed 's/old_word/new_word/g' input_file.txt

2. Replace all occurrences of a phone number format (e.g., 123-456-7890) with a new format (e.g., (123) 456-7890):

   sed 's/\([0-9]\{3\}\)-\([0-9]\{3\}\)-\([0-9]\{4\}\)/(\1) \2-\3/g' input_file.txt

3. Replace all occurrences of multiple spaces with a single space:

   sed 's/  */ /g' input_file.txt

4. Remove all occurrences of a specific word or pattern:

   sed 's/unwanted_word//g' input_file.txt

These examples demonstrate the versatility of sed in performing global substitutions on text files. By using regular expressions, you can target and replace complex patterns with ease.

Advanced sed Substitution Techniques

Capturing Groups

sed allows you to use capturing groups to reference parts of the matched pattern in the replacement string. Capturing groups are defined using parentheses () in the pattern.

Example:

sed 's/\([0-9]\{3\}\)-\([0-9]\{3\}\)-\([0-9]\{4\}\)/(\1) \2-\3/' input_file.txt

In this example, the phone number format 123-456-7890 is captured in three groups: \1 (area code), \2 (prefix), and \3 (line number). The replacement string uses these captured groups to rearrange the format to (123) 456-7890.

Conditional Substitutions

sed also supports conditional substitutions, where the replacement is only performed if a certain condition is met. This is done using the t command.

Example:

sed '/^#/!s/old_string/new_string/g' input_file.txt

In this example, the substitution is only performed on lines that do not start with a # (i.e., non-comment lines).

In-Place Editing

By default, sed outputs the modified text to the console. To perform in-place editing and modify the original file, you can use the -i option.

Example:

sed -i 's/old_string/new_string/g' input_file.txt

This will replace all occurrences of old_string with new_string in the input_file.txt file.

Multiline Substitutions

sed can also handle multiline substitutions using the newline character \n in the replacement string.

Example:

sed 's/pattern1/replacement1\nreplacement2/g' input_file.txt

This will replace each occurrence of pattern1 with two lines: replacement1 and replacement2.

These advanced techniques allow you to perform more complex and powerful text manipulations with sed, tailoring the substitutions to your specific needs.

Summary

In this comprehensive guide, you have learned how to leverage the sed command in Linux to perform global substitution on text files. By mastering the techniques covered, you can now efficiently automate and optimize your text processing tasks, making your Linux workflows more productive and efficient.