Introduction
Linux wildcards, also known as globbing patterns, are special characters used in the shell to match and expand file names. Understanding how to use these powerful tools can greatly improve your efficiency when working with the Linux command line. This tutorial will guide you through the basics of Linux wildcards and demonstrate how to leverage them to copy files with specific patterns, helping you streamline your file management tasks.
Understanding Basic Wildcards
In Linux, wildcards are special characters that help you match multiple files using patterns. They are particularly useful when you need to manage many files at once. Let's explore the most common wildcards by examining the files in our current directory.
First, let's see all the files we have to work with:
cd ~/project/wildcards_lab
ls
You should see a list of files with various extensions like this:
backup data_02.csv doc2.pdf file3.txt image3.jpg readme.md
config.yml data_03.csv documents images report1.txt report_final.txt
data_01.csv doc1.pdf file1.txt image1.jpg report2.txt script.sh
file2.txt image2.jpg
The Asterisk (*) Wildcard
The asterisk * is the most common wildcard. It matches any number of characters (including zero). Let's see it in action:
To list all text files (ending with .txt):
ls *.txt
You should see:
file1.txt file2.txt file3.txt report1.txt report2.txt report_final.txt
To list all image files (ending with .jpg):
ls *.jpg
Output:
image1.jpg image2.jpg image3.jpg
The Question Mark (?) Wildcard
The question mark ? matches exactly one character. Let's use it to find files:
ls file?.txt
This command will list files that start with "file", followed by exactly one character, and end with ".txt".
Output:
file1.txt file2.txt file3.txt
The Square Brackets ([]) Wildcard
Square brackets [] let you match any single character from a specified set:
ls data_0[1-2].csv
This command lists CSV files whose names match "data_0" followed by either 1 or 2, then ".csv".
Output:
data_01.csv data_02.csv
Listing Hidden Files
In Linux, files starting with a dot (.) are hidden files. We can use wildcards to find them too:
ls .*
Output will include:
. .. .hidden_file1 .hidden_file2
Note: The . and .. represent the current directory and parent directory, respectively.
Try these commands and observe the results. Understanding these basic wildcards is essential before we move on to using them for file copying operations.
Copying Files Using Wildcards
Now that you understand the basic wildcards, let's use them to copy files efficiently. The general syntax for copying files is:
cp [options] source_file(s) destination
Let's create some organizing directories and practice copying files with different patterns.
Copying All Files with a Specific Extension
Let's copy all text files to the backup directory:
cd ~/project/wildcards_lab
cp *.txt backup/
This command copies all files ending with .txt to the backup directory. Let's verify:
ls backup/
You should see:
file1.txt file2.txt file3.txt report1.txt report2.txt report_final.txt
Copying Files Starting with Specific Patterns
Next, let's copy all image files to the images directory:
cp image*.jpg images/
This copies all files starting with "image" and ending with ".jpg" to the images directory. Let's check:
ls images/
Output:
image1.jpg image2.jpg image3.jpg
Copying Files with Multiple Patterns Using Brace Expansion
The brace expansion {pattern1,pattern2} allows you to specify multiple patterns. Let's copy both PDF and CSV files to the documents directory:
cp *.{pdf,csv} documents/
This command copies all files ending with either .pdf or .csv to the documents directory. Let's verify:
ls documents/
Output:
data_01.csv data_02.csv data_03.csv doc1.pdf doc2.pdf
Copying Files with Multiple Patterns using or (||)
We can also use the || operator to copy files matching different patterns:
cp report*.txt script.sh backup/
This copies all files starting with "report" and ending with ".txt", as well as the script.sh file, to the backup directory. Let's check what's in the backup directory now:
ls backup/
Updated output:
file1.txt file2.txt file3.txt report1.txt report2.txt report_final.txt script.sh
Copying Files While Preserving Attributes
When copying files, you might want to preserve attributes like timestamps and permissions. Use the -p option:
cp -p config.yml backup/
To verify that the attributes were preserved:
ls -l config.yml
ls -l backup/config.yml
The timestamps and permissions should be identical for both files.
Try experimenting with these commands and explore how different wildcard patterns help you copy files efficiently.
Advanced Wildcard Techniques
Now that you have mastered the basics, let's explore some more advanced wildcard patterns to perform more specific file operations.
Creating a New Directory for Advanced Operations
First, let's create a new directory to practice advanced techniques:
cd ~/project/wildcards_lab
mkdir advanced
Using Character Classes with Square Brackets
Character classes allow you to match a single character from a set of characters:
## Copy files that start with either 'd' or 'f'
cp [df]*.* advanced/
This copies all files starting with either 'd' or 'f' to the advanced directory. Let's check:
ls advanced/
Output:
data_01.csv data_02.csv data_03.csv doc1.pdf doc2.pdf file1.txt file2.txt file3.txt
Using Negation in Character Classes
You can use the ! or ^ symbol after the opening bracket to negate the character class. Let's copy files that don't start with 'r' or 'i':
mkdir advanced/filtered
cp [^ri]*.txt advanced/filtered/
Let's verify:
ls advanced/filtered/
Output:
file1.txt file2.txt file3.txt
Combining Multiple Wildcards
Let's combine different wildcards to create more specific patterns:
## Copy files that start with 'data_0', followed by a single digit, and end with '.csv'
cp data_0?.csv advanced/
Let's verify this copied our CSV files correctly:
ls advanced/*.csv
Output:
advanced/data_01.csv advanced/data_02.csv advanced/data_03.csv
Using Extended Globbing
Bash provides extended globbing patterns, which are more powerful than standard wildcards. First, enable extended globbing:
shopt -s extglob
Now we can use patterns like:
?(pattern): Matches zero or one occurrence of the pattern*(pattern): Matches zero or more occurrences+(pattern): Matches one or more occurrences@(pattern): Matches exactly one occurrence!(pattern): Matches anything except the pattern
Let's try one example:
## Copy files that have a number followed by .txt
mkdir advanced/numbered
cp +([0-9])*.txt advanced/numbered/
The result might be empty since our original files don't match this exact pattern. Let's create a file that does:
touch 123file.txt
cp +([0-9])*.txt advanced/numbered/
ls advanced/numbered/
Output:
123file.txt
Using Brace Expansion for Complex Patterns
Brace expansion is useful for creating complex patterns:
## Create a directory for report files
mkdir advanced/reports
## Copy all report files and configuration files
cp {report*.txt,*.yml} advanced/reports/
Let's check:
ls advanced/reports/
Output:
config.yml report1.txt report2.txt report_final.txt
These advanced techniques will help you be more precise when selecting files, saving you time and effort in managing your file system. Try experimenting with combinations of these wildcards to see how they can help you manage files more efficiently.
Practical File Management Exercise
In this final step, we will put together everything we've learned to solve a practical file management problem. This exercise will reinforce your understanding of wildcards and demonstrate their practical application.
Scenario
Imagine you are a system administrator responsible for organizing a directory containing various types of files. You need to:
- Identify and categorize files by type
- Make a backup of important files
- Extract files based on specific naming patterns
Let's simulate this scenario:
Creating a Simulated Project Directory
First, let's create a new project directory with various types of files:
cd ~/project/wildcards_lab
mkdir project_files
cd project_files
## Create various types of files
touch project_doc_v1.txt project_doc_v2.txt project_doc_final.txt
touch data_2023_01.csv data_2023_02.csv data_2023_03.csv
touch config_dev.yml config_test.yml config_prod.yml
touch backup_script_v1.sh backup_script_v2.sh
touch image001.png image002.png image003.png
touch .project_config .project_log
Let's check our directory:
ls
Output:
backup_script_v1.sh config_prod.yml data_2023_02.csv image002.png project_doc_v1.txt
backup_script_v2.sh config_test.yml data_2023_03.csv image003.png project_doc_v2.txt
config_dev.yml data_2023_01.csv image001.png project_doc_final.txt
Note: The hidden files (starting with .) won't appear in the basic ls output.
Organizing Files by Type
Now, let's create directories for different file types and organize our files:
## Create directories
mkdir docs configs scripts data images backups
## Copy text documents to docs directory
cp *doc*.txt docs/
## Copy configuration files to configs directory
cp *.yml configs/
## Copy scripts to scripts directory
cp *.sh scripts/
## Copy data files to data directory
cp *.csv data/
## Copy images to images directory
cp *.png images/
Let's verify our organization:
echo "Documents:"
ls docs/
echo "Configurations:"
ls configs/
echo "Scripts:"
ls scripts/
echo "Data files:"
ls data/
echo "Images:"
ls images/
Output:
Documents:
project_doc_final.txt project_doc_v1.txt project_doc_v2.txt
Configurations:
config_dev.yml config_prod.yml config_test.yml
Scripts:
backup_script_v1.sh backup_script_v2.sh
Data files:
data_2023_01.csv data_2023_02.csv data_2023_03.csv
Images:
image001.png image002.png image003.png
Creating a Comprehensive Backup
Let's create a comprehensive backup that includes all important files:
## Create a backup of all non-image files
cp !(*.png|*backup*) backups/
## Let's see what's in our backups directory
ls backups/
Output:
config_dev.yml config_test.yml data_2023_01.csv data_2023_03.csv project_doc_final.txt project_doc_v2.txt
config_prod.yml data_2023_02.csv project_doc_v1.txt
Finding Files with Specific Patterns
Now, let's find files that match specific patterns:
## Find all files from 2023
echo "Files from 2023:"
ls *2023*
## Find all final or production versions
echo "Final/Production versions:"
ls *final* *prod*
## Find all version 2 files
echo "Version 2 files:"
ls *v2*
Output:
Files from 2023:
data_2023_01.csv data_2023_02.csv data_2023_03.csv
Final/Production versions:
config_prod.yml project_doc_final.txt
Version 2 files:
backup_script_v2.sh project_doc_v2.txt
Practical Application: Creating a Deployment Package
Finally, let's create a deployment package containing only production configurations and final documentation:
mkdir deployment_package
cp *prod*.yml *final*.txt deployment_package/
## Check our deployment package
ls deployment_package/
Output:
config_prod.yml project_doc_final.txt
Congratulations! You've successfully completed a practical file management exercise using Linux wildcards. These skills will be invaluable for efficiently managing files in a Linux environment, automating repetitive tasks, and organizing your file system effectively.
Summary
In this lab, you have learned how to effectively use Linux wildcards to copy and manage files with specific patterns. You started with basic wildcards like the asterisk (*), question mark (?), and square brackets ([]), and then progressed to more advanced techniques such as character classes, negation, and extended globbing.
Here are the key skills you have acquired:
- Using basic wildcards (*,?,[ ]) to match file patterns
- Copying files with specific extensions or naming patterns
- Using advanced wildcard techniques for more precise file selection
- Combining multiple wildcards and brace expansion for complex patterns
- Applying these skills to organize and manage files in a practical scenario
These skills will significantly enhance your productivity when working with the Linux command line, allowing you to automate file management tasks and efficiently organize your file system. As you continue to work with Linux, these wildcard techniques will become invaluable tools in your daily operations.
Remember that practice makes perfect. The more you use these wildcards in your daily work, the more comfortable and proficient you will become with them.
