Linux split Command with Practical Examples

Introduction

In this lab, you will learn how to use the Linux split command to divide a large file into smaller, more manageable parts. The split command is a powerful tool that allows you to split files based on size, line count, or other criteria, making it useful for working with large log files, database backups, or other types of data that need to be processed or transferred in smaller pieces. You will start by understanding the purpose of the split command, then learn how to customize the command options to split a file into multiple parts. This lab provides practical examples to help you become proficient in using the split command for your text processing and editing needs.

Linux Commands Cheat Sheet

Skills Graph

Understand the Purpose of the split Command

In this step, you will learn about the purpose and usage of the split command in Linux. The split command is a powerful tool that allows you to divide a large file into smaller, more manageable pieces.

The primary use case for the split command is to split large files into smaller chunks, which can be helpful when working with files that are too large to handle or transfer easily. This can be particularly useful when working with large log files, database backups, or other types of data that need to be processed or transferred in smaller pieces.

To get started, let's create a large file that we can use for our examples:

head -n 10000 /dev/urandom > large_file.txt

This command will create a file named large_file.txt with 10,000 lines of random data.

Now, let's use the split command to divide this file into smaller chunks:

split -b 1m large_file.txt split_

This command will split the large_file.txt file into multiple files, each with a maximum size of 1 megabyte (1m), and prefix the filenames with "split_".

Example output:

split_aa
split_ab
split_ac
split_ad

In the next step, you will learn how to customize the split command options to further control the file splitting process.

Split a File into Multiple Parts

In this step, you will learn how to use the split command to split a file into multiple parts.

Let's continue using the large_file.txt file we created in the previous step. We can split this file into smaller chunks using the split command with different options:

## Split the file into 5 equal-sized parts
split -n 5 large_file.txt split_part_

## Split the file into parts with a maximum size of 500 KB
split -b 500k large_file.txt split_part_

## Split the file into parts with a line-based approach
split -l 1000 large_file.txt split_part_

Example output:

split_part_aa
split_part_ab
split_part_ac
split_part_ad
split_part_ae

The first command splits the file into 5 equal-sized parts, the second command splits the file into parts with a maximum size of 500 KB, and the third command splits the file into parts with a maximum of 1,000 lines per file.

You can use the ls command to verify that the files have been created:

ls -l split_*

Example output:

-rw-r--r-- 1 labex labex 2000000 Apr 12 12:34 split_part_aa
-rw-r--r-- 1 labex labex 2000000 Apr 12 12:34 split_part_ab
-rw-r--r-- 1 labex labex 2000000 Apr 12 12:34 split_part_ac
-rw-r--r-- 1 labex labex 2000000 Apr 12 12:34 split_part_ad
-rw-r--r-- 1 labex labex 2000000 Apr 12 12:34 split_part_ae

In the next step, you will learn how to customize the split command options even further.

Customize the Split Command Options

In this final step, you will learn how to customize the split command options to better suit your needs.

The split command provides several options that allow you to control the file splitting process. Here are a few examples:

## Split the file into parts with a prefix of "custom_"
split -d -a 2 -b 1m large_file.txt custom_

## Split the file into parts with a prefix of "custom_" and a suffix of ".txt"
split -d -a 2 -b 1m -d large_file.txt custom_.txt

## Split the file into parts with a custom suffix
split -d -a 2 -b 1m large_file.txt part_

## Split the file into parts based on a specific pattern
split -d -a 2 -l 1000 large_file.txt pattern_

Let's go through these examples:

• The -d option tells split to use numeric suffixes instead of alphabetic ones.
• The -a 2 option sets the length of the suffix to 2 characters.
• The -b 1m option sets the maximum size of each output file to 1 megabyte.
• The -d option in the second example tells split to use numeric suffixes and append the .txt extension to the output files.
• The part_ prefix in the third example creates output files named part_00, part_01, and so on.
• The pattern_ prefix in the fourth example creates output files named pattern_00, pattern_01, and so on.

You can experiment with these options to find the best way to split your files based on your specific needs.

Summary

In this lab, you learned about the purpose and usage of the split command in Linux. The split command is a powerful tool that allows you to divide a large file into smaller, more manageable pieces, which can be particularly useful when working with large log files, database backups, or other types of data that need to be processed or transferred in smaller chunks. You then learned how to use the split command with different options to split a file into multiple parts, including splitting the file into a specified number of equal-sized parts, splitting the file into parts with a maximum size, and splitting the file based on a line-based approach.

Linux Commands Cheat Sheet