LabEx
Join Free

Python File and Directory Path Manipulation

PythonBeginner
Practice Now

Introduction

In this lab, you will learn how to handle file and directory paths in Python using the modern pathlib module. While the older os.path module is still common, pathlib offers an object-oriented approach that makes path manipulation more intuitive and readable. You will learn to create, inspect, validate, and manipulate paths, files, and directories in a way that works across different operating systems like Windows, macOS, and Linux.

By the end of this lab, you will be proficient in:

  • Joining path components correctly.
  • Creating and managing directories.
  • Differentiating between absolute and relative paths.
  • Validating the existence and type of paths.
  • Performing common file operations like copying, moving, and deleting.
  • Traversing directory trees to find files and folders.

Basic Path Operations

In this step, you will learn the fundamentals of path manipulation using the pathlib module. This includes joining path components, getting the current working directory, and creating new directories.

The pathlib module provides an elegant, cross-platform way to handle filesystem paths. On Windows, paths use backslashes (\), while on Unix-like systems (Linux, macOS), they use forward slashes (/). Using pathlib ensures your code works on any platform without manual adjustments.

Note: If you are not familiar with using WebIDE (VS Code like), please learn it first at this lab.

First, let's see how to join path components. The / operator is an intuitive shortcut for joinpath().

  1. Open the main.py file from the file explorer on the left.
  2. Replace the content of main.py with the following code to join path components and print the result.
from pathlib import Path

## Join path components using the / operator
my_path = Path('usr') / 'bin' / 'spam'
print(my_path)
  1. Run the script from the terminal:
python3 main.py

You will see the following output, which demonstrates how pathlib correctly forms a Unix-style path:

usr/bin/spam

Next, let's get the current working directory using Path.cwd().

  1. Replace the content of main.py with the following code (this replaces the previous code):
from pathlib import Path

## Get the current working directory
cwd = Path.cwd()
print(cwd)
  1. Run the script again:
python3 main.py

The output will show the current working directory, which is /home/labex/project in this environment:

/home/labex/project

Finally, let's create a new directory. A FileNotFoundError occurs if you try to create a directory where the parent directory doesn't exist. To avoid this, use mkdir(parents=True), which creates all necessary parent directories.

  1. Replace the content of main.py with the following code to create a nested directory structure:
from pathlib import Path

## Get the current working directory
cwd = Path.cwd()

## Define the path for the new nested directories
new_dirs = cwd / 'delicious' / 'walnut' / 'waffles'

## Create the directories, including any missing parents
new_dirs.mkdir(parents=True, exist_ok=True)

print(f"Successfully created: {new_dirs}")

The exist_ok=True argument prevents an error if the directory already exists. This means you can run the script multiple times without errors.

  1. Run the script:
python3 main.py

The script will create the nested directories and print a confirmation message:

Successfully created: /home/labex/project/delicious/walnut/waffles

You can verify the directory creation by running ls -R delicious in the terminal.

Absolute vs Relative Paths

In this step, you will learn to distinguish and work with absolute and relative paths.

  • An absolute path starts from the root of the filesystem (e.g., / on Linux or C:\ on Windows).
  • A relative path is defined from the current working directory.

pathlib provides methods to check if a path is absolute, resolve a path to its absolute form, and find a relative path from a starting point.

First, let's check if a path is absolute using is_absolute().

  1. Replace the content of main.py with the following code:
from pathlib import Path

## Check if paths are absolute
print(f"Path('/') is absolute: {Path('/').is_absolute()}")
print(f"Path('..') is absolute: {Path('..').is_absolute()}")
  1. Run the script:
python3 main.py

The output confirms that / is an absolute path, while .. (parent directory) is relative.

Path('/') is absolute: True
Path('..') is absolute: False

Next, you can get the full absolute path of a relative path using resolve(). This is useful for converting relative paths like . (current directory) or .. (parent directory) into their full, unambiguous form.

  1. Replace the content of main.py with the following code:
from pathlib import Path

## Resolve a relative path to its absolute form
## '..' refers to the parent directory of the current working directory
resolved_path = Path('..').resolve()

print(f"Current working directory: {Path.cwd()}")
print(f"Resolved path of '..': {resolved_path}")
  1. Execute the script:
python3 main.py

The output will show the current directory and the resolved absolute path of its parent.

Current working directory: /home/labex/project
Resolved path of '..': /home/labex

Finally, you can compute a relative path from one path to another using relative_to().

  1. Replace the content of main.py with the following code to calculate a relative path:
from pathlib import Path

## Get a relative path from a starting point
## This shows the path to 'passwd' relative to the root directory '/'
relative_path = Path('/etc/passwd').relative_to('/')
print(f"Path to '/etc/passwd' relative to '/': {relative_path}")
  1. Run the script:
python3 main.py

The output shows the path relative to the root directory:

Path to '/etc/passwd' relative to '/': etc/passwd

Path Validation and Inspection

In this step, you will learn how to check the properties of a path, such as whether it exists and whether it's a file or a directory. You will also learn how to retrieve file metadata like its size.

A common task is to verify if a file or directory exists before trying to access it. The exists() method is perfect for this.

  1. Open main.py and replace its content with the following code. This will check for the existence of the current directory (.), a file (setup.py), and a non-existent file.
from pathlib import Path

## Check if paths exist
print(f"Path('.') exists: {Path('.').exists()}")
print(f"Path('setup.py') exists: {Path('setup.py').exists()}")
print(f"Path('nonexistentfile') exists: {Path('nonexistentfile').exists()}")
  1. Run the script:
python3 main.py

The output will confirm which paths exist:

Path('.') exists: True
Path('setup.py') exists: True
Path('nonexistentfile') exists: False

You can also be more specific and check if a path points to a file with is_file() or a directory with is_dir().

  1. Replace the content of main.py with the following code to perform these checks:
from pathlib import Path

## Check if a path is a file
print(f"Path('setup.py').is_file(): {Path('setup.py').is_file()}")
print(f"Path('/home').is_file(): {Path('/home').is_file()}")

## Check if a path is a directory
print(f"Path('/').is_dir(): {Path('/').is_dir()}")
print(f"Path('setup.py').is_dir(): {Path('setup.py').is_dir()}")
  1. Execute the script:
python3 main.py

The output will correctly identify each path's type:

Path('setup.py').is_file(): True
Path('/home').is_file(): False
Path('/').is_dir(): True
Path('setup.py').is_dir(): False

Finally, to get detailed file information, such as its size, you can use the stat() method. It returns an object with various attributes, including st_size for the file size in bytes.

  1. Replace the content of main.py with the following code to get the size of a system file:
from pathlib import Path
import os

## Define the path to a file
file_path = Path('/bin/python3')

## Get file statistics
stat_info = file_path.stat()

## Print the size in bytes
print(f"File: {file_path}")
print(f"Size: {stat_info.st_size} bytes")
  1. Run the script:
python3 main.py

The output will display the size of the Python executable in bytes. The exact size may vary.

File: /bin/python3
Size: 10024 bytes

File and Folder Management

In this step, you will learn to perform common file system operations like copying, moving, renaming, and deleting files and folders using the shutil module in combination with pathlib.

The shutil module provides high-level functions for file operations. Let's start with copying files. shutil.copy() copies a single file to a new location.

  1. Open the manage_files.py file from the file explorer on the left.
  2. Replace the content of manage_files.py with the following code. This will copy spam.txt into the /tmp/delicious directory.
import shutil
from pathlib import Path

## Define source and destination
source_file = Path('/tmp/spam.txt')
destination_dir = Path('/tmp/delicious')

## Copy the file
shutil.copy(source_file, destination_dir)

print(f"Copied '{source_file}' to '{destination_dir}'")

## You can also rename the file while copying
shutil.copy('/tmp/eggs.txt', '/tmp/delicious/eggs2.txt')
print("Copied and renamed 'eggs.txt' to 'eggs2.txt'")
  1. Run the script:
python3 manage_files.py

To copy an entire directory tree, use shutil.copytree().

  1. Add the following lines to the end of manage_files.py (append these lines after the existing code):
## Copy an entire directory tree
shutil.copytree('/tmp/bacon', '/tmp/bacon_backup')
print("Copied directory '/tmp/bacon' to '/tmp/bacon_backup'")

Note: If you run this script multiple times, shutil.copytree() will fail because the destination directory already exists. This is expected behavior. For now, just run it once.

  1. Run the script again to see it in action.

Next, let's move and rename files with shutil.move(). This function can either move a file to a different directory or rename it.

  1. Replace the entire content of manage_files.py with the following code to move bacon.txt:
import shutil
from pathlib import Path

## Create the destination directory if it doesn't exist
Path('/tmp/eggs').mkdir(parents=True, exist_ok=True)

## Move a file to a directory (only if source exists)
bacon_source = Path('/tmp/bacon.txt')
if bacon_source.exists():
    shutil.move('/tmp/bacon.txt', '/tmp/eggs')
    print("Moved '/tmp/bacon.txt' to '/tmp/eggs'")
else:
    print("'/tmp/bacon.txt' not found (may have been moved already)")

## Move and rename a file
## First, let's recreate the file to move
Path('/tmp/bacon.txt').touch()
shutil.move('/tmp/bacon.txt', '/tmp/eggs/new_bacon.txt')
print("Moved and renamed '/tmp/bacon.txt' to '/tmp/eggs/new_bacon.txt'")

Note: The code checks if the source file exists before moving it, so you can run this script multiple times safely. The second part will always create a new file and move it.

  1. Run the script.

Finally, let's look at deleting files and folders.

  • Path.unlink() deletes a single file.
  • Path.rmdir() deletes an empty directory.
  • shutil.rmtree() deletes a directory and all its contents.
  1. Replace the entire content of manage_files.py with the following:
from pathlib import Path
import shutil

## Create a file to delete
file_to_delete = Path('/tmp/deleteme.txt')
file_to_delete.touch()
print(f"Created '{file_to_delete}'")

## Delete the file
file_to_delete.unlink()
print(f"Deleted '{file_to_delete}'")

## Create a directory to delete
dir_to_delete = Path('/tmp/temp_dir')
dir_to_delete.mkdir(exist_ok=True)
## rmdir() only works on empty directories
dir_to_delete.rmdir()
print(f"Deleted empty directory '{dir_to_delete}'")

## Use shutil.rmtree() to delete a directory and its contents
## Note: This will fail if the directory doesn't exist, which is fine for this example
if Path('/tmp/bacon_backup').exists():
    shutil.rmtree('/tmp/bacon_backup')
    print("Deleted directory '/tmp/bacon_backup' and all its contents")
else:
    print("Directory '/tmp/bacon_backup' does not exist (may have been deleted already)")

Note: If you run this script multiple times, some operations may fail because files or directories have already been deleted. This is normal and demonstrates that these operations are not idempotent (they can't be safely repeated).

  1. Run the script to perform the deletion operations.

Directory Traversal

In this step, you will learn how to list the contents of a directory and recursively traverse a directory tree.

To list the contents of a directory, you can use the iterdir() method, which returns an iterator of all items (files and directories) in the path.

  1. First, let's set up a sample directory structure. Run this command in the terminal:
mkdir -p ~/project/data/level1 && touch ~/project/data/file0.txt ~/project/data/level1/file1.txt
  1. Now, open main.py and replace its content with the following code to list the contents of the /usr/bin directory.
from pathlib import Path

## List contents of a directory
print("Contents of '/usr/bin':")
count = 0
for f in Path('/usr/bin').iterdir():
    if count < 5: ## Print first 5 items to keep output short
        print(f)
        count += 1
    else:
        break
  1. Run the script:
python3 main.py

The output will show the first few files and directories inside /usr/bin. The exact list may vary.

Contents of '/usr/bin':
/usr/bin/[
/usr/bin/addpart
/usr/bin/apt
/usr/bin/apt-cache
/usr/bin/apt-cdrom

You can combine iterdir() with stat() to calculate the total size of all files in a directory.

  1. Replace the content of main.py with the following code. Note that this code only sums the sizes of direct children and does not recurse into subdirectories.
from pathlib import Path

total_size = 0
directory = Path('/usr/bin')

for sub_path in directory.iterdir():
    if sub_path.is_file(): ## Ensure we only add size of files
        total_size += sub_path.stat().st_size

print(f"Total size of files in '{directory}': {total_size} bytes")
  1. Execute the script. It will print the total size of all files directly within /usr/bin.

For more complex tasks, you might need to "walk" an entire directory tree, visiting all subdirectories and files. The rglob() method is perfect for this. It recursively finds all paths matching a specific pattern. Using * as the pattern matches everything.

  1. First, create a more complex directory structure for this example. Run this in the terminal:
mkdir -p /tmp/delicious/cats /tmp/delicious/walnut/waffles
touch /tmp/delicious/spam.txt /tmp/delicious/cats/catnames.txt /tmp/delicious/walnut/waffles/butter.txt

Note: You can run these commands multiple times safely. The mkdir -p command won't fail if directories already exist, and touch will update file timestamps if files already exist.

  1. Replace the content of main.py with the following code to walk the /tmp/delicious directory:
from pathlib import Path

p = Path('/tmp/delicious')
print(f"Walking directory tree for: {p}")

for i in p.rglob('*'):
    print(i)
  1. Run the script:
python3 main.py

The output will list every file and folder within /tmp/delicious and its subdirectories, demonstrating how rglob() traverses the entire tree.

Walking directory tree for: /tmp/delicious
/tmp/delicious/spam.txt
/tmp/delicious/eggs2.txt
/tmp/delicious/cats
/tmp/delicious/walnut
/tmp/delicious/cats/catnames.txt
/tmp/delicious/walnut/waffles
/tmp/delicious/walnut/waffles/butter.txt

Summary

Congratulations on completing the lab! You have learned how to effectively manage file and directory paths in Python using the modern pathlib module.

Throughout this lab, you have practiced:

  • Creating and joining paths in a cross-platform manner using Path objects and the / operator.
  • Getting the current working directory and creating nested directories with mkdir(parents=True).
  • Differentiating between absolute and relative paths and converting between them using is_absolute(), resolve(), and relative_to().
  • Validating paths with exists(), is_file(), and is_dir(), and inspecting file metadata with stat().
  • Performing essential file management tasks like copying, moving, and deleting with the shutil module and pathlib methods.
  • Traversing directory structures to list contents with iterdir() and recursively find items with rglob().

These skills are fundamental for writing robust and maintainable Python applications that interact with the file system.

Related Python Courses
Quick Start with Python

Quick Start with Python

python
Python for Beginners

Python for Beginners

python
Python Cheatsheet

Python Cheatsheet

python