Python File I/O: Reading and Writing Files

Introduction

In this lab, you will learn the fundamental process of reading from and writing to files in Python. File I/O (Input/Output) is a crucial skill for any programmer, as it allows your programs to persist data, read configurations, and interact with the file system.

We will focus on using the with statement, which is the modern and recommended way to handle files in Python. It ensures that files are automatically closed even if errors occur, preventing resource leaks.

Throughout this lab, you will:

Understand the basic file I/O process.
Open and read files using different methods (read(), readlines(), and iteration).
Write new content to files and append content to existing files.

All operations will be performed within the LabEx WebIDE. Let's get started.

The File Reading and Writing Process

In this step, you will learn about the basic process of file I/O in Python and why the with statement is essential.

To read from or write to a file in Python, you generally follow three steps:

Open the file.
Read from or write to the file.
Close the file.

Manually managing the closing of a file can be error-prone. If your program encounters an error while the file is open, it might not close properly. To solve this, Python provides the with statement. It automatically handles the closing of the file for you, even if exceptions occur, ensuring that system resources are managed correctly.

Note: If you are not familiar with using WebIDE (VS Code like), please learn it first at this lab.

The basic syntax is:

with open('filename.txt', 'mode') as file_variable:
    ## Perform operations on the file

open(): The built-in function to open a file.
'filename.txt': The path to the file.
'mode': A character indicating how the file will be used (e.g., 'r' for read, 'w' for write).
file_variable: The variable you will use to interact with the file object.

This step is purely conceptual. In the next steps, you will apply this knowledge to perform actual file operations.

Opening and Reading Files

In this step, you will learn how to open and read files using the open() function. We will cover reading the entire file, reading it line by line into a list, and iterating over the lines, which is highly efficient for large files.

The setup script has already created two files for you: hi.txt and sonnet29.txt.

Reading an Entire File with `read()`

The read() method reads the entire content of a file into a single string.

In the file explorer on the left, open the read_file.py file.
Add the following code to it. This script opens hi.txt, reads its content, and prints it to the console.

## Read file using 'with' statement: automatically closes file when done
with open('/home/labex/project/hi.txt') as hello_file:
    hello_content = hello_file.read()  ## Read entire file content

print(hello_content)

To run the script, open a terminal and execute the following command:

python3 read_file.py

You should see the content of hi.txt printed.

Hello World!

Reading a File into a List of Lines with `readlines()`

The readlines() method reads all lines from a file and returns them as a list of strings. Each string in the list represents a line and includes the newline character \n at the end.

Modify read_file.py with the following code to read sonnet29.txt.

## readlines() method: returns list of strings, one per line
with open('/home/labex/project/sonnet29.txt') as sonnet_file:
    lines = sonnet_file.readlines()  ## Returns list with each line as a string

print(lines)

Run the script again:

python3 read_file.py

The output will be a list of strings, where each string is a line from the file.

['When, in disgrace with fortune and men\'s eyes,\n', 'I all alone beweep my outcast state,\n', 'And trouble deaf heaven with my bootless cries,\n', 'And look upon myself and curse my fate,\n']

Iterating Through a File Line by Line

For large files, reading the entire content into memory can be inefficient. A better approach is to iterate over the file object directly. This reads the file line by line without loading the whole file into memory.

Modify read_file.py one last time with the following code.

## Iterate through file line by line (memory efficient for large files)
with open('/home/labex/project/sonnet29.txt') as sonnet_file:
    for line in sonnet_file:  ## File object is iterable
        print(line, end='')  ## Print without extra newline

Note: We use end='' in the print() function to prevent adding an extra newline, since each line from the file already ends with one.

Run the script:

python3 read_file.py

This will print the content of the file, perfectly formatted.

When, in disgrace with fortune and men's eyes,
I all alone beweep my outcast state,
And trouble deaf heaven with my bootless cries,
And look upon myself and curse my fate,

Writing to Files

In this step, you will explore how to write data to files. We will cover two different modes: write mode ('w'), which overwrites any existing content, and append mode ('a'), which adds new data to the end of a file.

Writing to a File with 'w' Mode

When you open a file in write mode ('w'), a new file is created if it doesn't exist. If the file already exists, its entire content is erased before the new content is written.

In the file explorer, open the write_file.py file.
Add the following code. This script will create a file named bacon.txt and write a single line of text to it. The write() method returns the number of characters written.

## Write to file: 'w' mode overwrites existing file
with open('/home/labex/project/bacon.txt', 'w') as bacon_file:  ## 'w' = write mode
    chars_written = bacon_file.write('Hello world!\n')  ## Returns number of characters written

print(chars_written)

Run the script from the terminal:

python3 write_file.py

The output shows the number of characters written to the file.

You can now see a new file, bacon.txt, in your file explorer.

Appending to a File with 'a' Mode

Append mode ('a') is used to add data to the end of an existing file without deleting its current content. If the file does not exist, it will be created.

Modify write_file.py to append another line to bacon.txt.

## Append to file: 'a' mode appends to existing file
with open('/home/labex/project/bacon.txt', 'a') as bacon_file:  ## 'a' = append mode
    chars_written = bacon_file.write('Bacon is not a vegetable.')

print(chars_written)

Run the script again:

python3 write_file.py

The output shows the number of characters from the new string.

Verifying the Final Content

To confirm that both write and append operations worked correctly, let's read the content of bacon.txt.

Modify write_file.py one last time to read and print the content of bacon.txt.

with open('/home/labex/project/bacon.txt') as bacon_file:
    content = bacon_file.read()

print(content)

Run the script:

python3 write_file.py

The output should show the combined content from both the write and append operations.

Hello world!
Bacon is not a vegetable.

Summary

Congratulations on completing this lab on reading and writing files in Python!

In this lab, you have learned the essential skills for file I/O. You started by understanding the importance of the with statement for safe and automatic file handling.

You then practiced several ways to read files:

read() to get the entire content as a single string.
readlines() to get a list of all lines.
Iterating directly over the file object for a memory-efficient way to process large files line by line.

Finally, you explored how to write to files:

Using write mode ('w') to create a new file or overwrite an existing one.
Using append mode ('a') to add content to the end of a file without erasing its previous data.

These fundamental operations are building blocks for more complex applications that need to store data, read configurations, or process text files.