Python Debugging

Introduction

In computer programming, debugging is the essential process of finding and resolving defects or "bugs" that prevent a program from operating correctly. While bugs are an inevitable part of software development, Python provides a powerful set of tools to help you diagnose and fix them efficiently.

In this lab, you will learn several fundamental debugging techniques in Python. We will cover how to:

• Manually raise and handle exceptions to control program flow during errors.
• Capture detailed error tracebacks as strings for logging purposes.
• Use assertions as sanity checks to catch logical errors early.
• Implement a flexible logging system to trace your program's execution and diagnose issues without interrupting its flow.

Raising Exceptions

In this step, you will learn how to manually trigger exceptions using the raise statement. Exceptions are a way to signal that an error has occurred. A raise statement consists of the raise keyword, a call to the Exception() function, and a helpful error message string passed to Exception().

It's a common practice to raise exceptions within a function when its preconditions are not met, and then handle these exceptions in the code that calls the function using try and except blocks.

Note: If you are not familiar with using WebIDE (VS Code like), please learn it first at this lab.

Let's try this with an example.

1. Open the file raise_exception.py from the file explorer on the left.
2. Add the following code to the file. This code defines a function box_print that raises exceptions for invalid arguments and then calls this function with both valid and invalid inputs, catching the exceptions.

def box_print(symbol, width, height):
    if len(symbol) != 1:
      raise Exception('Symbol must be a single character string.')
    if width <= 2:
      raise Exception('Width must be greater than 2.')
    if height <= 2:
      raise Exception('Height must be greater than 2.')
    print(symbol * width)
    for i in range(height - 2):
        print(symbol + (' ' * (width - 2)) + symbol)
    print(symbol * width)

## Handle exceptions when calling the function
for sym, w, h in (('*', 4, 4), ('O', 20, 5), ('x', 1, 3), ('ZZ', 3, 3)):
    try:
        box_print(sym, w, h)
    except Exception as err:  ## Catch exception and print error message
        print('An exception happened: ' + str(err))

1. Now, run the script from the terminal to see how the exceptions are raised and handled.

python3 raise_exception.py

You will see the following output. The first two calls to box_print succeed, while the last two fail, triggering the except block to print the custom error messages.

****
*  *
*  *
****
OOOOOOOOOOOOOOOOOOOO
O                  O
O                  O
O                  O
OOOOOOOOOOOOOOOOOOOO
An exception happened: Width must be greater than 2.
An exception happened: Symbol must be a single character string.

Getting the Traceback as a String

When an unhandled exception occurs, Python prints a traceback, which is a detailed report of the function calls that led to the error. Sometimes, you want to handle an exception gracefully but still record the full traceback for later debugging. The traceback module allows you to do this.

In this step, you'll learn to use traceback.format_exc() to capture the traceback as a string and save it to a file.

1. Open the file get_traceback.py from the file explorer.
2. Add the following code. This script imports the traceback module, then uses a try...except block to catch an exception. Inside the except block, it writes the full traceback to a file named errorInfo.txt.

import traceback

try:
    raise Exception('This is the error message.')
except:
    with open('errorInfo.txt', 'w') as error_file:
        error_file.write(traceback.format_exc())  ## Write traceback to file
    print('The traceback info was written to errorInfo.txt.')

1. Execute the script in the terminal.

python3 get_traceback.py

The script will print a confirmation message:

The traceback info was written to errorInfo.txt.

1. Now, check the contents of the newly created errorInfo.txt file. You can open it from the file explorer or use the cat command in the terminal.

cat errorInfo.txt

The file will contain the full traceback, similar to this:

Traceback (most recent call last):
  File "/home/labex/project/get_traceback.py", line 4, in <module>
    raise Exception('This is the error message.')
Exception: This is the error message.

This technique is extremely useful for logging errors in production applications without crashing the program.

Using Assertions

Assertions are sanity checks to ensure your code is not doing something obviously wrong. An assert statement checks if a condition is true. If the condition is false, it raises an AssertionError, which should ideally crash your program. This "fail-fast" approach helps you identify the root cause of a bug more quickly.

An assert statement consists of the assert keyword, a condition to check, and an optional error message.

In this step, you'll use an assertion to validate a program's state.

1. Open the file use_assertions.py from the file explorer.
2. Add the following code. It checks the status of pod_bay_door_status. The first assertion passes silently, but the second one will fail and raise an AssertionError.

## assert statement: check condition, raise AssertionError if False
pod_bay_door_status = 'open'
assert pod_bay_door_status == 'open', 'The pod bay doors need to be "open".'  ## Passes

pod_bay_door_status = 'I\'m sorry, Dave. I\'m afraid I can\'t do that.'
assert pod_bay_door_status == 'open', 'The pod bay doors need to be "open".'  ## Raises AssertionError

1. Run the script from the terminal.

python3 use_assertions.py

Because the second assertion fails, the program will crash and you will see an AssertionError with your custom message.

Traceback (most recent call last):
  File "/home/labex/project/use_assertions.py", line 6, in <module>
    assert pod_bay_door_status == 'open', 'The pod bay doors need to be "open".'
AssertionError: The pod bay doors need to be "open".

Unlike exceptions, you should not handle assertions with try...except. An AssertionError signals a bug in the program that needs to be fixed. It's also worth noting that assertions can be globally disabled by running Python with the -O (optimize) flag, so they should not be used for input validation that must always be enforced.

Implementing Logging

While print() statements can be used for debugging, a more powerful and flexible approach is to use Python's built-in logging module. Logging allows you to track events as your program runs and categorize messages by severity.

In this step, you'll add logging to a program to help diagnose a bug.

1. First, open the file factorial_log.py from the file explorer.
2. Add the following code. This program attempts to calculate the factorial of a number but contains a bug. We've added logging.debug() calls to trace the values of variables at each step. The logging.basicConfig() function configures the logging system to display messages of level DEBUG and higher.

import logging
logging.basicConfig(level=logging.DEBUG, format=' %(asctime)s - %(levelname)s - %(message)s')
logging.debug('Start of program')

def factorial(n):
    logging.debug('Start of factorial(%s)' % (n))
    total = 1
    for i in range(1, n + 1):
        total *= i
        logging.debug('i is ' + str(i) + ', total is ' + str(total))
    logging.debug('End of factorial(%s)' % (n))
    return total

print(factorial(5))
logging.debug('End of program')

1. Run the script from the terminal.

python3 factorial_log.py

You will see detailed log output interspersed with the program's regular output.

 %(asctime)s - DEBUG - Start of program
 %(asctime)s - DEBUG - Start of factorial(5)
 %(asctime)s - DEBUG - i is 1, total is 1
 %(asctime)s - DEBUG - i is 2, total is 2
 %(asctime)s - DEBUG - i is 3, total is 6
 %(asctime)s - DEBUG - i is 4, total is 24
 %(asctime)s - DEBUG - i is 5, total is 120
 %(asctime)s - DEBUG - End of factorial(5)
120
 %(asctime)s - DEBUG - End of program

(Note: The original markdown had a bug in the example code. The corrected code is provided here for a better learning experience. The original bug was range(n + 1) which starts from 0, making the total always 0. The fix is range(1, n + 1).)

By examining the log messages, you can clearly see the value of i and total in each iteration of the loop, which is a powerful way to understand the program's flow and find bugs.

Configuring Logging Behavior

The logging module is highly configurable. You can control the severity level of messages to display, disable logging entirely, or redirect log messages to a file instead of the screen.

Logging Levels

There are five standard logging levels, each with a corresponding function:

When you set a logging level in basicConfig, Python will display messages at that level and all higher levels. For example, setting level=logging.INFO will show INFO, WARNING, ERROR, and CRITICAL messages, but not DEBUG.

Disabling Logging

After debugging, you might want to turn off all log messages without removing the logging calls from your code. You can do this with logging.disable(). It takes a level as an argument and disables all log messages at that level and below. For example, logging.disable(logging.CRITICAL) will suppress every log message.

Logging to a File

Instead of printing logs to the console, you can save them to a file by passing the filename keyword argument to logging.basicConfig().

Let's practice this.

1. Open the file configure_logging.py.
2. Add the following code. It configures logging to write all messages of level DEBUG and higher to a file named myProgramLog.txt.

import logging

## Configure logging to write to a file
logging.basicConfig(filename='myProgramLog.txt', level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')

logging.debug('This is a debug message.')
logging.info('This is an info message.')
logging.warning('This is a warning message.')

1. Run the script from the terminal.

python3 configure_logging.py

This time, you won't see any output in the terminal.

1. Check the contents of the myProgramLog.txt file.

cat myProgramLog.txt

You will see that all the log messages have been written to the file, complete with timestamps and severity levels.

2023-10-27 10:30:00,123 - DEBUG - This is a debug message.
2023-10-27 10:30:00,123 - INFO - This is an info message.
2023-10-27 10:30:00,123 - WARNING - This is a warning message.

This concludes our tour of basic Python debugging techniques.

Summary

In this lab, you have learned and practiced several essential Python debugging techniques. You started by using the raise statement to create and trigger custom exceptions, and handled them with try...except blocks. You then learned how to capture a full exception traceback as a string using traceback.format_exc() for logging purposes.

Next, you explored assert statements as a way to perform sanity checks and "fail fast" when your program enters an unexpected state. Finally, you delved into the logging module, learning how to configure it to display messages, control output with different logging levels, and write logs to a file for persistent records.

These skills are fundamental for writing robust, maintainable, and easier-to-debug Python applications.