Learn More About Closures

Introduction

In this lab, you will learn more about closures in Python. Closures are a powerful programming concept that enables functions to remember and access variables from their enclosing scope, even after the outer function has completed execution.

You will also understand closures as a data structure, explore them as a code generator, and discover how to implement type - checking with closures. This lab will help you uncover some of the more unusual and powerful aspects of Python closures.

Closures as a Data Structure

In Python, closures offer a powerful way to encapsulate data. Encapsulation means keeping data private and controlling access to it. With closures, you can create functions that manage and modify private data without having to use classes or global variables. Global variables can be accessed and modified from anywhere in your code, which can lead to unexpected behavior. Classes, on the other hand, require a more complex structure. Closures provide a simpler alternative for data encapsulation.

Let's create a file called counter.py to demonstrate this concept:

Open the WebIDE and create a new file named counter.py in the /home/labex/project directory. This is where we'll write the code that defines our closure-based counter.
Add the following code to the file:

def counter(value):
    """
    Create a counter with increment and decrement functions.

    Args:
        value: Initial value of the counter

    Returns:
        Two functions: one to increment the counter, one to decrement it
    """
    def incr():
        nonlocal value
        value += 1
        return value

    def decr():
        nonlocal value
        value -= 1
        return value

    return incr, decr

In this code, we define a function called counter(). This function takes an initial value as an argument. Inside the counter() function, we define two inner functions: incr() and decr(). These inner functions share access to the same value variable. The nonlocal keyword is used to tell Python that we want to modify the value variable from the enclosing scope (the counter() function). Without the nonlocal keyword, Python would create a new local variable inside the inner functions instead of modifying the value from the outer scope.

Now let's create a test file to see this in action. Create a new file named test_counter.py with the following content:

from counter import counter

## Create a counter starting at 0
up, down = counter(0)

## Increment the counter several times
print("Incrementing the counter:")
print(up())  ## Should print 1
print(up())  ## Should print 2
print(up())  ## Should print 3

## Decrement the counter
print("\nDecrementing the counter:")
print(down())  ## Should print 2
print(down())  ## Should print 1

In this test file, we first import the counter() function from the counter.py file. Then we create a counter starting at 0 by calling counter(0) and unpacking the returned functions into up and down. We then call the up() function several times to increment the counter and print the results. After that, we call the down() function to decrement the counter and print the results.

Run the test file by executing the following command in the terminal:

python3 test_counter.py

You should see the following output:

Incrementing the counter:
1
2
3

Decrementing the counter:
2
1

Notice how there is no class definition involved here. The up() and down() functions are manipulating a shared value that is neither a global variable nor an instance attribute. This value is stored in the closure, making it accessible only to the functions returned by counter().

This is an example of how closures can be used as a data structure. The enclosed variable value is maintained between function calls, and it's private to the functions that access it. This means that no other part of your code can directly access or modify this value variable, providing a level of data protection.

Closures as a Code Generator

In this step, we'll learn how closures can be used to generate code dynamically. Specifically, we'll build a type-checking system for class attributes using closures.

First, let's understand what closures are. A closure is a function object that remembers values in the enclosing scope even if they are not present in memory. In Python, closures are created when a nested function references a value from its enclosing function.

Now, we'll start implementing our type-checking system.

Create a new file named typedproperty.py in the /home/labex/project directory with the following code:

## typedproperty.py

def typedproperty(name, expected_type):
    """
    Create a property with type checking.

    Args:
        name: The name of the property
        expected_type: The expected type of the property value

    Returns:
        A property object that performs type checking
    """
    private_name = '_' + name

    @property
    def value(self):
        return getattr(self, private_name)

    @value.setter
    def value(self, val):
        if not isinstance(val, expected_type):
            raise TypeError(f'Expected {expected_type}')
        setattr(self, private_name, val)

    return value

In this code, the typedproperty function is a closure. It takes two arguments: name and expected_type. The @property decorator is used to create a getter method for the property, which retrieves the value of the private attribute. The @value.setter decorator creates a setter method that checks if the value being set is of the expected type. If not, it raises a TypeError.

Now let's create a class that uses these typed properties. Create a file named stock.py with the following code:

from typedproperty import typedproperty

class Stock:
    """A class representing a stock with type-checked attributes."""

    name = typedproperty('name', str)
    shares = typedproperty('shares', int)
    price = typedproperty('price', float)

    def __init__(self, name, shares, price):
        self.name = name
        self.shares = shares
        self.price = price

In the Stock class, we use the typedproperty function to create type-checked attributes for name, shares, and price. When we create an instance of the Stock class, the type checking will be applied automatically.

Let's create a test file to see this in action. Create a file named test_stock.py with the following code:

from stock import Stock

## Create a stock with correct types
s = Stock('GOOG', 100, 490.1)
print(f"Stock name: {s.name}")
print(f"Stock shares: {s.shares}")
print(f"Stock price: {s.price}")

## Try to set an attribute with the wrong type
try:
    s.shares = "hundred"  ## This should raise a TypeError
    print("Type check failed")
except TypeError as e:
    print(f"Type check succeeded: {e}")

In this test file, we first create a Stock object with correct types. Then we try to set the shares attribute to a string, which should raise a TypeError because the expected type is an integer.

Run the test file:

python3 test_stock.py

You should see output similar to:

Stock name: GOOG
Stock shares: 100
Stock price: 490.1
Type check succeeded: Expected <class 'int'>

This output shows that the type checking is working correctly.

Now, let's enhance typedproperty.py by adding convenience functions for common types. Add the following code to the end of the file:

def String(name):
    """Create a string property with type checking."""
    return typedproperty(name, str)

def Integer(name):
    """Create an integer property with type checking."""
    return typedproperty(name, int)

def Float(name):
    """Create a float property with type checking."""
    return typedproperty(name, float)

These functions are just wrappers around the typedproperty function, making it easier to create properties of common types.

Create a new file named stock_enhanced.py that uses these convenience functions:

from typedproperty import String, Integer, Float

class Stock:
    """A class representing a stock with type-checked attributes."""

    name = String('name')
    shares = Integer('shares')
    price = Float('price')

    def __init__(self, name, shares, price):
        self.name = name
        self.shares = shares
        self.price = price

This Stock class uses the convenience functions to create type-checked attributes, which makes the code more readable.

Create a test file test_stock_enhanced.py to test the enhanced version:

from stock_enhanced import Stock

## Create a stock with correct types
s = Stock('GOOG', 100, 490.1)
print(f"Stock name: {s.name}")
print(f"Stock shares: {s.shares}")
print(f"Stock price: {s.price}")

## Try to set an attribute with the wrong type
try:
    s.price = "490.1"  ## This should raise a TypeError
    print("Type check failed")
except TypeError as e:
    print(f"Type check succeeded: {e}")

This test file is similar to the previous one, but it tests the enhanced Stock class.

Run the test:

python3 test_stock_enhanced.py

You should see output similar to:

Stock name: GOOG
Stock shares: 100
Stock price: 490.1
Type check succeeded: Expected <class 'float'>

In this step, we've demonstrated how closures can be used to generate code. The typedproperty function creates property objects that perform type checking, and the String, Integer, and Float functions create specialized properties for common types.

Eliminating Property Names with Descriptors

In the previous step, when creating typed properties, we had to explicitly state the property names. This is redundant because the property names are already specified in the class definition. In this step, we'll use descriptors to get rid of this redundancy.

A descriptor in Python is a special object that controls how attribute access works. When you implement the __set_name__ method in a descriptor, it can automatically grab the attribute name from the class definition.

Let's start by creating a new file.

Create a new file named improved_typedproperty.py with the following code:

## improved_typedproperty.py

class TypedProperty:
    """
    A descriptor that performs type checking.

    This descriptor automatically captures the attribute name from the class definition.
    """
    def __init__(self, expected_type):
        self.expected_type = expected_type
        self.name = None

    def __set_name__(self, owner, name):
        ## This method is called when the descriptor is assigned to a class attribute
        self.name = name

    def __get__(self, instance, owner):
        if instance is None:
            return self
        return instance.__dict__.get(self.name)

    def __set__(self, instance, value):
        if not isinstance(value, self.expected_type):
            raise TypeError(f'Expected {self.expected_type}')
        instance.__dict__[self.name] = value

## Convenience functions
def String():
    """Create a string property with type checking."""
    return TypedProperty(str)

def Integer():
    """Create an integer property with type checking."""
    return TypedProperty(int)

def Float():
    """Create a float property with type checking."""
    return TypedProperty(float)

This code defines a descriptor class called TypedProperty that checks the type of values assigned to attributes. The __set_name__ method is called automatically when the descriptor is assigned to a class attribute. This allows the descriptor to capture the attribute name without us having to specify it manually.

Next, we'll create a class that uses these improved typed properties.

Create a new file named stock_improved.py that uses the improved typed properties:

from improved_typedproperty import String, Integer, Float

class Stock:
    """A class representing a stock with type-checked attributes."""

    ## No need to specify property names anymore
    name = String()
    shares = Integer()
    price = Float()

    def __init__(self, name, shares, price):
        self.name = name
        self.shares = shares
        self.price = price

Notice that we don't need to specify the property names when creating the typed properties. The descriptor will automatically get the attribute name from the class definition.

Now, let's test our improved class.

Create a test file test_stock_improved.py to test the improved version:

from stock_improved import Stock

## Create a stock with correct types
s = Stock('GOOG', 100, 490.1)
print(f"Stock name: {s.name}")
print(f"Stock shares: {s.shares}")
print(f"Stock price: {s.price}")

## Try setting attributes with wrong types
try:
    s.name = 123  ## Should raise TypeError
    print("Name type check failed")
except TypeError as e:
    print(f"Name type check succeeded: {e}")

try:
    s.shares = "hundred"  ## Should raise TypeError
    print("Shares type check failed")
except TypeError as e:
    print(f"Shares type check succeeded: {e}")

try:
    s.price = "490.1"  ## Should raise TypeError
    print("Price type check failed")
except TypeError as e:
    print(f"Price type check succeeded: {e}")

Finally, we'll run the test to see if everything works as expected.

Run the test:

python3 test_stock_improved.py

You should see output similar to:

Stock name: GOOG
Stock shares: 100
Stock price: 490.1
Name type check succeeded: Expected <class 'str'>
Shares type check succeeded: Expected <class 'int'>
Price type check succeeded: Expected <class 'float'>

In this step, we've made our type-checking system better by using descriptors and the __set_name__ method. This gets rid of the redundant property name specification, making the code shorter and less likely to have errors.

The __set_name__ method is a very useful feature of descriptors. It lets them automatically gather information about how they're used in a class definition. This can be used to create APIs that are easier to understand and use.

Summary

In this lab, you have learned about advanced aspects of closures in Python. First, you explored using closures as a data structure, which can encapsulate data and enable functions to maintain state between calls without relying on classes or global variables. Second, you saw how closures can act as a code generator, generating property objects with type checking for a more functional approach to attribute validation.

You also discovered how to use the descriptor protocol and the __set_name__ method to create elegant type - checking attributes that automatically capture their names from class definitions. These techniques showcase the power and flexibility of closures, allowing you to implement complex behaviors concisely. Understanding closures and descriptors gives you more tools for creating maintainable and robust Python code.