Decorator Chaining and Parameterized Decorators

Introduction

In this lab, you will learn about decorators in Python, a powerful feature that can modify the behavior of functions and methods. Decorators are commonly used for tasks such as logging, performance measurement, access control, and type checking.

You will learn how to chain multiple decorators, create decorators that accept parameters, preserve function metadata when using decorators, and apply decorators to different types of class methods. The files you will work with are logcall.py, validate.py, and sample.py.

Skills Graph

Preserving Function Metadata in Decorators

In Python, decorators are a powerful tool that allows you to modify the behavior of functions. However, when you use a decorator to wrap a function, there's a little problem. By default, the original function's metadata, such as its name, documentation string (docstring), and annotations, gets lost. Metadata is important because it helps in introspection (examining the code's structure) and generating documentation. Let's first verify this issue.

Open your terminal in the WebIDE. We'll run some Python commands to see what happens when we use a decorator. The following commands will create a simple function add wrapped with a decorator and then print the function and its docstring.

cd ~/project
python3 -c "from logcall import logged; @logged
def add(x,y):
    'Adds two things'
    return x+y
    
print(add)
print(add.__doc__)"

When you run these commands, you'll see output similar to this:

<function wrapper at 0x...>
None

Notice that instead of showing the function name as add, it shows wrapper. And the docstring, which should be 'Adds two things', is None. This can be a big problem when you're using tools that rely on this metadata, like introspection tools or documentation generators.

Fixing the Problem with functools.wraps

Python's functools module comes to the rescue. It provides a wraps decorator that can help us preserve the function metadata. Let's see how we can modify our logged decorator to use wraps.

1. First, open the logcall.py file in the WebIDE. You can navigate to the project directory using the following command in the terminal:

cd ~/project

2. Now, update the logged decorator in logcall.py with the following code. The @wraps(func) decorator is the key here. It copies all the metadata from the original function func to the wrapper function.

from functools import wraps

def logged(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        print(f"Calling {func.__name__}")
        return func(*args, **kwargs)
    return wrapper

3. The @wraps(func) decorator does an important job. It takes all the metadata (like the name, docstring, and annotations) from the original function func and attaches it to the wrapper function. This way, when we use the decorated function, it will have the correct metadata.

4. Let's test our improved decorator. Run the following commands in the terminal:

python3 -c "from logcall import logged; @logged
def add(x,y):
    'Adds two things'
    return x+y
    
print(add)
print(add.__doc__)"

Now you should see:

<function add at 0x...>
Adds two things

Great! The function name and docstring are preserved. This means that our decorator is now working as expected, and the metadata of the original function is intact.

Fixing the validate.py Decorator

Now, let's apply the same fix to the validated decorator in validate.py. This decorator is used to validate the types of function arguments and the return value based on the function's annotations.

1. Open validate.py in the WebIDE.

2. Update the validated decorator with the @wraps decorator. The following code shows how to do it. The @wraps(func) decorator is added to the wrapper function inside the validated decorator to preserve the metadata.

from functools import wraps

class Integer:
    @classmethod
    def __instancecheck__(cls, x):
        return isinstance(x, int)

def validated(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        ## Get function annotations
        annotations = func.__annotations__
        ## Check arguments against annotations
        for arg_name, arg_value in zip(func.__code__.co_varnames, args):
            if arg_name in annotations and not isinstance(arg_value, annotations[arg_name]):
                raise TypeError(f'Expected {arg_name} to be {annotations[arg_name].__name__}')

        ## Run the function and get the result
        result = func(*args, **kwargs)

        ## Check the return value
        if 'return' in annotations and not isinstance(result, annotations['return']):
            raise TypeError(f'Expected return value to be {annotations["return"].__name__}')

        return result
    return wrapper

3. Let's test that our validated decorator now preserves metadata. Run the following commands in the terminal:

python3 -c "from validate import validated, Integer; @validated
def multiply(x: Integer, y: Integer) -> Integer:
    'Multiplies two integers'
    return x * y
    
print(multiply)
print(multiply.__doc__)"

You should see:

<function multiply at 0......>
Multiplies two integers

Now both decorators, logged and validated, properly preserve the metadata of the functions they decorate. This ensures that when you use these decorators, the functions will still have their original names, docstrings, and annotations, which is very useful for code readability and maintainability.

Creating Decorators with Arguments

So far, we've been using the @logged decorator, which always prints a fixed message. But what if you want to customize the message format? In this section, we'll learn how to create a new decorator that can accept arguments, giving you more flexibility in how you use decorators.

Understanding Parameterized Decorators

A parameterized decorator is a special type of function. Instead of directly modifying another function, it returns a decorator. The general structure of a parameterized decorator looks like this:

def decorator_with_args(arg1, arg2, ...):
    def actual_decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            ## Use arg1, arg2, ... here
            ## Call the original function
            return func(*args, **kwargs)
        return wrapper
    return actual_decorator

When you use @decorator_with_args(value1, value2) in your code, Python first calls decorator_with_args(value1, value2). This call returns the actual decorator, which is then applied to the function that follows the @ syntax. This two - step process is key to how parameterized decorators work.

Creating the logformat Decorator

Let's create a @logformat(fmt) decorator that takes a format string as an argument. This will allow us to customize the logging message.

1. Open logcall.py in the WebIDE and add the new decorator. The code below shows how to define both the existing logged decorator and the new logformat decorator:

from functools import wraps

def logged(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        print(f"Calling {func.__name__}")
        return func(*args, **kwargs)
    return wrapper

def logformat(fmt):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            print(fmt.format(func=func))
            return func(*args, **kwargs)
        return wrapper
    return decorator

In the logformat decorator, the outer function logformat takes a format string fmt as an argument. It then returns the decorator function, which is the actual decorator that modifies the target function.

2. Now, let's test our new decorator by modifying sample.py. The following code shows how to use both the logged and logformat decorators on different functions:

from logcall import logged, logformat

@logged
def add(x, y):
    "Adds two numbers"
    return x + y

@logged
def sub(x, y):
    "Subtracts y from x"
    return x - y

@logformat('{func.__code__.co_filename}:{func.__name__}')
def mul(x, y):
    "Multiplies two numbers"
    return x * y

Here, the add and sub functions use the logged decorator, while the mul function uses the logformat decorator with a custom format string.

3. Run the updated sample.py to see the results. Open your terminal and run the following command:

cd ~/project
python3 -c "import sample; print(sample.add(2, 3)); print(sample.mul(2, 3))"

You should see output similar to:

Calling add
5
sample.py:mul
6

This output shows that the logged decorator prints the function name as expected, and the logformat decorator uses the custom format string to print the file name and function name.

Redefining the logged Decorator Using logformat

Now that we have a more flexible logformat decorator, we can redefine our original logged decorator using it. This will help us reuse code and maintain a consistent logging format.

1. Update logcall.py with the following code:

from functools import wraps

def logformat(fmt):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            print(fmt.format(func=func))
            return func(*args, **kwargs)
        return wrapper
    return decorator

## Define logged using logformat
logged = lambda func: logformat("Calling {func.__name__}")(func)

Here, we use a lambda function to define the logged decorator in terms of the logformat decorator. The lambda function takes a function func and applies the logformat decorator with a specific format string.

2. Test that the redefined logged decorator still works. Open your terminal and run the following command:

cd ~/project
python3 -c "from logcall import logged; @logged
def greet(name):
    return f'Hello, {name}'
    
print(greet('World'))"

You should see:

Calling greet
Hello, World

This shows that the redefined logged decorator works as expected, and we've successfully reused the logformat decorator to achieve a consistent logging format.

Applying Decorators to Class Methods

Now, we're going to explore how decorators interact with class methods. This can be a bit tricky because Python has different types of methods: instance methods, class methods, static methods, and properties. Decorators are functions that take another function and extend the behavior of the latter function without explicitly modifying it. When applying decorators to class methods, we need to pay attention to how they work with these different method types.

Understanding the Challenge

Let's see what happens when we apply our @logged decorator to different types of methods. The @logged decorator is likely used to log information about the method calls.

1. Create a new file methods.py in the WebIDE. This file will contain our class with different types of methods decorated with the @logged decorator.

from logcall import logged

class Spam:
    @logged
    def instance_method(self):
        print("Instance method called")
        return "instance result"

    @logged
    @classmethod
    def class_method(cls):
        print("Class method called")
        return "class result"

    @logged
    @staticmethod
    def static_method():
        print("Static method called")
        return "static result"

    @logged
    @property
    def property_method(self):
        print("Property method called")
        return "property result"

In this code, we have a class Spam with four different types of methods. Each method is decorated with the @logged decorator, and some are also decorated with other built - in decorators like @classmethod, @staticmethod, and @property.

2. Let's test how it works. We'll run a Python command in the terminal to call these methods and see the output.

cd ~/project
python3 -c "from methods import Spam; s = Spam(); print(s.instance_method()); print(Spam.class_method()); print(Spam.static_method()); print(s.property_method)"

When you run this command, you may notice some issues:

• The @property decorator might not work correctly with our @logged decorator. The @property decorator is used to define a method as a property, and it has a specific way of working. When combined with the @logged decorator, there could be conflicts.
• The order of decorators matters for @classmethod and @staticmethod. The order in which decorators are applied can change the behavior of the method.

The Order of Decorators

When you apply multiple decorators, they are applied from bottom to top. This means that the decorator closest to the method definition is applied first, and then the ones above it are applied in sequence. For example:

@decorator1
@decorator2
def func():
    pass

This is equivalent to:

func = decorator1(decorator2(func))

In this example, decorator2 is applied to func first, and then decorator1 is applied to the result of decorator2(func).

Fixing the Decorator Order

Let's update our methods.py file to fix the decorator order. By changing the order of the decorators, we can make sure that each method works as expected.

from logcall import logged

class Spam:
    @logged
    def instance_method(self):
        print("Instance method called")
        return "instance result"

    @classmethod
    @logged
    def class_method(cls):
        print("Class method called")
        return "class result"

    @staticmethod
    @logged
    def static_method():
        print("Static method called")
        return "static result"

    @property
    @logged
    def property_method(self):
        print("Property method called")
        return "property result"

In this updated version:

• For instance_method, the order doesn't matter. Instance methods are called on an instance of the class, and the @logged decorator can be applied in any order without affecting its basic functionality.
• For class_method, we apply @classmethod after @logged. The @classmethod decorator changes the way the method is called, and applying it after @logged ensures that the logging works correctly.
• For static_method, we apply @staticmethod after @logged. Similar to the @classmethod, the @staticmethod decorator has its own behavior, and the order with the @logged decorator needs to be correct.
• For property_method, we apply @property after @logged. This ensures that the property behavior is maintained while also getting the logging functionality.

3. Let's test the updated code. We'll run the same command as before to see if the issues are fixed.

cd ~/project
python3 -c "from methods import Spam; s = Spam(); print(s.instance_method()); print(Spam.class_method()); print(Spam.static_method()); print(s.property_method)"

You should now see proper logging for all method types:

Calling instance_method
Instance method called
instance result
Calling class_method
Class method called
class result
Calling static_method
Static method called
static result
Calling property_method
Property method called
property result

Best Practices for Method Decorators

When working with method decorators, follow these best practices:

1. Apply method - transforming decorators (@classmethod, @staticmethod, @property) after your custom decorators. This ensures that the custom decorators can perform their logging or other operations first, and then the built - in decorators can transform the method as intended.
2. Be aware that the decorator execution happens at class definition time, not at method call time. This means that any setup or initialization code in the decorator will run when the class is defined, not when the method is called.
3. For more complex cases, you might need to create specialized decorators for different method types. Different method types have different behaviors, and a one - size - fits - all decorator may not work in all situations.

Creating a Type Enforcement Decorator with Arguments

In the previous steps, we learned about the @validated decorator. This decorator is used to enforce type annotations in Python functions. Type annotations are a way to specify the expected types of function arguments and return values. Now, we're going to take it a step further. We'll create a more flexible decorator that can accept type specifications as arguments. This means we can define the types we want for each argument and the return value in a more explicit way.

Understanding the Goal

Our goal is to create an @enforce() decorator. This decorator will allow us to specify type constraints using keyword arguments. Here's an example of how it will work:

@enforce(x=Integer, y=Integer, return_=Integer)
def add(x, y):
    return x + y

In this example, we're using the @enforce decorator to specify that the x and y arguments of the add function should be of type Integer, and the return value should also be of type Integer. This decorator will behave similarly to our previous @validated decorator, but it gives us more control over the type specifications.

Creating the enforce Decorator

1. First, open the validate.py file in the WebIDE. We'll add our new decorator to this file. Here's the code we'll add:

from functools import wraps

class Integer:
    @classmethod
    def __instancecheck__(cls, x):
        return isinstance(x, int)

def validated(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        ## Get function annotations
        annotations = func.__annotations__
        ## Check arguments against annotations
        for arg_name, arg_value in zip(func.__code__.co_varnames, args):
            if arg_name in annotations and not isinstance(arg_value, annotations[arg_name]):
                raise TypeError(f'Expected {arg_name} to be {annotations[arg_name].__name__}')

        ## Run the function and get the result
        result = func(*args, **kwargs)

        ## Check the return value
        if 'return' in annotations and not isinstance(result, annotations['return']):
            raise TypeError(f'Expected return value to be {annotations["return"].__name__}')

        return result
    return wrapper

def enforce(**type_specs):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            ## Check argument types
            for arg_name, arg_value in zip(func.__code__.co_varnames, args):
                if arg_name in type_specs and not isinstance(arg_value, type_specs[arg_name]):
                    raise TypeError(f'Expected {arg_name} to be {type_specs[arg_name].__name__}')

            ## Run the function and get the result
            result = func(*args, **kwargs)

            ## Check the return value
            if 'return_' in type_specs and not isinstance(result, type_specs['return_']):
                raise TypeError(f'Expected return value to be {type_specs["return_"].__name__}')

            return result
        return wrapper
    return decorator

Let's break down what this code does. The Integer class is used to define a custom type. The validated decorator checks the types of function arguments and the return value based on the function's type annotations. The enforce decorator is the new one we're creating. It takes keyword arguments that specify the types for each argument and the return value. Inside the wrapper function of the enforce decorator, we check if the types of the arguments and the return value match the specified types. If not, we raise a TypeError.

2. Now, let's test our new @enforce decorator. We'll run some test cases to see if it works as expected. Here's the code to run the tests:

cd ~/project
python3 -c "from validate import enforce, Integer

@enforce(x=Integer, y=Integer, return_=Integer)
def add(x, y):
    return x + y

## This should work
print(add(2, 3))

## This should raise a TypeError
try:
    print(add('2', 3))
except TypeError as e:
    print(f'Error: {e}')

## This should raise a TypeError
try:
    @enforce(x=Integer, y=Integer, return_=Integer)
    def bad_add(x, y):
        return str(x + y)
    print(bad_add(2, 3))
except TypeError as e:
    print(f'Error: {e}')"

In this test code, we first define an add function with the @enforce decorator. We then call the add function with valid arguments, which should work without any errors. Next, we call the add function with an invalid argument, which should raise a TypeError. Finally, we define a bad_add function that returns a value of the wrong type, which should also raise a TypeError.

When you run this test code, you should see output similar to the following:

5
Error: Expected x to be Integer
Error: Expected return value to be Integer

This output shows that our @enforce decorator is working correctly. It raises a TypeError when the types of the arguments or the return value don't match the specified types.

Comparing the Two Approaches

Both the @validated and @enforce decorators achieve the same goal of enforcing type constraints, but they do it in different ways.

1. The @validated decorator uses Python's built-in type annotations. Here's an example:

   @validated
   def add(x: Integer, y: Integer) -> Integer:
       return x + y

   With this approach, we specify the types directly in the function definition using type annotations. This is a built-in feature of Python, and it provides better support in Integrated Development Environments (IDEs). IDEs can use these type annotations to provide code completion, type checking, and other helpful features.

2. The @enforce decorator, on the other hand, uses keyword arguments to specify the types. Here's an example:

   @enforce(x=Integer, y=Integer, return_=Integer)
   def add(x, y):
       return x + y

   This approach is more explicit because we're directly passing the type specifications as arguments to the decorator. It can be useful when working with libraries that rely on other annotation systems.

Each approach has its own advantages. Type annotations are a native part of Python and offer better IDE support, while the @enforce approach gives us more flexibility and explicitness. You can choose the approach that best suits your needs depending on the project you're working on.

Summary

In this lab, you have learned how to create and use decorators effectively. You learned to preserve function metadata with functools.wraps, create parameter - accepting decorators, handle multiple decorators and understand their application order. You also learned to apply decorators to different class methods and create a type - enforcement decorator that takes arguments.

These decorator patterns are commonly used in Python frameworks like Flask, Django, and pytest. Mastering decorators will enable you to write more maintainable and reusable code. To further your learning, you can explore context managers, class - based decorators, using decorators for caching, and advanced type checking with decorators.