Learn About Descriptors

Introduction

In this lab, you will learn about descriptors in Python, a powerful mechanism for customizing attribute access in objects. Descriptors allow you to define how attributes are accessed, set, and deleted, giving you control over object behavior and enabling validation logic implementation.

The objectives of this lab include understanding the descriptor protocol, creating and using custom descriptors, implementing data validation with descriptors, and optimizing descriptor implementations. You will create several files during the lab, including descrip.py, stock.py, and validate.py.

Understanding the Descriptor Protocol

In this step, we're going to learn how descriptors work in Python by creating a simple Stock class. Descriptors in Python are a powerful feature that allow you to customize how attributes are accessed, set, and deleted. The descriptor protocol consists of three special methods: __get__(), __set__(), and __delete__(). These methods define how the descriptor behaves when an attribute is accessed, assigned a value, or deleted, respectively.

First, we need to create a new file called stock.py in the project directory. This file will contain our Stock class. Here's the code you should put in the stock.py file:

## stock.py

class Stock:
    __slots__ = ['_name', '_shares', '_price']

    def __init__(self, name, shares, price):
        self.name = name
        self.shares = shares
        self.price = price

    @property
    def name(self):
        return self._name

    @name.setter
    def name(self, value):
        if not isinstance(value, str):
            raise TypeError('Expected a string')
        self._name = value

    @property
    def shares(self):
        return self._shares

    @shares.setter
    def shares(self, value):
        if not isinstance(value, int):
            raise TypeError('Expected an integer')
        if value < 0:
            raise ValueError('Expected a positive value')
        self._shares = value

    @property
    def price(self):
        return self._price

    @price.setter
    def price(self, value):
        if not isinstance(value, (int, float)):
            raise TypeError('Expected a number')
        if value < 0:
            raise ValueError('Expected a positive value')
        self._price = value

    def cost(self):
        return self.shares * self.price

    def sell(self, amount):
        self.shares -= amount

    def __repr__(self):
        return f'Stock({self.name!r}, {self.shares!r}, {self.price!r})'

In this Stock class, we're using the property decorator to define getter and setter methods for the name, shares, and price attributes. These getter and setter methods act as descriptors, which means they control how these attributes are accessed and set. For example, the setter methods validate the input values to ensure they are of the correct type and within an acceptable range.

Now that we have our stock.py file ready, let's open a Python shell to experiment with the Stock class and see how descriptors work in action. To do this, open your terminal and run the following commands:

cd ~/project
python3 -i stock.py

The -i option in the python3 command tells Python to start an interactive shell after executing the stock.py file. This way, we can directly interact with the Stock class we just defined.

In the Python shell, let's create a stock object and try accessing its attributes. Here's how you can do it:

s = Stock('GOOG', 100, 490.10)
s.name      ## Should return 'GOOG'
s.shares    ## Should return 100

When you access the name and shares attributes of the s object, Python is actually using the descriptor's __get__ method behind the scenes. The property decorators in our class are implemented using descriptors, which means they handle the access and assignment of attributes in a controlled way.

Let's take a closer look at the class dictionary to see the descriptor objects. The class dictionary contains all the attributes and methods defined in the class. You can view the keys of the class dictionary using the following code:

Stock.__dict__.keys()

You should see output similar to this:

dict_keys(['__module__', '__slots__', '__init__', 'name', '_name', 'shares', '_shares', 'price', '_price', 'cost', 'sell', '__repr__', '__doc__'])

The keys name, shares, and price represent the descriptor objects created by the property decorators.

Now, let's examine how descriptors work by manually calling their methods. We'll use the shares descriptor as an example. Here's how you can do it:

## Get the descriptor object for 'shares'
q = Stock.__dict__['shares']

## Use the __get__ method to retrieve the value
q.__get__(s, Stock)  ## Should return 100

## Use the __set__ method to set a new value
q.__set__(s, 75)
s.shares  ## Should now be 75

## Try to set an invalid value
try:
    q.__set__(s, '75')  ## Should raise TypeError
except TypeError as e:
    print(f"Error: {e}")

When you access an attribute like s.shares, Python calls the __get__ method of the descriptor to retrieve the value. When you assign a value like s.shares = 75, Python calls the __set__ method of the descriptor. The descriptor can then validate the data and raise errors if the input value is not valid.

Once you're done experimenting with the Stock class and descriptors, you can exit the Python shell by running the following command:

exit()

Creating Custom Descriptors

In this step, we're going to create our own descriptor class. But first, let's understand what a descriptor is. A descriptor is a Python object that implements the descriptor protocol, which consists of the __get__, __set__, and __delete__ methods. These methods allow the descriptor to manage how an attribute is accessed, set, and deleted. By creating our own descriptor class, we can better understand how this protocol works.

Create a new file called descrip.py in the project directory. This file will contain our custom descriptor class. Here's the code:

## descrip.py

class Descriptor:
    def __init__(self, name):
        self.name = name

    def __get__(self, instance, cls):
        print(f'{self.name}:__get__')
        ## In a real descriptor, you would return a value here

    def __set__(self, instance, value):
        print(f'{self.name}:__set__ {value}')
        ## In a real descriptor, you would store the value here

    def __delete__(self, instance):
        print(f'{self.name}:__delete__')
        ## In a real descriptor, you would delete the value here

In the Descriptor class, the __init__ method initializes the descriptor with a name. The __get__ method is called when the attribute is accessed, the __set__ method is called when the attribute is set, and the __delete__ method is called when the attribute is deleted.

Now, let's create a test file to experiment with our custom descriptor. This will help us see how the descriptor behaves in different scenarios. Create a file named test_descrip.py with the following code:

## test_descrip.py

from descrip import Descriptor

class Foo:
    a = Descriptor('a')
    b = Descriptor('b')
    c = Descriptor('c')

## Create an instance and try accessing the attributes
if __name__ == '__main__':
    f = Foo()
    print("Accessing attribute f.a:")
    f.a

    print("\nAccessing attribute f.b:")
    f.b

    print("\nSetting attribute f.a = 23:")
    f.a = 23

    print("\nDeleting attribute f.a:")
    del f.a

In the test_descrip.py file, we import the Descriptor class from descrip.py. Then we create a class Foo with three attributes a, b, and c, each managed by a descriptor. We create an instance of Foo and perform operations like accessing, setting, and deleting attributes to see how the descriptor methods are called.

Now let's run this test file to see the descriptors in action. Open your terminal, navigate to the project directory, and run the test file using the following commands:

cd ~/project
python3 test_descrip.py

You should see output like this:

Accessing attribute f.a:
a:__get__

Accessing attribute f.b:
b:__get__

Setting attribute f.a = 23:
a:__set__ 23

Deleting attribute f.a:
a:__delete__

As you can see, each time you access, set, or delete an attribute that is managed by a descriptor, the corresponding magic method (__get__, __set__, or __delete__) is called.

Let's also examine our descriptor interactively. This will allow us to test the descriptor in real - time and see the results immediately. Open your terminal, navigate to the project directory, and start an interactive Python session with the descrip.py file:

cd ~/project
python3 -i descrip.py

Now type these commands in the interactive Python session to see how the descriptor protocol works:

class Foo:
    a = Descriptor('a')
    b = Descriptor('b')
    c = Descriptor('c')

f = Foo()
f.a         ## Should call __get__
f.b         ## Should call __get__
f.a = 23    ## Should call __set__
del f.a     ## Should call __delete__
exit()

The key insight here is that descriptors provide a way to intercept and customize attribute access. This makes them powerful for implementing data validation, computed attributes, and other advanced behaviors. By using descriptors, you can have more control over how your class attributes are accessed, set, and deleted.

Implementing Validators Using Descriptors

In this step, we're going to create a validation system using descriptors. But first, let's understand what descriptors are and why we're using them. Descriptors are Python objects that implement the descriptor protocol, which includes the __get__, __set__, or __delete__ methods. They allow you to customize how an attribute is accessed, set, or deleted on an object. In our case, we'll use descriptors to create a validation system that ensures data integrity. This means that the data stored in our objects will always meet certain criteria, like being of a specific type or having a positive value.

Now, let's start creating our validation system. We'll create a new file called validate.py in the project directory. This file will contain the classes that implement our validators.

## validate.py

class Validator:
    def __init__(self, name):
        self.name = name

    @classmethod
    def check(cls, value):
        return value

    def __set__(self, instance, value):
        instance.__dict__[self.name] = self.check(value)


class String(Validator):
    expected_type = str

    @classmethod
    def check(cls, value):
        if not isinstance(value, cls.expected_type):
            raise TypeError(f'Expected {cls.expected_type}')
        return value


class PositiveInteger(Validator):
    expected_type = int

    @classmethod
    def check(cls, value):
        if not isinstance(value, cls.expected_type):
            raise TypeError(f'Expected {cls.expected_type}')
        if value < 0:
            raise ValueError('Expected a positive value')
        return value


class PositiveFloat(Validator):
    expected_type = float

    @classmethod
    def check(cls, value):
        if not isinstance(value, (int, float)):
            raise TypeError('Expected a number')
        if value < 0:
            raise ValueError('Expected a positive value')
        return float(value)

In the validate.py file, we first define a base class called Validator. This class has an __init__ method that takes a name parameter, which will be used to identify the attribute being validated. The check method is a class method that simply returns the value passed to it. The __set__ method is a descriptor method that is called when an attribute is set on an object. It calls the check method to validate the value and then stores the validated value in the object's dictionary.

We then define three subclasses of Validator: String, PositiveInteger, and PositiveFloat. Each of these subclasses overrides the check method to perform specific validation checks. The String class checks if the value is a string, the PositiveInteger class checks if the value is a positive integer, and the PositiveFloat class checks if the value is a positive number (either an integer or a float).

Now that we have our validators defined, let's modify our Stock class to use these validators. We'll create a new file called stock_with_validators.py and import the validators from the validate.py file.

## stock_with_validators.py

from validate import String, PositiveInteger, PositiveFloat

class Stock:
    name = String('name')
    shares = PositiveInteger('shares')
    price = PositiveFloat('price')

    def __init__(self, name, shares, price):
        self.name = name
        self.shares = shares
        self.price = price

    def cost(self):
        return self.shares * self.price

    def sell(self, amount):
        self.shares -= amount

    def __repr__(self):
        return f'Stock({self.name!r}, {self.shares!r}, {self.price!r})'

In the stock_with_validators.py file, we define the Stock class and use the validators as class attributes. This means that whenever an attribute is set on a Stock object, the corresponding validator's __set__ method will be called to validate the value. The __init__ method initializes the attributes of the Stock object, and the cost, sell, and __repr__ methods provide additional functionality.

Now, let's test our validator-based Stock class. We'll open a terminal, navigate to the project directory, and run the stock_with_validators.py file in interactive mode.

cd ~/project
python3 -i stock_with_validators.py

Once the Python interpreter is running, we can try some commands to test the validation system.

## Create a stock with valid values
s = Stock('GOOG', 100, 490.10)
print(s.name)    ## Should return 'GOOG'
print(s.shares)  ## Should return 100
print(s.price)   ## Should return 490.1

## Try changing to valid values
s.shares = 75
print(s.shares)  ## Should return 75

## Try setting invalid values
try:
    s.shares = '75'  ## Should raise TypeError
except TypeError as e:
    print(f"Error setting shares to string: {e}")

try:
    s.shares = -50  ## Should raise ValueError
except ValueError as e:
    print(f"Error setting negative shares: {e}")

exit()

In the test code, we first create a Stock object with valid values and print its attributes to verify that they are set correctly. We then try to change the shares attribute to a valid value and print it again to confirm the change. Finally, we try to set the shares attribute to an invalid value (a string and a negative number) and catch the exceptions that are raised by the validators.

Notice how our code is now much cleaner. The Stock class no longer needs to implement all those property methods - the validators handle all the type checking and constraints.

Descriptors have allowed us to create a reusable validation system that can be applied to any class attribute. This is a powerful pattern for maintaining data integrity across your application.

Improving Descriptor Implementation

In this step, we're going to enhance our descriptor implementation. You might have noticed that in some cases, we've been specifying names redundantly. This can make our code a bit messy and harder to maintain. To solve this problem, we'll use the __set_name__ method, a useful feature introduced in Python 3.6.

The __set_name__ method is called automatically when the class is defined. Its main job is to set the name of the descriptor for us, so we don't have to do it manually every time. This will make our code cleaner and more efficient.

Now, let's update your validate.py file to include the __set_name__ method. Here's how the updated code will look:

## validate.py

class Validator:
    def __init__(self, name=None):
        self.name = name

    def __set_name__(self, cls, name):
        ## This gets called when the class is defined
        ## It automatically sets the name of the descriptor
        if self.name is None:
            self.name = name

    @classmethod
    def check(cls, value):
        return value

    def __set__(self, instance, value):
        instance.__dict__[self.name] = self.check(value)


class String(Validator):
    expected_type = str

    @classmethod
    def check(cls, value):
        if not isinstance(value, cls.expected_type):
            raise TypeError(f'Expected {cls.expected_type}')
        return value


class PositiveInteger(Validator):
    expected_type = int

    @classmethod
    def check(cls, value):
        if not isinstance(value, cls.expected_type):
            raise TypeError(f'Expected {cls.expected_type}')
        if value < 0:
            raise ValueError('Expected a positive value')
        return value


class PositiveFloat(Validator):
    expected_type = float

    @classmethod
    def check(cls, value):
        if not isinstance(value, (int, float)):
            raise TypeError('Expected a number')
        if value < 0:
            raise ValueError('Expected a positive value')
        return float(value)

In the code above, the __set_name__ method in the Validator class checks if the name attribute is None. If it is, it sets the name to the actual attribute name used in the class definition. This way, we don't have to specify the name explicitly when creating instances of the descriptor classes.

Now that we've updated the validate.py file, we can create an improved version of our Stock class. This new version won't require us to specify the names redundantly. Here's the code for the improved Stock class:

## improved_stock.py

from validate import String, PositiveInteger, PositiveFloat

class Stock:
    name = String()  ## No need to specify 'name' anymore
    shares = PositiveInteger()
    price = PositiveFloat()

    def __init__(self, name, shares, price):
        self.name = name
        self.shares = shares
        self.price = price

    def cost(self):
        return self.shares * self.price

    def sell(self, amount):
        self.shares -= amount

    def __repr__(self):
        return f'Stock({self.name!r}, {self.shares!r}, {self.price!r})'

In this Stock class, we simply create instances of the String, PositiveInteger, and PositiveFloat descriptor classes without specifying the names. The __set_name__ method in the Validator class will take care of setting the names automatically.

Let's test our improved Stock class. First, open your terminal and navigate to the project directory. Then, run the improved_stock.py file in interactive mode. Here are the commands to do that:

cd ~/project
python3 -i improved_stock.py

Once you're in the interactive Python session, you can try the following commands to test the functionality of the Stock class:

s = Stock('GOOG', 100, 490.10)
print(s.name)    ## Should return 'GOOG'
print(s.shares)  ## Should return 100
print(s.price)   ## Should return 490.1

## Try changing values
s.shares = 75
print(s.shares)  ## Should return 75

## Try invalid values
try:
    s.shares = '75'  ## Should raise TypeError
except TypeError as e:
    print(f"Error: {e}")

try:
    s.price = -10.5  ## Should raise ValueError
except ValueError as e:
    print(f"Error: {e}")

exit()

These commands create an instance of the Stock class, print its attributes, change the value of an attribute, and then try to set invalid values to see if the appropriate errors are raised.

The __set_name__ method automatically sets the descriptor's name when the class is defined. This makes your code cleaner and less redundant, as you no longer need to specify the attribute name twice.

This improvement demonstrates how Python's descriptor protocol continues to evolve, making it easier to write clean, maintainable code.

Summary

In this lab, you have learned about Python descriptors, a powerful feature enabling customization of attribute access in classes. You explored the descriptor protocol, including the __get__, __set__, and __delete__ methods. You also created a basic descriptor class to intercept attribute access and used descriptors to implement a validation system for data integrity.

Moreover, you enhanced your descriptors with the __set_name__ method to reduce redundancy. Descriptors are widely used in Python libraries and frameworks such as Django and SQLAlchemy. Understanding them provides deeper insights into Python and helps you write more elegant and maintainable code.