Metaclasses in Action: Unlocking the Power of Python's Metaprogramming

Introduction

Objectives:

Metaclasses in action
Explode your brain

Files Modified: structure.py, validate.py

Skills Graph

%%%%{init: {'theme':'neutral'}}%%%% flowchart RL python(("`Python`")) -.-> python/BasicConceptsGroup(["`Basic Concepts`"]) python(("`Python`")) -.-> python/FunctionsGroup(["`Functions`"]) python(("`Python`")) -.-> python/FileHandlingGroup(["`File Handling`"]) python(("`Python`")) -.-> python/ControlFlowGroup(["`Control Flow`"]) python(("`Python`")) -.-> python/DataStructuresGroup(["`Data Structures`"]) python(("`Python`")) -.-> python/ModulesandPackagesGroup(["`Modules and Packages`"]) python(("`Python`")) -.-> python/ObjectOrientedProgrammingGroup(["`Object-Oriented Programming`"]) python(("`Python`")) -.-> python/ErrorandExceptionHandlingGroup(["`Error and Exception Handling`"]) python(("`Python`")) -.-> python/PythonStandardLibraryGroup(["`Python Standard Library`"]) python/BasicConceptsGroup -.-> python/comments("`Comments`") python/FunctionsGroup -.-> python/keyword_arguments("`Keyword Arguments`") python/FileHandlingGroup -.-> python/with_statement("`Using with Statement`") python/BasicConceptsGroup -.-> python/variables_data_types("`Variables and Data Types`") python/BasicConceptsGroup -.-> python/numeric_types("`Numeric Types`") python/BasicConceptsGroup -.-> python/strings("`Strings`") python/ControlFlowGroup -.-> python/conditional_statements("`Conditional Statements`") python/ControlFlowGroup -.-> python/for_loops("`For Loops`") python/ControlFlowGroup -.-> python/list_comprehensions("`List Comprehensions`") python/DataStructuresGroup -.-> python/lists("`Lists`") python/DataStructuresGroup -.-> python/tuples("`Tuples`") python/DataStructuresGroup -.-> python/dictionaries("`Dictionaries`") python/DataStructuresGroup -.-> python/sets("`Sets`") python/FunctionsGroup -.-> python/function_definition("`Function Definition`") python/FunctionsGroup -.-> python/default_arguments("`Default Arguments`") python/FunctionsGroup -.-> python/lambda_functions("`Lambda Functions`") python/ModulesandPackagesGroup -.-> python/importing_modules("`Importing Modules`") python/ModulesandPackagesGroup -.-> python/using_packages("`Using Packages`") python/ModulesandPackagesGroup -.-> python/standard_libraries("`Common Standard Libraries`") python/ObjectOrientedProgrammingGroup -.-> python/classes_objects("`Classes and Objects`") python/ObjectOrientedProgrammingGroup -.-> python/constructor("`Constructor`") python/ObjectOrientedProgrammingGroup -.-> python/polymorphism("`Polymorphism`") python/ObjectOrientedProgrammingGroup -.-> python/encapsulation("`Encapsulation`") python/ErrorandExceptionHandlingGroup -.-> python/catching_exceptions("`Catching Exceptions`") python/ErrorandExceptionHandlingGroup -.-> python/raising_exceptions("`Raising Exceptions`") python/PythonStandardLibraryGroup -.-> python/data_collections("`Data Collections`") python/BasicConceptsGroup -.-> python/python_shell("`Python Shell`") python/FunctionsGroup -.-> python/build_in_functions("`Build-in Functions`") subgraph Lab Skills python/comments -.-> lab-132521{{"`Metaclasses in Action`"}} python/keyword_arguments -.-> lab-132521{{"`Metaclasses in Action`"}} python/with_statement -.-> lab-132521{{"`Metaclasses in Action`"}} python/variables_data_types -.-> lab-132521{{"`Metaclasses in Action`"}} python/numeric_types -.-> lab-132521{{"`Metaclasses in Action`"}} python/strings -.-> lab-132521{{"`Metaclasses in Action`"}} python/conditional_statements -.-> lab-132521{{"`Metaclasses in Action`"}} python/for_loops -.-> lab-132521{{"`Metaclasses in Action`"}} python/list_comprehensions -.-> lab-132521{{"`Metaclasses in Action`"}} python/lists -.-> lab-132521{{"`Metaclasses in Action`"}} python/tuples -.-> lab-132521{{"`Metaclasses in Action`"}} python/dictionaries -.-> lab-132521{{"`Metaclasses in Action`"}} python/sets -.-> lab-132521{{"`Metaclasses in Action`"}} python/function_definition -.-> lab-132521{{"`Metaclasses in Action`"}} python/default_arguments -.-> lab-132521{{"`Metaclasses in Action`"}} python/lambda_functions -.-> lab-132521{{"`Metaclasses in Action`"}} python/importing_modules -.-> lab-132521{{"`Metaclasses in Action`"}} python/using_packages -.-> lab-132521{{"`Metaclasses in Action`"}} python/standard_libraries -.-> lab-132521{{"`Metaclasses in Action`"}} python/classes_objects -.-> lab-132521{{"`Metaclasses in Action`"}} python/constructor -.-> lab-132521{{"`Metaclasses in Action`"}} python/polymorphism -.-> lab-132521{{"`Metaclasses in Action`"}} python/encapsulation -.-> lab-132521{{"`Metaclasses in Action`"}} python/catching_exceptions -.-> lab-132521{{"`Metaclasses in Action`"}} python/raising_exceptions -.-> lab-132521{{"`Metaclasses in Action`"}} python/data_collections -.-> lab-132521{{"`Metaclasses in Action`"}} python/python_shell -.-> lab-132521{{"`Metaclasses in Action`"}} python/build_in_functions -.-> lab-132521{{"`Metaclasses in Action`"}} end

The Final Frontier

In Exercise 7.3, we made it possible to define type-checked structures as follows:

from validate import String, PositiveInteger, PositiveFloat
from structure import Structure

class Stock(Structure):
    name = String()
    shares = PositiveInteger()
    price = PositiveFloat()

    @property
    def cost(self):
        return self.shares * self.price

    def sell(self, nshares: PositiveInteger):
        self.shares -= nshares

There are a lot of things going on under the covers. However, one annoyance concerns all of those type-name imports at the top (e.g., String, PositiveInteger, etc.). That's just the kind of thing that might lead to a from validate import * statement. One interesting thing about a metaclass is that it can be used to control the process by which a class gets defined. This includes managing the environment of a class definition itself. Let's tackle those imports.

The first step in managing all of the validator names is to collect them. Go to the file validate.py and modify the Validator base class with this extra bit of code involving __init_subclass__() again:

## validate.py

class Validator:
    ...

    ## Collect all derived classes into a dict
    validators = { }
    @classmethod
    def __init_subclass__(cls):
        cls.validators[cls.__name__] = cls

That's not much, but it's creating a little namespace of all of the Validator subclasses. Take a look at it:

>>> from validate import Validator
>>> Validator.validators
{'Float': <class 'validate.Float'>,
 'Integer': <class 'validate.Integer'>,
 'NonEmpty': <class 'validate.NonEmpty'>,
 'NonEmptyString': <class 'validate.NonEmptyString'>,
 'Positive': <class 'validate.Positive'>,
 'PositiveFloat': <class 'validate.PositiveFloat'>,
 'PositiveInteger': <class 'validate.PositiveInteger'>,
 'String': <class 'validate.String'>,
 'Typed': <class 'validate.Typed'>}
>>>

Now that you've done that, let's inject this namespace into namespace of classes defined from Structure. Define the following metaclass:

## structure.py
...

from validate import Validator
from collections import ChainMap

class StructureMeta(type):
    @classmethod
    def __prepare__(meta, clsname, bases):
        return ChainMap({}, Validator.validators)

    @staticmethod
    def __new__(meta, name, bases, methods):
        methods = methods.maps[0]
        return super().__new__(meta, name, bases, methods)

class Structure(metaclass=StructureMeta):
    ...

In this code, the __prepare__() method is making a special ChainMap mapping that consists of an empty dictionary and a dictionary of all of the defined validators. The empty dictionary that's listed first is going to collect all of the definitions made inside the class body. The Validator.validators dictionary is going to make all of the type definitions available to for use as descriptors or argument type annotations.

The __new__() method discards extra the validator dictionary and passes the remaining definitions onto the type constructor. It's ingenious, but it lets you drop the annoying imports:

## stock.py

from structure import Structure

class Stock(Structure):
    name = String()
    shares = PositiveInteger()
    price = PositiveFloat()

    @property
    def cost(self):
        return self.shares * self.price

    def sell(self, nshares: PositiveInteger):
        self.shares -= nshares

Stare in Amazement

Try running your teststock.py unit tests across this new file. Most of them should be passing now. For kicks, try your Stock class with some of the earlier code for tableformatting and reading data. It should all work.

>>> from stock import Stock
>>> from reader import read_csv_as_instances
>>> portfolio = read_csv_as_instances('portfolio.csv', Stock)
>>> portfolio
[Stock('AA',100,32.2), Stock('IBM',50,91.1), Stock('CAT',150,83.44), Stock('MSFT',200,51.23), Stock('GE',95,40.37), Stock('MSFT',50,65.1), Stock('IBM',100,70.44)]
>>> from tableformat import create_formatter, print_table
>>> formatter = create_formatter('text')
>>> print_table(portfolio, ['name','shares','price'], formatter)
      name     shares      price
---------- ---------- ----------
        AA        100       32.2
       IBM         50       91.1
       CAT        150      83.44
      MSFT        200      51.23
        GE         95      40.37
      MSFT         50       65.1
       IBM        100      70.44
>>>

Again, marvel at the final stock.py file and observe how clean the code looks. Just try not think about everything that is happening under the hood with the Structure base class.

Summary

Congratulations! You have completed the Metaclasses in Action lab. You can practice more labs in LabEx to improve your skills.

Metaclasses in Action

Introduction

Skills Graph

The Final Frontier

Stare in Amazement

Summary

Other Python Tutorials you may like