Circular and Dynamic Module Imports

Introduction

Objectives:

• Explore circular imports
• Dynamic module imports

Skills Graph

Preparation

In the last exercise, you split the tableformat.py file up into submodules. The last part of the resulting tableformat/formatter.py file has turned into a mess of imports.

## tableformat.py
...

class TableFormatter(ABC):
    @abstractmethod
    def headings(self, headers):
        pass

    @abstractmethod
    def row(self, rowdata):
        pass

from .formats.text import TextTableFormatter
from .formats.csv import CSVTableFormatter
from .formats.html import HTMLTableFormatter

...

def create_formatter(name, column_formats=None, upper_headers=False):
    if name == 'text':
        formatter_cls = TextTableFormatter
    elif name == 'csv':
        formatter_cls = CSVTableFormatter
    elif name == 'html':
        formatter_cls = HTMLTableFormatter
    else:
        raise RuntimeError('Unknown format %s' % name)

    if column_formats:
        class formatter_cls(ColumnFormatMixin, formatter_cls):
              formats = column_formats

    if upper_headers:
        class formatter_cls(UpperHeadersMixin, formatter_cls):
            pass

    return formatter_cls()

The imports in the middle of the file are required because the create_formatter() function needs them to find the appropriate classes. Really, the whole thing is a mess.

Circular Imports

Try moving the following import statements to the top of the formatter.py file:

## formatter.py

from .formats.text import TextTableFormatter
from .formats.csv import CSVTableFormatter
from .formats.html import HTMLTableFormatter

class TableFormatter(ABC):
    @abstractmethod
    def headings(self, headers):
        pass

    @abstractmethod
    def row(self, rowdata):
        pass

...

Observe that nothing works anymore. Try running the stock.py program and notice the error about TableFormatter not being defined. The order of import statements matters and you can't just move the imports anywhere you want.

Move the import statements back where they were. Sigh.

Subclass Registration

Try the following experiment and observe:

>>> from structly.tableformat.formats.text import TextTableFormatter
>>> TextTableFormatter.__module__
'structly.tableformat.formats.text'
>>> TextTableFormatter.__module__.split('.')[-1]
'text'
>>>

Modify the TableFormatter base class by adding a dictionary and an __init_subclass__() method:

class TableFormatter(ABC):
    _formats = { }

    @classmethod
    def __init_subclass__(cls):
        name = cls.__module__.split('.')[-1]
        TableFormatter._formats[name] = cls

    @abstractmethod
    def headings(self, headers):
        pass

    @abstractmethod
    def row(self, rowdata):
        pass

This makes the parent class track all of its subclasses. Check it out:

>>> from structly.tableformat.formatter import TableFormatter
>>> TableFormatter._formats
{'text': <class 'structly.tableformat.formats.text.TextTableFormatter'>,
 'csv': <class 'structly.tableformat.formats.csv.CSVTableFormatter'>,
 'html': <class 'structly.tableformat.formats.html.HTMLTableFormatter'>}
>>>

Modify the create_formatter() function to look up the class in this dictionary instead:

def create_formatter(name, column_formats=None, upper_headers=False):
    formatter_cls = TableFormatter._formats.get(name)
    if not formatter_cls:
        raise RuntimeError('Unknown format %s' % name)

    if column_formats:
        class formatter_cls(ColumnFormatMixin, formatter_cls):
              formats = column_formats

    if upper_headers:
        class formatter_cls(UpperHeadersMixin, formatter_cls):
            pass

    return formatter_cls()

Run the stock.py program. Make sure it still works after you've made these changes. Just a note that all of the import statements are still there. You've mainly just cleaned up the code a bit and eliminated the hard-wired class names.

Dynamic Imports

You're now ready for the final frontier. Delete the following import statements altogether:

## formatter.py
...

from .formats.text import TextTableFormatter     ## DELETE
from .formats.csv import CSVTableFormatter       ## DELETE
from .formats.html import HTMLTableFormatter     ## DELETE
...

Run your stock.py code again--it should fail with an error. It knows nothing about the text formatter. Fix it by adding this tiny fragment of code to create_formatter():

def create_formatter(name, column_formats=None, upper_headers=False):
    if name not in TableFormatter._formats:
        __import__(f'{__package__}.formats.{name}')
    ...

This code attempts a dynamic import of a formatter module if nothing is known about the name. The import alone (if it works) will register the class with the _formats dictionary and everything will just work. Magic!

Try running the stock.py code and make sure it works afterwards.

Summary

Congratulations! You have completed the Circular and Dynamic Module Imports lab. You can practice more labs in LabEx to improve your skills.