Mastering Yield Statement Management in Python

Introduction

Objectives:

Managing what happens at the yield statements

Files Modified: follow.py, cofollow.py

Skills Graph

%%%%{init: {'theme':'neutral'}}%%%% flowchart RL python(("`Python`")) -.-> python/BasicConceptsGroup(["`Basic Concepts`"]) python(("`Python`")) -.-> python/FunctionsGroup(["`Functions`"]) python(("`Python`")) -.-> python/FileHandlingGroup(["`File Handling`"]) python(("`Python`")) -.-> python/ControlFlowGroup(["`Control Flow`"]) python(("`Python`")) -.-> python/DataStructuresGroup(["`Data Structures`"]) python(("`Python`")) -.-> python/ModulesandPackagesGroup(["`Modules and Packages`"]) python(("`Python`")) -.-> python/ErrorandExceptionHandlingGroup(["`Error and Exception Handling`"]) python(("`Python`")) -.-> python/AdvancedTopicsGroup(["`Advanced Topics`"]) python(("`Python`")) -.-> python/PythonStandardLibraryGroup(["`Python Standard Library`"]) python/BasicConceptsGroup -.-> python/comments("`Comments`") python/FunctionsGroup -.-> python/keyword_arguments("`Keyword Arguments`") python/FileHandlingGroup -.-> python/with_statement("`Using with Statement`") python/BasicConceptsGroup -.-> python/variables_data_types("`Variables and Data Types`") python/BasicConceptsGroup -.-> python/numeric_types("`Numeric Types`") python/BasicConceptsGroup -.-> python/booleans("`Booleans`") python/BasicConceptsGroup -.-> python/type_conversion("`Type Conversion`") python/ControlFlowGroup -.-> python/conditional_statements("`Conditional Statements`") python/ControlFlowGroup -.-> python/for_loops("`For Loops`") python/ControlFlowGroup -.-> python/while_loops("`While Loops`") python/ControlFlowGroup -.-> python/break_continue("`Break and Continue`") python/DataStructuresGroup -.-> python/tuples("`Tuples`") python/FunctionsGroup -.-> python/function_definition("`Function Definition`") python/ModulesandPackagesGroup -.-> python/importing_modules("`Importing Modules`") python/ModulesandPackagesGroup -.-> python/using_packages("`Using Packages`") python/ModulesandPackagesGroup -.-> python/standard_libraries("`Common Standard Libraries`") python/ErrorandExceptionHandlingGroup -.-> python/catching_exceptions("`Catching Exceptions`") python/FileHandlingGroup -.-> python/file_opening_closing("`Opening and Closing Files`") python/AdvancedTopicsGroup -.-> python/iterators("`Iterators`") python/AdvancedTopicsGroup -.-> python/generators("`Generators`") python/PythonStandardLibraryGroup -.-> python/os_system("`Operating System and System`") python/BasicConceptsGroup -.-> python/python_shell("`Python Shell`") python/FunctionsGroup -.-> python/build_in_functions("`Build-in Functions`") subgraph Lab Skills python/comments -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/keyword_arguments -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/with_statement -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/variables_data_types -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/numeric_types -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/booleans -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/type_conversion -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/conditional_statements -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/for_loops -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/while_loops -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/break_continue -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/tuples -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/function_definition -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/importing_modules -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/using_packages -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/standard_libraries -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/catching_exceptions -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/file_opening_closing -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/iterators -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/generators -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/os_system -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/python_shell -.-> lab-132525{{"`Yield Statement Management in Python`"}} python/build_in_functions -.-> lab-132525{{"`Yield Statement Management in Python`"}} end

Closing a Generator

A common question concerning generators is their lifetime and garbage collection. For example, the follow() generator runs forever in an infinite while loop. What happens if the iteration loop that's driving it stops? Also, is there anyway to prematurely terminate the generator?

Modify the follow() function so that all of the code is enclosed in a try-except block like this:

def follow(filename):
    try:
        with open(filename,'r') as f:
            f.seek(0,os.SEEK_END)
            while True:
                 line = f.readline()
                 if line == '':
                     time.sleep(0.1)    ## Sleep briefly to avoid busy wait
                     continue
                 yield line
    except GeneratorExit:
        print('Following Done')

Now, try a few experiments:

>>> from follow import follow
>>> ## Experiment: Garbage collection of a running generator
>>> f = follow('stocklog.csv')
>>> next(f)
'"MO",70.29,"6/11/2007","09:30.09",-0.01,70.25,70.30,70.29,365314\n'
>>> del f
Following Done
>>> ## Experiment: Closing a generator
>>> f = follow('stocklog.csv')
>>> for line in f:
        print(line,end='')
        if 'IBM' in line:
            f.close()

"VZ",42.91,"6/11/2007","09:34.28",-0.16,42.95,42.91,42.78,210151
"HPQ",45.76,"6/11/2007","09:34.29",0.06,45.80,45.76,45.59,257169
"GM",31.45,"6/11/2007","09:34.31",0.45,31.00,31.50,31.45,582429
...
"IBM",102.86,"6/11/2007","09:34.44",-0.21,102.87,102.86,102.77,147550
Following Done
>>> for line in f:
        print(line, end='')    ## No output: generator is done

>>>

In these experiments you can see that a GeneratorExit exception is raised when a generator is garbage-collected or explicitly closed via its close() method.

One additional area of exploration is whether or not you can resume iteration on a generator if you break out of a for-loop. For example, try this:

>>> f = follow('stocklog.csv')
>>> for line in f:
        print(line,end='')
        if 'IBM' in line:
            break

"CAT",78.36,"6/11/2007","09:37.19",-0.16,78.32,78.36,77.99,237714
"VZ",42.99,"6/11/2007","09:37.20",-0.08,42.95,42.99,42.78,268459
...
"IBM",102.91,"6/11/2007","09:37.31",-0.16,102.87,102.91,102.77,190859
>>> ## Resume iteration
>>> for line in f:
        print(line,end='')
        if 'IBM' in line:
            break

"AA",39.58,"6/11/2007","09:39.28",-0.08,39.67,39.58,39.31,243159
"HPQ",45.94,"6/11/2007","09:39.29",0.24,45.80,45.94,45.59,408919
...
"IBM",102.95,"6/11/2007","09:39.44",-0.12,102.87,102.95,102.77,225350
>>> del f
Following Done
>>>

In general, you can break out of running iteration and resume it later if you need to. You just need to make sure the generator object isn't forcefully closed or garbage collected somehow.

Raising Exceptions

In the file cofollow.py, you created a coroutine printer(). Modify the code to catch and report exceptions like this:

## cofollow.py
...
@consumer
def printer():
    while True:
        try:
            item = yield
            print(item)
        except Exception as e:
            print('ERROR: %r' % e)

Now, try an experiment:

>>> from cofollow import printer
>>> p = printer()
>>> p.send('hello')
hello
>>> p.send(42)
42
>>> p.throw(ValueError('It failed'))
ERROR: ValueError('It failed',)
>>> try:
        int('n/a')
    except ValueError as e:
        p.throw(e)

ERROR: ValueError("invalid literal for int() with base 10: 'n/a'",)
>>>

Notice how the running generator is not terminated by the exception. This is merely allowing the yield statement to signal an error instead of receiving a value.

Summary

Congratulations! You have completed the Yield Statements lab. You can practice more labs in LabEx to improve your skills.

Yield Statement Management in Python

Introduction

Skills Graph

Closing a Generator

Raising Exceptions

Summary

Other Python Tutorials you may like