Mastering Exception Handling and Logging in Python

Introduction

Objectives:

Learn more about exception handling and logging

Files Modified: reader.py

Skills Graph

%%%%{init: {'theme':'neutral'}}%%%% flowchart RL python(("`Python`")) -.-> python/BasicConceptsGroup(["`Basic Concepts`"]) python(("`Python`")) -.-> python/FileHandlingGroup(["`File Handling`"]) python(("`Python`")) -.-> python/ControlFlowGroup(["`Control Flow`"]) python(("`Python`")) -.-> python/DataStructuresGroup(["`Data Structures`"]) python(("`Python`")) -.-> python/FunctionsGroup(["`Functions`"]) python(("`Python`")) -.-> python/ModulesandPackagesGroup(["`Modules and Packages`"]) python(("`Python`")) -.-> python/ErrorandExceptionHandlingGroup(["`Error and Exception Handling`"]) python(("`Python`")) -.-> python/AdvancedTopicsGroup(["`Advanced Topics`"]) python(("`Python`")) -.-> python/PythonStandardLibraryGroup(["`Python Standard Library`"]) python/BasicConceptsGroup -.-> python/comments("`Comments`") python/FileHandlingGroup -.-> python/with_statement("`Using with Statement`") python/BasicConceptsGroup -.-> python/variables_data_types("`Variables and Data Types`") python/BasicConceptsGroup -.-> python/numeric_types("`Numeric Types`") python/BasicConceptsGroup -.-> python/strings("`Strings`") python/ControlFlowGroup -.-> python/conditional_statements("`Conditional Statements`") python/ControlFlowGroup -.-> python/for_loops("`For Loops`") python/DataStructuresGroup -.-> python/lists("`Lists`") python/DataStructuresGroup -.-> python/tuples("`Tuples`") python/DataStructuresGroup -.-> python/dictionaries("`Dictionaries`") python/FunctionsGroup -.-> python/function_definition("`Function Definition`") python/FunctionsGroup -.-> python/default_arguments("`Default Arguments`") python/FunctionsGroup -.-> python/lambda_functions("`Lambda Functions`") python/ModulesandPackagesGroup -.-> python/importing_modules("`Importing Modules`") python/ModulesandPackagesGroup -.-> python/using_packages("`Using Packages`") python/ModulesandPackagesGroup -.-> python/standard_libraries("`Common Standard Libraries`") python/ErrorandExceptionHandlingGroup -.-> python/catching_exceptions("`Catching Exceptions`") python/FileHandlingGroup -.-> python/file_opening_closing("`Opening and Closing Files`") python/AdvancedTopicsGroup -.-> python/iterators("`Iterators`") python/PythonStandardLibraryGroup -.-> python/data_collections("`Data Collections`") python/BasicConceptsGroup -.-> python/python_shell("`Python Shell`") python/FunctionsGroup -.-> python/build_in_functions("`Build-in Functions`") subgraph Lab Skills python/comments -.-> lab-132507{{"`Exception Handling and Logging`"}} python/with_statement -.-> lab-132507{{"`Exception Handling and Logging`"}} python/variables_data_types -.-> lab-132507{{"`Exception Handling and Logging`"}} python/numeric_types -.-> lab-132507{{"`Exception Handling and Logging`"}} python/strings -.-> lab-132507{{"`Exception Handling and Logging`"}} python/conditional_statements -.-> lab-132507{{"`Exception Handling and Logging`"}} python/for_loops -.-> lab-132507{{"`Exception Handling and Logging`"}} python/lists -.-> lab-132507{{"`Exception Handling and Logging`"}} python/tuples -.-> lab-132507{{"`Exception Handling and Logging`"}} python/dictionaries -.-> lab-132507{{"`Exception Handling and Logging`"}} python/function_definition -.-> lab-132507{{"`Exception Handling and Logging`"}} python/default_arguments -.-> lab-132507{{"`Exception Handling and Logging`"}} python/lambda_functions -.-> lab-132507{{"`Exception Handling and Logging`"}} python/importing_modules -.-> lab-132507{{"`Exception Handling and Logging`"}} python/using_packages -.-> lab-132507{{"`Exception Handling and Logging`"}} python/standard_libraries -.-> lab-132507{{"`Exception Handling and Logging`"}} python/catching_exceptions -.-> lab-132507{{"`Exception Handling and Logging`"}} python/file_opening_closing -.-> lab-132507{{"`Exception Handling and Logging`"}} python/iterators -.-> lab-132507{{"`Exception Handling and Logging`"}} python/data_collections -.-> lab-132507{{"`Exception Handling and Logging`"}} python/python_shell -.-> lab-132507{{"`Exception Handling and Logging`"}} python/build_in_functions -.-> lab-132507{{"`Exception Handling and Logging`"}} end

Preparation

In the reader.py file, there is a central function convert_csv() that does most of the work. This function crashes if you run it on data with missing or bad data. For example:

$ python
>>> from reader import read_csv_as_dicts

>>> port = read_csv_as_dicts('missing.csv', types=[str, int, float])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "reader.py", line 24, in read_csv_as_dicts
    return csv_as_dicts(file, types, headers=headers)
  File "reader.py", line 13, in csv_as_dicts
    lambda headers, row: { name: func(val) for name, func, val in zip(headers, types, row) })
  File "reader.py", line 9, in convert_csv
    return list(map(lambda row: converter(headers, row), rows))
  File "reader.py", line 9, in <lambda>
    return list(map(lambda row: converter(headers, row), rows))
  File "reader.py", line 13, in <lambda>
    lambda headers, row: { name: func(val) for name, func, val in zip(headers, types, row) })
  File "reader.py", line 13, in <dictcomp>
    lambda headers, row: { name: func(val) for name, func, val in zip(headers, types, row) })
ValueError: invalid literal for int() with base 10: ''
>>>

Catching Exceptions

Instead of crashing on bad data, modify the code to issue a warning message instead. The final result should be a list of the rows that were successfully converted. For example:

>>> port = read_csv_as_dicts('missing.csv', types=[str, int, float])
Row 4: Bad row: ['C', '', '53.08']
Row 7: Bad row: ['DIS', '50', 'N/A']
Row 8: Bad row: ['GE', '', '37.23']
Row 13: Bad row: ['INTC', '', '21.84']
Row 17: Bad row: ['MCD', '', '51.11']
Row 19: Bad row: ['MO', '', '70.09']
Row 22: Bad row: ['PFE', '', '26.40']
Row 26: Bad row: ['VZ', '', '42.92']
>>> len(port)
20
>>>

Note: Making this change may be a bit tricky because of your previous use of the map() built-in function. You may have to abandon that approach since there's no easy way to catch and handle exceptions in map().

Logging

Modify the code so that warning messages are issued using the logging module. In addition, give optional debugging information indicating the reason for failure. For example:

>>> import reader
>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> port = reader.read_csv_as_dicts('missing.csv', types=[str, int, float])
WARNING:reader:Row 4: Bad row: ['C', '', '53.08']
DEBUG:reader:Row 4: Reason: invalid literal for int() with base 10: ''
WARNING:reader:Row 7: Bad row: ['DIS', '50', 'N/A']
DEBUG:reader:Row 7: Reason: could not convert string to float: 'N/A'
WARNING:reader:Row 8: Bad row: ['GE', '', '37.23']
DEBUG:reader:Row 8: Reason: invalid literal for int() with base 10: ''
WARNING:reader:Row 13: Bad row: ['INTC', '', '21.84']
DEBUG:reader:Row 13: Reason: invalid literal for int() with base 10: ''
WARNING:reader:Row 17: Bad row: ['MCD', '', '51.11']
DEBUG:reader:Row 17: Reason: invalid literal for int() with base 10: ''
WARNING:reader:Row 19: Bad row: ['MO', '', '70.09']
DEBUG:reader:Row 19: Reason: invalid literal for int() with base 10: ''
WARNING:reader:Row 22: Bad row: ['PFE', '', '26.40']
DEBUG:reader:Row 22: Reason: invalid literal for int() with base 10: ''
WARNING:reader:Row 26: Bad row: ['VZ', '', '42.92']
DEBUG:reader:Row 26: Reason: invalid literal for int() with base 10: ''
>>>

Summary

Congratulations! You have completed the Exception Handling and Logging lab. You can practice more labs in LabEx to improve your skills.

Exception Handling and Logging