Handling Errors With try and except

Concepts covered: Lambda

What an uncaught error does

The traceback lesson showed what happens when a line fails: Python prints the traceback and the program stops on that line. Every line after it never runs. For a script that parses 1 field typed at the keyboard, stopping is fine. For a nightly job on the 40,000th line of a 2 million line export, stopping throws away 39,999 lines of finished work because 1 line held 12x where a number belonged.

raw = "12x"
count = int(raw)
print("parsed", count)

The last line of its traceback reads ValueError: invalid literal for int() with base 10: '12x'. ValueError is the exception's type, and the type is what an except clause matches on.

try and except

A try block holds the lines that might fail. An except block names an exception type and holds the lines to run when that type is raised inside the try. When the failing line raises, Python abandons the rest of the try block, runs the matching except block, and continues with the first line after it as though nothing had gone wrong.

raw = "12x"
try:
    count = int(raw)
    print("parsed", count)
except ValueError:
    print("could not parse", raw)
print("program continues")

Output:

could not parse 12x
program continues

print("parsed", count) never ran: the jump happened at int(raw), 1 line above it, and count was never assigned. Change raw to "12" and the output flips to parsed 12 followed by program continues, with the except block skipped entirely. The except block runs only when the try is abandoned, and the jump undoes nothing. Every line above the failing one has already run, and whatever those lines assigned or wrote stays done, which is why a try should hold only the lines that can fail. Below, the third line of the try fails after the first 2 have printed and assigned.

try:
    print("step 1 done")
    total = 5
    total = total + int("x")
    print("step 4 done")
except ValueError:
    print("handler sees total =", total)

Output:

step 1 done
handler sees total = 5

Keeping the error message

Writing except ValueError as err stores the exception object in err. Printing it gives the same message the traceback would have shown, so a log can record what went wrong while the program keeps going. Watch count in the run below: it keeps the 0 assigned before the try, because the assignment inside the try never completed.

raw = "12x"
count = 0
try:
    count = int(raw)
    print("parsed", count)
except ValueError as err:
    print("skipped:", err)
print("count is now", count)

skipped: invalid literal for int() with base 10: '12x'
count is now 0

Matching the right type

An except clause catches only the type it names. except ValueError does nothing for a ZeroDivisionError, which then travels on as if no try were there and stops the program. Pick the type from the last line of the traceback you are trying to survive.

Several except clauses can follow 1 try, each for a different type; Python checks them top to bottom and runs the first that matches. Writing except: with no type catches everything, including the NameError from a typo, so the bug hides behind a skipped message for weeks while the output comes out wrong. Name the type.

finally runs no matter what

A finally block after the except clauses runs whether the try succeeded, was rescued by an except, or raised something nothing caught. It is where cleanup goes: closing a file, releasing a connection, writing a final log line.

try:
    value = 10 / 0
    print("value is", value)
except ZeroDivisionError:
    print("division failed")
finally:
    print("cleanup runs either way")
print("still running")

Output:

division failed
cleanup runs either way
still running

Do

• Name the exception type you expect, taken from the traceback
• Keep the try block to the lines that can fail
• Print or log err so every skipped record is visible

Avoid

• Write a bare except: that swallows typos and real bugs alike
• Wrap a whole program in a single try
• Catch an error and continue with no message at all

Practice

What does this program print before it ends?

try:
    result = 10 / 0
except ValueError:
    print("bad value")
print("done")

• bad value, then done
• Nothing: the program stops with a ZeroDivisionError traceback
• done, then a ZeroDivisionError traceback
• done only, because the division is skipped

A handler can do more than print. When a field is missing or malformed, the except block can assign a fallback value, so the lines after the try always find the variable set. Below, an empty duration field becomes 0 seconds instead of stopping the job, and the program carries on with a usable number whichever block ran.

raw = ""
try:
    seconds = int(raw)
except ValueError:
    seconds = 0
print("seconds is", seconds)

About this lesson

This section is part of the Python Foundations: Advanced lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.