Local vs Global Scope

Concepts covered: Scope

Every call to a function gets its own private set of names, called its local scope. Parameters live there, along with every name the function assigns, and Python discards the whole set when the call returns. Names assigned at the top level of the file live in the global scope, which lasts for the whole run. Scope rules decide which of 2 same-named variables a line of code sees.

rows_loaded = 0

def load(table, count):
    rows_loaded = count
    print("inside load:", table, rows_loaded)

def report():
    print("inside report:", rows_loaded)

load("orders", 40)
report()
print("outside:", rows_loaded)
inside load: orders 40
inside report: 0
outside: 0

The assignment inside load created a second rows_loaded, local to that call, and set it to 40. The global one never changed. report only reads the name, so it finds the global 0, and the last line prints 0 too. A local name that hides a global with the same spelling is called shadowing.

Where Python looks up a name

When a function defined at the top level of a file reads a name, Python searches 3 places in order and uses the first match. Every function in this lesson is defined that way:

OrderScopeWhat lives there
1LocalParameters and names assigned in the current call, such as `table` and `count`
2GlobalNames assigned at the top level of the file, such as `rows_loaded`
3Built-inNames Python provides, such as `print`, `range` and `isinstance`

A name found in none of them raises NameError. That includes a function's local after the call has returned: once load finishes, count exists nowhere, so a top-level print(count) fails with NameError: name 'count' is not defined.

Assignment decides scope for the whole body

Python decides which names are local before the function runs, by scanning its body. A name assigned anywhere in the body, including through +=, is local everywhere in that body, even on lines above the assignment. Predict what this does:

What happens when this program runs?

processed = 0

def mark_done():
    processed += 1
    print(processed)

mark_done()

Python makes this decision even when the assignment never runs. Below, report(False) skips the if body, yet print(mode) still raises the same UnboundLocalError. The line mode = "overwrite" exists in the body, and that alone makes mode local throughout:

mode = "append"

def report(force):
    print(mode)
    if force:
        mode = "overwrite"

report(False)

Changing a global on purpose

A global statement at the top of a function tells Python that a name means the global variable, so assignments update it instead of creating a local:

processed = 0

def mark_done():
    global processed
    processed += 1

mark_done()
mark_done()
print(processed)

Output:

2

That works, but now any function can change processed, and tracking down a wrong count means checking every function that declares it global. Passing the value in and returning the new one keeps each change visible where the call happens:

def mark_done(processed):
    return processed + 1

processed = 0
processed = mark_done(processed)
processed = mark_done(processed)
print(processed)

Output:

2
With global
  • mark_done() changes processed, and the call line doesn't show it
  • Finding every writer means searching each function for global processed
  • Every pipeline that calls it shares one counter
Pass in, return out
  • processed = mark_done(processed) shows the input and the update
  • The function depends only on its parameters, so it can be tested alone
  • Each caller picks the counter to update, so 2 jobs can keep separate counts

About this lesson

This section is part of the Functions: Intermediate lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.