Functional Programming: Intermediate
Scikit-learn's data preprocessing pipelines use functools.partial and higher-order functions to configure reusable transformation steps that work identically in development, staging, and production without any environment-specific code. Data scientists at companies like Instacart compose these partial functions into pipelines where each step receives data from the previous one, making complex ML preprocessing both testable and reproducible. The functools patterns in this lesson, including partial application and reduce(), are how professional Python developers build flexible, composable systems.
Keyword Arguments
Call functions with named arguments
- Positional - matched to parameters by their position in the call
- Keyword - matched by explicitly naming the parameter
- You can mix both, but positional arguments must come first
Positional vs Keyword Args
Why Use Keyword Arguments?
- calculate(100, 0.05, True, False)
- What do these arguments mean?
- Easy to mix up the order
- Must look at function definition
- calculate(
- amount=100,
- rate=0.05,
- compound=True,
- round_result=False)
Optional Parameter Skipping
Keyword-Only Arguments
You can require certain arguments to be passed only by name using * in the parameter list. Everything after the * must be provided as a keyword argument:
> A greet function has parameters name, greeting="Hi", and punctuation="!". You want to change the greeting to "Welcome" while keeping the default punctuation. Pick the right calling syntax.
def greet(name, greeting="Hi", punctuation="!"): return greeting + ", " + name + punctuation result = greet("Alice", ) print(result)
Positional and Keyword Mix
Multiple Return Values
Return multiple values from functions
- Write return a, b, c to return multiple values
- Python automatically creates a tuple (a, b, c)
- The caller can unpack into separate variables or keep as a tuple
Returning Multiple Values
Separate return values with commas. Python automatically creates a tuple:
Multiple Return Patterns
Ignoring Some Return Values
Sometimes you only need some of the returned values. Use _ as a placeholder for values you want to ignore:
The _ is a valid variable name but signals "I am intentionally ignoring this value." The *rest syntax collects remaining values into a list.
Success Status with Return
> A function returns three values: total, average, and count. Pick the correct built-in to compute the total of all numbers, and the built-in that counts how many elements are in the list.
def get_stats(nums): total = (nums) avg = total / (nums) return total, avg, len(nums) result = get_stats([10, 20, 30]) total, avg, count = result print(total) print(count)
Nested Functions
Encapsulate helpers with nesting
- Hide implementation details only relevant to the containing function
- Keep related code together and avoid namespace pollution
- Access variables from the enclosing function's scope
Basic Nested Function
The functions add_title and add_punctuation exist only inside format_greeting. They are helpers that only make sense in that context. Notice how they can access formal from the enclosing function.
Organizing Complex Logic
- Encapsulation: Hide helpers that are only used internally
- Organization: Keep related code physically together
- Namespace cleanliness: Avoid cluttering module scope
- Scope access: Inner functions can use outer variables
- Readability: Break complex logic into named pieces
> A nested helper function accesses the threshold from its enclosing scope. Pick the built-in that keeps only items passing the test, and the one that counts how many survived.
def process(items, threshold=10): def is_valid(x): return x >= threshold result = list((is_valid, items)) return (result) print(process([5, 15, 8, 20, 3]))
Closures
Build function factories with closures
- An inner function references variables from an enclosing function
- That inner function is returned or passed elsewhere
- The inner function closes over the enclosing variables, keeping them alive
Creating a Closure
Each call to make_multiplier creates a new multiply function that remembers the factor value from when it was created. Even though make_multiplier has finished executing, the returned function still has access to factor.
How Closures Work
- Inner function references variable from enclosing scope
- Python captures the variable itself, not just its current value
- Captured variables are stored in the function's closure
- Each call to outer function creates a new closure
- Closures can share captured variables with other closures
Function Factory Pattern
Closures with Mutable State
The nonlocal keyword tells Python to modify the count variable from the enclosing scope, rather than creating a new local variable. Without nonlocal, the assignment count = count + 1 would create a new local variable.
nonlocal when you need to modify an enclosing variable, not just read it. Reading works automatically; writing requires explicit declaration.Configurable Validators
> This closure tries to maintain a mutable counter, but assigning to count inside the inner function creates a new local variable instead of updating the enclosing one.
UnboundLocalError: cannot access local variable "count" where it is not associated with a value
- Use closures for lightweight state management
- Always declare nonlocal when modifying enclosing variables
- Name inner functions descriptively
- Keep closures focused on a single responsibility
- Use closures when a simple class would be clearer
- Forget nonlocal when assigning to outer variables
- Create deeply nested closures that are hard to follow
- Rely on closures for complex state with many variables
The nonlocal keyword is the key distinction between reading and writing enclosing variables. Reading works automatically, but writing requires an explicit nonlocal declaration so Python knows not to create a new local variable.
Closures provide an alternative to classes when you only need state associated with a single behavior. A factory that returns a configured closure is lighter-weight than a class with __init__ and a single method.
Docstrings
Document functions with docstrings
A docstring is a string literal that appears as the first statement in a function, class, or module. It documents what the code does, what parameters it expects, and what it returns. Unlike comments, docstrings are accessible at runtime and used by tools like help() and documentation generators.
Basic Docstrings
The docstring becomes the function's __doc__ attribute. IDEs, the help() function, and documentation generators all use this attribute.
Multi-line Docstrings
Docstring Structure
- Summary line: Brief description ending with a period (first line)
- Blank line: Separates summary from detailed description
- Detailed description: Explains behavior, context, side effects
- Args section: Documents each parameter with type and description
- Returns section: Describes return value(s) and their types
- Raises section: Lists exceptions that might be raised
- Example section: Shows usage with expected output
Docstring Best Practices
- """Does stuff."""
- Too vague - what stuff?
- No parameter documentation
- No return value info
- """Calculate monthly payment.
- Args:
- principal: Loan amount in dollars
- rate: Annual interest rate"""
When to Write Docstrings
- Public functions: Always document thoroughly
- Private helpers: Brief docstring if logic is complex
- Library code: Comprehensive docs with examples
- Internal scripts: At minimum, describe what the function does
- Simple one-liners: Docstring optional if purpose is obvious
> You are a data engineer at Palantir building a configurable data validation library where each check function is self-documenting, returns both a pass/fail flag and a diagnostic message, uses internal helper logic, and carries its configuration in a captured environment.
return values let every check function return both a boolean pass/fail and a human-readable diagnostic string in a single tuple without a wrapper class.nonlocal to modify captured variablesFunctions as building blocks
- Category
- Python
- Difficulty
- intermediate
- Duration
- 31 minutes
- Challenges
- 0 hands-on challenges
Topics covered: Keyword Arguments, Multiple Return Values, Nested Functions, Closures, Docstrings
Lesson Sections
- Keyword Arguments (concepts: pyArgs)
When calling a function, you can specify arguments by name instead of relying on position. This makes code clearer, especially for functions with many parameters or when the meaning of arguments is not obvious. Keyword arguments also let you skip optional parameters and provide only the ones you need. Positional vs Keyword Args You have been using positional arguments exclusively until now, where the order of values determines which parameter receives which value. Keyword arguments let you expli
- Multiple Return Values (concepts: pyUnpacking)
Functions often need to produce multiple related results. Python makes this natural by allowing functions to return multiple values at once. Python packages them into a tuple, which you can unpack into separate variables. This pattern is common throughout Python's standard library and professional codebases. Returning Multiple Values The return statement creates a tuple. You can unpack it immediately into separate variables (most common) or store the tuple for later access. Both approaches are v
- Nested Functions (concepts: pyFuncScope)
You can define functions inside other functions. The inner function, called a nested function, is only accessible within the outer function. This keeps helper logic private and encapsulated, preventing pollution of the broader namespace. Basic Nested Function Define helper functions inside the main function when they are only meaningful in that context: Organizing Complex Logic Nested functions help organize complex functions into clear, named steps without creating module-level functions that a
- Closures (concepts: pyFuncScope)
A closure is a function that remembers variables from the scope where it was created, even after that scope has finished executing. This powerful pattern lets you create customized functions and maintain state between calls without using global variables or classes. Creating a Closure When a nested function references a variable from its enclosing function and is returned, Python creates a closure that captures that variable: How Closures Work When you call make_multiplier(2), Python creates a n
- Docstrings (concepts: pyFuncDef)
Basic Docstrings Use triple quotes for docstrings. A simple docstring is a single line that describes what the function does: Multi-line Docstrings For functions with parameters, return values, or complex behavior, use a multi-line docstring that documents the interface completely: This multiline docstring format provides everything a user needs: a summary, detailed description, parameter documentation, return value, exceptions, and usage examples. Docstring Structure A well-structured docstring