Python Practice for Data Engineers

Write a function from scratch given a specification. These test your ability to translate requirements into working code. Topics include string manipulation, dictionary operations, list processing, set operations, and basic algorithms.

# Example: Group records by a composite key
def group_by_key(records: list[dict], keys: list[str]) -> dict:
    """Group a list of dicts by composite key.

    Args:
        records: List of dictionaries with consistent keys
        keys: List of key names to group by

    Returns:
        Dict mapping tuple of key values to list of records
    """
    groups = {}
    for record in records:
        key = tuple(record[k] for k in keys)
        if key not in groups:
            groups[key] = []
        groups[key].append(record)
    return groups

You receive broken code and need to find and fix the bug. These test your ability to read code carefully, trace execution, and spot common Python pitfalls: off-by-one errors, mutable default arguments, incorrect type handling, missing edge cases, and silent failures.

# Example: Find and fix the bug
def deduplicate(records, key_field):
    """Remove duplicate records based on key_field.
    Keep the first occurrence."""
    seen = set()
    result = []
    for record in records:
        key = record[key_field]
        if key not in seen:
            result.append(record)
            # BUG: forgot to add key to seen set
            # FIX: seen.add(key)
    return result

Transform data from one structure to another. These mirror real pipeline work: flattening nested JSON, pivoting rows to columns, merging datasets, computing aggregations, and reshaping data for downstream consumers. The input is always a Python data structure (list of dicts, nested dict, etc.) and the output is a different structure.

# Example: Flatten nested event data
def flatten_events(events: list[dict]) -> list[dict]:
    """Flatten nested event payloads into flat records.

    Input:  [{"event": "click", "ts": "2024-01-01",
              "props": {"page": "/home", "button": "cta"}}]
    Output: [{"event": "click", "ts": "2024-01-01",
              "page": "/home", "button": "cta"}]
    """
    flat = []
    for event in events:
        row = {k: v for k, v in event.items() if k != "props"}
        if "props" in event and isinstance(event["props"], dict):
            row.update(event["props"])
        flat.append(row)
    return flat

Implement components of a data pipeline: validators, parsers, routers, retry logic, batching functions, and schema enforcers. These test whether you can write production-quality code that handles failures gracefully.

# Example: Validate and route records
def validate_and_route(records: list[dict],
                       required_fields: list[str]) -> dict:
    """Split records into valid and invalid based on required fields.

    Returns: {"valid": [...], "invalid": [...]}
    """
    valid = []
    invalid = []
    for record in records:
        missing = [f for f in required_fields if f not in record
                   or record[f] is None]
        if missing:
            record["_missing_fields"] = missing
            invalid.append(record)
        else:
            valid.append(record)
    return {"valid": valid, "invalid": invalid}

Handling Messy Data

Real data has nulls, empty strings, inconsistent types, and unexpected formats. Interviewers give you slightly messy input on purpose. They want to see if you check for None before accessing attributes, handle empty collections without crashing, and validate input types. A solution that works on clean data but crashes on an empty list scores poorly.

Pythonic Patterns

Interviewers notice if you write Java-style Python. Using enumerate instead of manual index tracking, dict.get() instead of key-check-then-access, list comprehensions instead of manual loops for simple transforms, and f-strings instead of string concatenation. These patterns signal experience with the language.

Choosing the Right Data Structure

Using a set for membership testing instead of scanning a list. Using a defaultdict instead of checking if a key exists. Using a Counter instead of manual counting. The right data structure choice often simplifies the code by 50% and shows the interviewer you think about performance naturally.

Clear Error Messages

In pipeline code, silent failures are worse than crashes. If your function receives invalid input, raising a ValueError with a descriptive message is better than returning None or an empty result. Interviewers who have built real pipelines value this because they have debugged silent data loss caused by swallowed exceptions.

Phone screen level. Single-function problems with clear specs. Basic data structure operations, simple string manipulation, straightforward transformations. If you struggle with these, focus here before moving up.

On-site level. Multi-step logic, nested data structures, edge case handling required. You need to think about the approach before coding. These match the difficulty of a typical 45-minute coding round at mid-tier to top-tier companies.

Senior-level and FAANG-level problems. Multi-function solutions, complex state management, performance considerations, and production-quality error handling required. These are stretch problems. If you can solve hard problems cleanly, you are well prepared for any data engineering Python round.