Nested Dictionaries and Lists

Concepts covered: Nested Dicts

An order event from an API arrives as a dict whose values are dicts and lists, several levels deep. Each bracket evaluates to an object and the next bracket applies to that object, so event["items"][1]["sku"] reads as: the items list, its second element, that element's sku.

event = {
    "order_id": "A-1001",
    "customer": {"id": "c-17", "address": {"city": "Leeds", "country": "GB"}},
    "items": [{"sku": "SKU-4", "qty": 2}, {"sku": "SKU-9", "qty": 1}],
}
print(event["customer"]["address"]["city"])
print(event["items"][1]["sku"])

event["customer"]["address"]["postcode"] = "LS1"
event["items"].append({"sku": "SKU-2", "qty": 5})
print(len(event["items"]), event["customer"]["address"])

Writes follow the same path: every bracket except the last reads an object, and the last one writes into it. The postcode lands in the inner address dict in place, and append grows the list the items key already holds. A write through a level that does not exist fails on the read before it, so event["billing"]["city"] = "York" raises KeyError: 'billing'.

When a level is missing or null

A bracket path raises KeyError at the first missing level, and the message names only that key, not where it sat in the path. The standard guard chains .get(key, {}): a missing level returns an empty dict, and the next .get runs against that. It covers absent keys. It does not cover a key that is present with the value None, which is how JSON null arrives.

event = {"order_id": "A-1002", "customer": None}

safe = event.get("missing", {}).get("address", {}).get("city", "unknown")
print(safe)
try:
    city = event.get("customer", {}).get("address", {}).get("city", "unknown")
except AttributeError as exc:
    print(f"AttributeError: {exc}")

Output:

unknown
AttributeError: 'NoneType' object has no attribute 'get'

The default in .get("customer", {}) applies only when the key is absent. Here the key is present, so .get returns None and the next call fails. Writing (event.get("customer") or {}) treats absent and null alike, since None or {} evaluates to {}. Any other falsy value at that level, such as an empty string, also becomes {}, which is usually what code reading nested data wants.

Past 3 levels, chained guards stop being readable. A small helper walks a path and returns the default as soon as a level is not a dict:

def dig(record, *path, default=None):
    current = record
    for key in path:
        if not isinstance(current, dict):
            return default
        current = current.get(key)
    return default if current is None else current

print(dig(event, "customer", "address", "city", default="unknown"))

Complete the loop so every event prints its city or unknown, including the event whose customer is null and the one with no address.

Grouping many items under one key

A dict of lists answers questions like which orders each customer placed. The loop needs a list under the key before it can append, and setdefault(key, []) supplies it: on a customer's first order it stores a new empty list and returns it, and on every later order it returns the list already stored. .append then grows that stored list in place.

orders = [
    ("c-17", "A-1001"),
    ("c-22", "A-1002"),
    ("c-17", "A-1003"),
    ("c-31", "A-1004"),
]
by_customer = {}
for customer, order_id in orders:
    by_customer.setdefault(customer, []).append(order_id)
print(by_customer)

{'c-17': ['A-1001', 'A-1003'], 'c-22': ['A-1002'], 'c-31': ['A-1004']}

Writing by_customer[customer] = [order_id] instead replaces the list on every order, and c-17 ends up holding only A-1003. The setdefault version works because a list is mutable, so the caller changes the stored object. A running total per key holds an int, which += on the returned value cannot change inside the dict, so totals use totals[key] = totals.get(key, 0) + amount. collections.defaultdict(list) does the same grouping job from the standard library.

About this lesson

This section is part of the Dictionaries: Intermediate lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.