Iterating and Dictionary Views

Concepts covered: Dict Iteration, Dict Methods

3 ways to walk a dict

A for loop over a dict yields its keys. 3 methods choose something else for each pass: .keys() yields keys, .values() yields values, and .items() yields (key, value) tuples that the loop header unpacks into 2 names. All of them walk entries in insertion order, guaranteed since Python 3.7, meaning the order in which each key was first added. For sorted order, call sorted() explicitly.

row = {"order_id": "A-1001", "customer": "c-17", "amount": 42.5, "status": "paid"}

for key in row:
    print(key, end=" ")
print()
for value in row.values():
    print(value, end=" ")
print()
for key, value in row.items():
    print(f"{key}={value}")

Output:

order_id customer amount status 
A-1001 c-17 42.5 paid 
order_id=A-1001
customer=c-17
amount=42.5
status=paid

Choose by what the loop body reads. Summing amounts needs .values(). Comparing column names needs the bare loop or .keys(), which yield the same keys in the same order, so teams pick one by convention. Anything that uses both halves of an entry needs .items(). Looping over keys and reading row[key] inside also works, but it pays a second hash lookup per entry and hides from the next reader that the body wanted pairs.

Views read the live dict

The 3 methods return view objects (dict_keys, dict_values, dict_items) that hold no copy of the data. A view reads the dict each time it is used, so creating one costs the same for 4 keys as for 4 million, and it shows keys added after the view was made.

status_counts = {"paid": 3, "refunded": 1}
statuses = status_counts.keys()
print(statuses)

status_counts["failed"] = 2
print(statuses)
print(len(statuses), "failed" in statuses)

snapshot = list(statuses)
status_counts["pending"] = 5
print(snapshot)

Output:

dict_keys(['paid', 'refunded'])
dict_keys(['paid', 'refunded', 'failed'])
3 True
['paid', 'refunded', 'failed']

list(statuses) copied the keys at that moment, so the later pending key never reached snapshot. A snapshot is what code needs before indexing (statuses[0] raises TypeError: 'dict_keys' object is not subscriptable) and before adding or removing keys while looping. Changing a dict's size during a loop over its live view raises RuntimeError, a trap the Advanced tier covers in its gotchas section.

Keys views act like sets

A keys view tests membership with in at hash speed and supports the set operators &, |, - and ^, each returning a real set. That turns a schema drift check into 2 subtractions. A values view supports neither: 42.5 in row.values() compares against every value in turn, so its cost grows with the dict.

EXPECTED = {"order_id": str, "customer": str, "amount": float, "currency": str}
row = {"order_id": "A-1001", "customer": "c-17", "amount": 42.5, "status": "paid"}

missing = sorted(EXPECTED.keys() - row.keys())
unexpected = sorted(row.keys() - EXPECTED.keys())
print(missing, unexpected)

Writing many keys with update

d.update(other) writes every pair from other into d. A key already present keeps its position and takes the new value; a new key goes on the end. The argument can be a dict, any iterable of 2-item pairs, or keyword arguments. The method changes d in place and returns None, so row = row.update(...) throws the dict away and leaves row bound to None.

row = {"order_id": "A-1001", "amount": 42.5, "status": "paid"}
row.update({"status": "refunded", "refund_id": "R-88"})
row.update([("currency", "GBP")])
row.update(source="web")
print(row)

Read or insert with setdefault

d.setdefault(key, default) returns d[key] when the key exists. When it does not, it stores default under that key first and returns the stored object. .get(key, default) returns the same value in both cases but never writes, and that write is the whole difference: after setdefault, the key is in the dict.

first_order = {}
events = [("c-17", "A-1001"), ("c-22", "A-1002"), ("c-17", "A-1003")]
for customer, order_id in events:
    first_order.setdefault(customer, order_id)
print(first_order)

Keeping the first value per key is one use. The other, returning a mutable object that the caller then fills, drives the grouping pattern in the nested-data section. The default argument is evaluated on every call, even when the key already exists, so setdefault(key, load_lookup()) calls load_lookup() once per record whether it is needed.

Put the views to work. The report below prints the largest payment, then every order id whose payment is above 10.

About this lesson

This section is part of the Dictionaries: Intermediate lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.