Dictionaries: Intermediate
Spotify's Discover Weekly algorithm uses Counter dictionaries to tally genre and artist frequencies across each user's listening history, transforming raw play events into a taste fingerprint that drives recommendations for 600 million listeners every week. A Counter turns a list of a thousand song plays into a ranked frequency map in a single line, replacing what would otherwise be a dozen lines of manual tallying. The intermediate dictionary patterns in this lesson, including Counter, defaultdict, and dict comprehensions, are the tools behind that kind of data transformation.
Iterating Over Dictionaries
Loop through keys, values, or both
Iterating Over Keys
You can also explicitly use the .keys() method, which does exactly the same thing but makes your intent clearer:
Iterating Over Values
When you only need the values and don't care about the keys, use .values(). This is useful for calculations or aggregations:
Iterating Keys and Values
Often you need both the key and the value together. The .items() method returns pairs, which you can unpack directly in the for loop. This is one of the most useful dictionary patterns:
The syntax for key, value in dict.items() is called tuple unpacking. Each item from .items() is a pair (technically a tuple), and Python automatically splits it into two variables. You can name these variables anything, but key/value or k/v are conventional.
> You need the numeric prices to calculate a total, and the item names to display a list. Choose the right iteration method for each task.
prices = { "apple": 1.50, "banana": 0.75, "orange": 2.00 } total = 0 for price in prices.(): total = total + price names = list(prices.()) print("Total:", total) print("Items:", names)
Choosing the right iteration method is about being explicit in your intent. .keys() signals "I only care about names," .values() signals "I only care about the data," and .items() signals "I need both together."
The .items() method is the most versatile and is what you will reach for most often in data transformation code. It lets you reference both the key and value by readable names rather than indexing into a tuple.
Dictionary View Objects
Use views, update, and setdefault
When you call .keys(), .values(), or .items(), Python doesn't create a list. Instead, it returns a special "view" object. Views are lightweight and always reflect the current state of the dictionary.
Notice that we didn't call .keys() again, but the view automatically includes the new "city" key. This is different from lists, where you'd need to recreate the list to see changes.
Converting Views to Lists
Sometimes you need an actual list of keys or values (for example, to sort them or access by index). Use the list() function to convert a view:
- Lightweight (low memory)
- Updates with dictionary
- Good for iteration
- Cannot index directly
- Uses more memory
- Static snapshot
- Can be indexed: list[0]
- Can be sorted/modified
The .update() Method
The .update() method adds or updates multiple key-value pairs at once. You can pass it another dictionary, and all its entries will be merged into the original:
If a key already exists, .update() overwrites it with the new value. This makes it useful for applying changes or defaults:
.update() as "merge this dictionary into mine." It's similar to adding items one by one with dict[key] = value, but more efficient for multiple items.The .setdefault() Method
The .setdefault() method is a combination of "check if key exists" and "add it if it doesn't." It returns the value for a key, but if the key doesn't exist, it first sets it to a default value:
This is particularly useful when building up dictionary data incrementally. A common pattern is using .setdefault() to initialize lists or counters:
- Returns value or default
- Never modifies the dict
- Read-only operation
- Use for safe lookups
- Returns value or default
- Adds key if missing
- Modifies the dict
- Use to initialize entries
Merging Dictionaries
Combine dicts with overrides intact
Method 1: Using .update()
As we just saw, .update() merges a dictionary into an existing one. This modifies the original dictionary:
Method 2: | Operator (3.9+)
Method 3: |= Operator
The |= operator is like .update() but uses the new merge syntax. It modifies the left dictionary in place:
- Creates new dictionary
- Originals unchanged
- Returns the merged result
- Pure functional style
- Modifies dict1 in place
- dict1 gains new items
- Returns None
- Same as .update()
> You want to apply user preferences on top of defaults, but keep the original defaults dictionary unchanged for other users. First take a snapshot, then merge.
defaults = { "theme": "light", "volume": 50 } user_prefs = { "volume": 80, "lang": "en" } combined = defaults.() combined.(user_prefs) print(combined)
Merge order matters when both dictionaries have the same key. The rightmost value always wins. Design your merge calls so that the higher-priority dictionary is on the right side of | or passed as the argument to .update().
Nested Dictionaries
Navigate multi-level data safely
Modifying Nested Values
Safe Nested Access
Accessing deeply nested data can fail if any intermediate key is missing. The safe approach is to check each level or use .get() with defaults:
.get("key", {}) returns an empty dictionary if the key is missing. This lets you continue chaining .get() calls without errors. It's a common idiom for safely navigating nested data.- Always use
.get()when a key might be missing - Chain
.get()calls with{}default for nested access - Check intermediate levels before deeply nested writes
- Consider
try/exceptfor complex nested structures - Document the expected shape of nested data
> A user dictionary has a "name" key inside "user" but no "age" key. Pick the access pattern that returns a safe default instead of crashing.
data = {"user": {"name": "Alice"}} result = print(result)
The try/except approach is an alternative to chained .get() for complex nested access. Wrap the access in a try block catching KeyError, which lets you write the direct bracket path and handle all missing-key cases in one except clause.
Dictionaries with Lists
Group multiple items under one key
Building Lists Over Time
A frequent task is grouping items into lists based on some criterion. The .setdefault() method we learned earlier is perfect for this:
Copying Dictionaries
Clone dicts without shared mutations
- Use .copy() when you need independent data
- Pass copies to functions that may modify them
- Be explicit about whether you want shared state
- Assume "new_dict = old_dict" creates a copy
- Modify a dictionary inside a function unknowingly
- Forget that both variables point to the same object
Shallow Copy with .copy()
The .copy() method creates a new dictionary with the same key-value pairs. Changes to the copy don't affect the original:
However, .copy() only creates a shallow copy. If your dictionary contains mutable values (like lists or nested dictionaries), those inner objects are NOT copied:
.copy() is sufficient.> You want to save a snapshot of the data before adding a new entry. After the snapshot, modifications to the original should not affect it.
data = {"x": 10, "y": 20} snapshot = data.() data.({"z": 30}) print(len(data)) print(len(snapshot))
Dictionary Comprehensions
Build filtered dicts in one line
The pattern is {key_expression: value_expression for item in iterable}. This is a concise way to transform data into dictionary form:
Filter with Comprehensions
Transforming Keys or Values
> You have a list of words ["hello", "world", "python"] and want to build a dictionary mapping each word to its character count. Pick the comprehension that does this correctly.
words = ["hello", "world", "python"] result = print(result)
Common Dictionary Patterns
Invert, count, and group with dicts
Inverting a Dictionary
Counting with Dictionaries
The expression dict.get(key, 0) + 1 returns the current count (or 0 if missing) and adds 1. This is a classic Python idiom for counting.
> This counting loop accesses counts[ch] directly on the first encounter, but the key does not exist yet. It needs a safe default value.
KeyError: 'h' - the key doesn't exist yet on first access
The grouping pattern collects items into lists under a shared key. Use setdefault or a defaultdict to accumulate values without checking for key existence first.
> You are a data engineer at Amplitude aggregating raw clickstream events into a nested session summary dictionary, iterating over every session's key-value pairs, merging config overrides, and using a dict comprehension to build a filtered view of high-value sessions for the weekly retention report.
.items() processes every session ID paired with its metrics dict to apply aggregation logic across the full event log.dict structures store each session's event counts, durations, and page sequences as inner dicts keyed by event type..update() merges a config-override dictionary into the base session summary without overwriting unrelated keys.dict comprehension builds the filtered high-value session view in a single expression by combining key selection and value transformation.for key in dict, .keys(), .values(), or .items().items() gives (key, value) pairs - the most versatile option.update() merges another dictionary into yours.setdefault() returns a value, creating the key first if needed| to merge dicts into a new dict (Python 3.9+)dict["a"]["b"].copy() makes a shallow copy; nested objects are shared{k: v for k, v in iterable}Navigate nested data with confidence
- Category
- Python
- Difficulty
- intermediate
- Duration
- 37 minutes
- Challenges
- 3 hands-on challenges
Topics covered: Iterating Over Dictionaries, Dictionary View Objects, Merging Dictionaries, Nested Dictionaries, Dictionaries with Lists, Copying Dictionaries, Dictionary Comprehensions, Common Dictionary Patterns
Lesson Sections
- Iterating Over Dictionaries
Iterating means looping through each item in a collection. With lists, you iterate over elements by position. With dictionaries, you have three choices: iterate over keys, values, or both. Let's explore each approach. Iterating Over Keys When you loop directly over a dictionary, you iterate over its keys. This is the default behavior and the most common pattern: Iterating Over Values Iterating Keys and Values Fill in the blanks to complete two tasks: sum up all the prices, then get a list of all
- Dictionary View Objects
Converting Views to Lists The .update() Method The .setdefault() Method
- Merging Dictionaries (concepts: pyDictMerge)
Combining two dictionaries into one is a common operation. Python provides several ways to do this, each with different trade-offs. Method 1: Using .update() Method 2: | Operator (3.9+) Modern Python (3.9 and later) introduced the | operator for merging dictionaries. This creates a NEW dictionary without modifying the originals: When both dictionaries have the same key, the value from the right-side dictionary (after the |) wins. This is intuitive: you're "applying" user preferences on top of de
- Nested Dictionaries (concepts: pyDictNested)
Dictionary values can be any Python type, including other dictionaries. This allows you to represent complex, hierarchical data. Real-world data is almost always nested. Each bracket access goes one level deeper. The expression company["address"] returns the nested dictionary, and then ["city"] accesses a key within that nested dictionary. Modifying Nested Values You can modify nested values using the same chained bracket syntax: Safe Nested Access Following a few simple rules prevents most nest
- Dictionaries with Lists
A very common pattern is using lists as dictionary values. This lets you group multiple items under a single key, such as all orders for a customer or all tags for an article: This pattern appears constantly in data engineering. API responses, configuration files, database records with multi-valued fields all use dictionaries with lists. Building Lists Over Time
- Copying Dictionaries
When you assign a dictionary to a new variable, you don't create a copy. Both variables point to the same dictionary. This is critical to understand: Shallow Copy with .copy() Fill in the blanks to take a snapshot of a dictionary, then modify the original. Think about what each operation returns. Understanding reference semantics vs copy semantics is crucial for avoiding bugs when passing dictionaries to functions. Functions receive a reference by default, so any mutation inside the function is
- Dictionary Comprehensions (concepts: pyDictComprehension)
Just like list comprehensions create lists in a single expression, dictionary comprehensions create dictionaries. The syntax uses curly braces with a key:value expression: Filter with Comprehensions You can add an if clause to filter which items are included: Transforming Keys or Values Comprehensions are powerful for transforming dictionary data: Try building a dictionary comprehension yourself. Choose the right expression to transform a list of names into a dictionary of name-length pairs. Dic
- Common Dictionary Patterns
These patterns appear constantly in real code. Once you recognize them, you will reach for them instinctively when solving similar problems. Inverting a Dictionary Swapping keys and values is useful when you need reverse lookups: Counting with Dictionaries We saw a counting example in the beginner lesson. Here's a cleaner version using .get(): This counting loop has a bug. The code tries to count letters but crashes on the first iteration. Remove the tile causing the error to fix it. Mastering d