Collections: Intermediate

sorted() takes any iterable, compares its items with <, and returns a new list in ascending order. The input is left as it was, so other code that still reads it sees the original order.

Strings compare character by character using each character's code, and the uppercase letters A to Z all have lower codes than the lowercase letters a to z. A mixed-case list of table names therefore comes back in an order nobody would pick by hand. The key argument fixes that:

key takes a function. sorted() calls it once on each item, sorts by the values it returns, and hands back the original items in that order. With key=str.lower, the comparisons run on 'users' and 'accounts', yet the output still reads 'Users' and 'Accounts'. The key decides the order and never changes what is in the list.

Any function that takes 1 argument can be a key, built-ins included. With key=len, table names line up by character count. events and orders both have 6 characters, and events stays ahead because it came first in the input:

Pipeline records are usually dicts, and 2 dicts cannot be compared with <, so you write a small function that returns the field to sort on. This program prints a line each time its key function runs. Compare the order of those calls with the order of the final output.

All 3 key calls happen first, in input order, before any item moves; the runs then come out by their returned minutes. The argument is key=by_minutes, with no parentheses, which hands sorted() the function itself. Writing key=by_minutes() would call it on the spot with no run to read. The same slip with a built-in, key=len(), fails before any sorting with TypeError: len() takes exactly one argument (0 given).

Both growth runs have the key ('growth', 7). With the keys fully tied, dedupe_users stays ahead of sync_crm because it came first in the input. Python's sort is stable: items with equal keys keep their original relative order. That guarantee is what makes a sorted report come out the same on every run over the same data.

Real columns have gaps, and Python has no rule for comparing None with a number. Decide what this program prints before you check the answer.

The fix is a key that never puts None and a number into the same comparison. Return a tuple whose first item marks the value as present (0) or missing (1). Every missing value then sorts after every real one, and the second items are only compared between values of the same kind.

Ranking flips the question: which items come first when bigger is better. reverse=True makes sorted() put the largest key first, and slicing the result with [:n] keeps the top N. Here are 5 tracks with a year of play counts, 2 of them tied at 310.

Each item of plays.items() is a (name, count) pair, so by_count returns pair[1]. Paper Moon and Night Drive tie at 310, and Paper Moon stays ahead because it was inserted first: reverse=True keeps equal keys in input order. Sorting ascending and then reversing with [::-1] gives a different answer:

The ascending sort keeps Paper Moon ahead of Night Drive, and [::-1] then flips the entire list, ties included. If one report uses reverse=True and another uses [::-1], the same data shows 2 different number-one tracks. Pick reverse=True for descending order with ties in input order.

Passing the dict itself to sorted() sorts its keys and returns them alone, alphabetically. To rank the names by their counts, use the dict's get method as the key: sorted() calls plays.get on each name and orders the names by the counts it returns.

key=plays.get carries no parentheses for the same reason key=by_minutes carried none: sorted() needs the method itself, ready to call on each name. Use this form when the report lists names only, and plays.items() when each line needs its count beside the name.

reverse=True flips every part of a tuple key together. A common ranking rule wants plays descending but names ascending, so tied tracks land alphabetically. For a numeric field, negate it inside the key: the most plays becomes the smallest key, and the sort itself stays ascending.

Say runs should be listed fastest first, and runs tied on minutes should list their names from Z to A. Names cannot be negated, so the first pass sorts by name with reverse=True and the second pass sorts by minutes:

After the first pass, sync_crm sits ahead of dedupe_users. The second pass sees them tied at 7 minutes and, being stable, leaves them in that order. With any number of fields the rule holds: sort by the least important field first and the most important field last, each pass with its own reverse.

When N is 1, sorting the whole list does work you then throw away. max() and min() accept the same key argument and return the single best item. When several items share the best key, they return the first one they meet.

Small inputs behave differently across these tools. A slice past the end of a list returns what exists, so ranked[:5] for a listener who played only 3 tracks all year returns those 3 with no error. max() on an empty input raises ValueError, so check that a list has items before asking for its best one.

map(function, iterable) calls the function on each item and produces the results in the same order. Raw extracts often deliver numbers as text, and map(int, raw) converts a whole column in a single expression. Predict both printed lines before you look:

map() returns a map object, which produces each value only when something asks for it, and produces it once. list() asks for every value, so the second list() call finds nothing left and returns []. Store the result of list() in a variable whenever you need the values twice.

Creating the map object converts nothing. Step through this program and watch when each converting line prints relative to the loop.

If map() converted eagerly, all 3 converting lines would print before the first running total. Instead each text value is converted only when the loop asks for it, with the running total updated in between. That timing also decides where an error surfaces: a bad value fails on the line that reads the map, which can be far from the line that built it.

Run it and map created prints first, because building the map converted nothing. The failure comes on line 4, when list() reads the values and int('n/a') runs: the traceback names line 4 and ends with ValueError: invalid literal for int() with base 10: 'n/a'. Line 2, where map() was written, never appears in it. When a step fails inside a list() or sum() call, trace the map object back to the data it was built from.

Any function that takes 1 item and returns 1 value works with map(). For records, the function usually builds a new dict from the old one, so the source list stays intact for any other step that reads it.

A map object is iterable like a list, so one map can feed another. str.strip and str.upper are methods named through their type, and they plug in the same way str.lower did as a sort key. Here country codes arrive padded and in mixed case:

Neither map does anything until list() reads the outer one. Then each code passes through both steps before the next code starts: ' pt' becomes 'pt', then 'PT'. Keep the list that list() returns, because both map objects are spent after that one read.

Given 2 iterables, map() pairs them up by position and calls the function with 1 item from each. It stops when the shorter one runs out:

filter(function, iterable) runs the function against every item and keeps those for which it returns True. A function used this way is called a predicate: it answers a yes-or-no question about 1 item. Like a map object, the result is a filter object that computes each item as you read it and can be read only once.

The 2 dicts printed are the same objects that sit in runs. Nothing in them changed, and they come out in their original order. filter() chooses which items survive and never edits them, which gives you 2 properties to check in any output: it holds at most as many items as the input, and every item in it came from the input.

A predicate can combine conditions with and and or, as long as it ends by returning True or False. It receives exactly 1 argument, the item, so a threshold it needs has to be written inside the function or read from a constant defined at module level.

load_orders at 42 minutes and build_daily at 19 pass both tests, so slow holds 2 dicts. The 2 failed runs fall at the status test, whatever their minutes. A threshold kept in a named constant such as SLOW_MINUTES changes in one place when the rule changes.

Looping over a dict gives its keys, so a predicate on values needs .items(), where each item is a (key, value) pair. Pass the surviving pairs to dict() to rebuild the mapping. Here only tables that loaded rows stay:

Earlier, None in a latency column broke sorted(), and a key placed it last. When missing readings should not appear in the result at all, a predicate built on is not None removes them, and sorted() takes the filter object directly:

Put filter() inside map() when the order is yours to choose. The transform then runs on fewer items, and the predicate sees the full original record with every field it might test, before map() reduces each record to a name.

You have walked 2 lists side by side with zip() in a for loop. Called on its own, zip() returns a zip object that yields tuples and can be consumed once. Whatever you pass it to sets the final shape: list() gives pairs and dict() turns those pairs into a mapping.

zip() accepts any number of iterables and builds a tuple with 1 item from each, so 3 parallel columns become 3-field rows that a for loop can unpack in a single line:

dict(zip(...)) has its own silent failure. When a key repeats, each later pair overwrites the earlier value, the same as assigning to that key twice:

The first load_orders runtime of 42 is gone. Without any error, 3 job entries became 2 dict entries. If the key column can repeat, compare len(job_ids) with the length of the dict you built, as the last line does, before trusting the mapping.

Extracts often arrive as a header row plus value rows, the layout of a CSV file or a query result. dict(zip(header, row)) labels each value with its column name, and wrapping that in a function lets map() apply it to every row.

zip() stops as soon as any input is exhausted, and it raises no error about the leftovers. Step through a mismatch where the runtime list lost its last value somewhere upstream.

build_daily is never paired, and nothing in the output says so. In a pipeline this is silent data loss. When 2 lists are supposed to match, compare their lengths before zipping and stop the step if they differ:

The * in a call spreads a list into separate arguments. zip(*pairs) therefore receives each pair as its own argument and groups all the first items together, then all the second items, splitting a list of pairs back into 2 tuples. Combined with sorted(), this orders 2 parallel lists by one of them while keeping every pairing intact.

Putting minutes first in the zip() call makes each runtime the first item of its tuple, so plain sorted() orders by runtime with no key function, and the job name only matters on a tie. The results come back as tuples; pass them to list() if later code needs lists.

Ties deserve their own check because a wrong tie order raises nothing. Records built from parallel lists keep the order of those lists, so a track's position in the source decides which of 2 tied tracks takes first place under reverse=True. Flip the ties with [::-1] and the top of the report changes while every count stays the same.