Lists: Advanced

A list comprehension builds a new list from an existing iterable in a single expression: [expression for item in iterable]. Python runs the loop, evaluates the expression once per item, and collects each result into a fresh list, in source order. Code that used to start with an empty list and fill it in a loop with append becomes a line that states what the list contains.

Both lists are equal, and amounts_cents is untouched: the comprehension only reads its source. The table names the parts you will find in every comprehension.

An if clause after the for decides which items reach the expression. Items that fail it are skipped, so the result can be shorter than the source. This payments feed mixes charges with refunds, stored as negative cents, and zero-value test events. The report wants charges only, in dollars.

To keep every item but compute it differently, put a conditional expression in the expression slot: a if condition else b. It's the conditional assignment you already write on a single line, placed before the for. Every source item produces an output, so the length never changes.

Both can appear together: ["big" if c > 10000 else "small" for c in xs if c > 0] drops non-positive amounts, then labels the rest.

A comprehension can hold more than one for. The clauses run in the order you would nest the loops, outer loop first, so reading them left to right as indented loops tells you what comes out. A common use turns a list of batches into a list of rows.

Reverse the clauses to [row for row in batch for batch in batches] and Python evaluates batch before any clause defines it: a NameError, or a silent wrong answer if an earlier batch variable is still around. A comprehension can also sit in the expression slot of another. [[r * c for c in range(3)] for r in range(2)] gives [[0, 0, 0], [0, 1, 2]], and every pass of the outer loop builds a separate inner list, a detail the next section depends on.

A for loop leaves its variable bound after it ends. A comprehension runs in its own scope, so its loop variable vanishes with it and never overwrites a name outside.

A comprehension exists to produce a list. [print(r) for r in rows] prints each row and also builds a throwaway list of None values, because print returns None. Write a for loop when the point is a side effect, or when the logic needs several conditions and transforms that would crowd a single line. When you only need to scan the values once, parentheses instead of brackets give a generator expression, as in any(c < 0 for c in amounts_cents), which stops at the first refund and builds no list.

When you write a comprehension that both filters and converts, fill its slots by asking 2 questions in order: which items should survive, and what each survivor should become. The first answer is the condition after the for, and the second is the expression before it. Only survivors reach the expression, so a division placed there never runs on an item the filter dropped.

A variable holds a reference to a list object stored elsewhere in memory. The statement b = a copies the reference, so afterwards both names reach the same object, and a change made through either name is visible through the other. You already know the tool that reveals this: is compares identity and returns True when 2 names point at a single object.

The append and the item assignment both went through alias, and original shows both, because there was only ever a single list. Python never copies a list unless you ask it to.

Calling a method changes the object, and so does assigning to an item or running del on one, so every name for it sees the change. A plain name = ... assignment looks similar but only points that single name at a different object, and it leaves the old object and its other names alone.

clean is raw prints True: the helper returned the very list it was given. Whether that's a bug depends on whether the caller still needed the raw rows.

a.copy() gives you a new list. a[:] and list(a) do the same: each creates a new outer list and fills it with the references the original holds. For a flat list of numbers or strings that is all the independence you need, because those items can only be replaced, never changed in place. Replacing a slot in the copy leaves the original's slot alone. Items that are themselves lists get shared between the copy and the original.

backup[1] = 60 replaced a slot in the new outer list, so config[1] is still 30. backup[0].append("apac") followed the reference in slot 0 into the inner list both outer lists point at, so config shows "apac" too. The last 2 lines confirm it: separate outer lists, a shared inner list.

Multiplying a list repeats its references. [0] * 3 repeats a reference to 0 3 times, which is harmless because a slot holding 0 can only be replaced. [[0] * 3] * 3 repeats a reference to the same inner list 3 times, so a change to any row shows in every row. A comprehension evaluates [0] * 3 anew for each row, so each row is its own list.

Python evaluates a default value once, when def runs. Every call that omits the argument receives that same object, so a list default collects items across calls.

The fix defaults to None and builds the list inside the function, so each call that omits tags gets a new empty list.

The None default fixes the shared list for a concrete reason. Every call that omits the argument still receives the same default object, but None is never mutated. The [] sits inside the function body, and the body runs on every call, so each call that reaches tags = [] builds a separate list.

A deep copy duplicates the whole structure. copy.deepcopy(x) from the standard copy module walks every list inside x, however deeply nested, and builds a new one for each, so nothing mutable is shared with the original. Numbers and strings are reused, which is safe because nobody can change them. The same module's copy.copy(x) makes a shallow copy, equivalent to x.copy() for a list.

Appending "apac" to the original's inner list reaches shallow, which shares that inner list, and misses deep, which got its own. The final line prints the identity checks: True for the shallow copy, False for the deep one.

== compares lists item by item, descending into nested lists, so a deep copy is equal to its source. is compares identity, and a deep copy shares no list with its source at any level.

If the original holds the same inner list twice, deepcopy copies it once and puts that copy in both places. During the call it records every object it has already copied and reuses the copy when it meets the object again. The result keeps the source's pattern of sharing internally while sharing nothing with the source. The same record lets it copy a list that contains itself without looping forever.

deepcopy also copies instances of your own classes, attribute by attribute. A template job definition can be deep-copied into a run and the run's step list changed, with the template left as it was.

With copy.copy(template) instead, the new Job would share template.steps, and the insert would change both.

A deep copy duplicates every container inside the structure. The time and memory it needs grow with the total number of nested lists and objects. On a batch of 200,000 rows, each a list of plain values, it builds 200,001 new lists. Some objects can't be copied at all: a generator or an open file raises a TypeError, because a half-consumed stream has no meaningful second copy.

When you know the shape, copy only the levels you will change. Rows stored as flat lists need 2 levels, and a comprehension handles them: [row[:] for row in rows]. It copies each row without inspecting the values inside, and it runs faster than deepcopy because it skips the per-object record keeping.

The copy functions live in a module, so a program needs import copy at the top and the module name in front of each call, as in copy.deepcopy(x). A bare deepcopy(x) raises NameError: name 'deepcopy' is not defined unless the program imported it by name with from copy import deepcopy.

A stack hands back items in the reverse of the order they arrived: the last item in is the first item out. A Python list does this with 2 methods that both work at the right-hand end. append(x) pushes x onto the top, and pop() with no argument removes the top item and returns it. Neither method moves the other items, so a push or a pop stays fast however tall the stack gets.

An empty list is falsy, so while stack: runs until the stack is drained and if stack: guards a pop. Without the guard, popping an empty stack raises an IndexError.

Checking that brackets nest correctly in a config string or a SQL expression is a stack problem. Push each opening bracket. When a closing bracket arrives, the top of the stack must be its partner, and anything else means the brackets cross. At the end, an empty stack means every opener was closed. The PAIRS lookup maps each closer to its opener, the same dict-based dispatch you have written before.

On "([)]", the function pushes ( and then [. When ) arrives, [ is on top, so the function returns False at the third character without reading the last. The not stack test in the same condition catches a closer with nothing open, like "())". It short-circuits before pop() could raise.

A queue is first-in first-out, and pop(0) gives that order from a list, but removing slot 0 shifts every remaining item one slot to the left, so each call does work proportional to the list's length. pop() at the end shifts nothing. Python has a structure built for removing from the front, covered in a later lesson; until then, keep lists for stacks.

Python list methods follow a convention: a method that changes the list in place returns None. That None signals that the work happened to the object you called it on. Functions and operators that leave the original alone hand back something new instead. Knowing which group a call belongs to tells you both what happens to your list and what you get back.

numbers.sort() reordered numbers and returned None, which landed in result. To keep the original order and get a sorted version, call the built-in sorted(), which reads any iterable and returns a new list.

pop() is the in-place method with a useful return value, the item it removed, and stacks rely on it. reversed(lst) returns an iterator that reads the list backwards on demand; wrap it in list() to get a new list.

Writing rows = rows.append(x) stores the None that append returned, and the name no longer reaches the list. The next method call on rows fails, often several lines after the line with the mistake.

For lists, a += b extends a in place, the same as a.extend(b), and every alias sees the new items. a = a + b builds a new list and points a at it, so aliases keep the old list. The augmented assignment you learned on numbers behaves differently here because a list can change in place and an integer cannot.

The in-place rule bites hardest when you sort nested data. row.sort() reorders whichever list row names, so if a working copy shares its inner lists with the source, sorting the copy's rows reorders the source's rows too. The copy has to reach the inner lists, the 2-level pattern from the deep copy section, before any in-place call runs.