Sets: Intermediate

Python offers union in 2 spellings. web | app reads as "in web or in app" and builds a new set. web.union(app) is the method spelling and builds an equal set. Neither form changes web or app: each keeps the IDs it started with, which matters when a later step still needs the per-feed sets, for a per-channel breakdown say.

Python prints a set in its internal storage order, and for strings that order can change between runs. The examples pass each set to sorted(), which returns a list of the items in ascending order, so every printout comes out the same.

The third printed line holds the lesson about counts. The feeds hold 4 and 3 IDs, 7 in total, yet the union holds 5, because 103 and 104 appear once each. Adding the sizes of 2 sets overcounts whenever they overlap, and summing per-feed counts is a common way for a dashboard to report more customers than exist. The size of that overcount is the overlap, which the next section computes directly. The last line shows web still holding its original 4 IDs.

The operator insists on a set on both sides: web | [106] raises TypeError. The method takes any iterable, meaning a list, a string, a range or a dict (whose keys it uses), and converts the input as it reads it. It also takes several arguments, merging all of them into a single new set.

The list carried 103 twice. The result kept a single 103, which web already held, and added 106 and 107. The second call merged a list, a range and another set at once, and 101, already in web, appeared a single time.

Daily partitions often arrive as a list holding a set per day. Start from an empty set and fold each day in with |. The empty set contributes nothing, so after the first step seen holds the first day's IDs, and every later step adds whatever that day brings that is new.

To add a word as 1 tag, hand union() a container holding it: the list ["sql"] yields the whole string, and so does tags | {"sql"}. Each argument is read on its own, so 1 call can mix a correct input with a careless one, as the third line shows: "hourly" arrives whole and "sql" splits into letters.

Each argument to union() is looped over on its own, so a list contributes whole IDs while a bare string contributes characters. A repeated ID in any argument lands in the result once, and the calling set keeps its contents.

Intersection keeps what the sets have in common: an element survives only if every operand contains it. Back in the Blend, the artists Maya and Theo both play form the intersection of their 2 artist sets. Python writes it maya & theo, read as "in maya and in theo", or maya.intersection(theo).

Swapping the operands changed nothing, since a & b and b & a hold the same elements. The last 2 lines settle the overcount from the union section: 4 artists plus 3 artists minus the 2 they share equals the 5 in the union. For 2 sources, the gap between the summed counts and the distinct count is exactly the size of their overlap. With 3 or more sources, an ID present in all 3 is counted 3 times, so the gap is no longer the size of any single overlap.

An intersection with no common elements is still a set, an empty one, and Python prints it as set(). The braces {} already mean an empty dict, so an empty set needs a name of its own. Comparing with overlap == set() or testing len(overlap) == 0 reads that result correctly.

A retention report asks which users were active on all 3 days of a period. Both forms accept more than 2 sets: mon & tue & wed, or mon.intersection(tue, wed). When the days arrive as a list, a loop does the same job, and its starting value decides whether the answer is right. Step through it as the set narrows day by day.

The loop starts from days[0] and each & can only remove IDs, so the set shrinks from 4 to 3 to 2. Starting from set() instead, the habit that serves union well, empties the result on the first step: an empty set shares nothing with anything. Union and intersection want opposite starting values.

Of the 4 incoming entries, 2 distinct IDs were already loaded: 5001 and 5003, with the repeated 5003 counted once. A load step that skips those keys avoids writing duplicate rows, and 5005, absent from loaded, never appears in the result.

The repair changes 1 line: start from the first set in the list. A period of a single day still works. Its slice [1:] is empty, the loop body never runs, and the answer is that day's set, which is correct: everyone active that day was active on every day of the period.

The pattern to carry forward: start from the first set, loop over the rest with &, and expect the result to shrink or hold steady. Its length is the number a retention report shows.

Difference answers a one-sided question: which elements of the left set are absent from the right set. today - yesterday keeps every ID active today that was not active yesterday. Anything in yesterday that today lacks is ignored, because the result only ever holds elements of the left operand. Flip the operands and the question flips with them.

The same 2 sets answer 2 different business questions. today - yesterday gives the new users a growth report counts, and yesterday - today gives the users who went quiet, the input to a churn alert. The False on the last line is the reason to name each result after its direction, as new_users and gone_users do.

a.difference(b, c) removes everything found in b or in c, and it takes a list wherever the operator would demand a set. The operator form chains: a - b - c runs left to right, so Python computes (a - b) - c. Both drop the same elements from a.

The operator line needed set(bots) because - refuses a list. Subtracting an ID the left set lacks is harmless: {1, 2} - {99} returns a set holding 1 and 2. That makes difference the safer tool for a cleanup list that may name IDs already gone, where calling remove(99) would raise KeyError. And signups printed in full on the last line: the method built a new set and left the original alone.

A reversed difference raises no error. The result holds only elements of the left operand, so when the left side is a short group of exceptions that all sit in the right side, the answer is an empty set on every run, printed as a clean [] and a count of 0.

Symmetric difference keeps the elements that sit in exactly 1 of 2 sets and drops the ones they share. Reconciliation is where it earns its keep: an orders service and a billing service each hold a set of order IDs for the day, and every ID present in only 1 of them is a record someone has to chase. Python writes it orders ^ billing.

The 2 True lines show what ^ is made of: the 2 one-sided differences joined together, or equally, the union with the intersection taken out. Operand order does not matter here, as the last line confirms with billing on the left.

The result says 7001, 7004 and 7005 disagree, and nothing about which system holds each. If the fix depends on the side, such as re-sending an order to billing versus voiding a charge, compute orders - billing and billing - orders separately. Use ^ when the question is "do these match, and which IDs differ": an empty result means the 2 sets hold exactly the same elements.

Applying ^ across 3 sets looks as if it should give the elements found in exactly 1 of them. Follow the ID 2, which sits in all 3 sets, through each step.

The first ^ drops 2 because a and b share it. The second ^ compares that result with c, which holds 2 while the result does not, so 2 comes back. Each ^ flips an element's presence, so a chain keeps the elements that appear in an odd number of the sets: 1 set, 3 sets, 5 sets all qualify. For elements in exactly 1 set, take away everything that appears in 2 or more.

The method compared a set with the list of column names that arrived in the data, and the result named both sides of the drift: a column the contract expects and a column nobody asked for. The False is the alert condition. For 2 sets in a short expression ^ is the compact choice; when the input is a list, or the line should explain itself to whoever reviews it, use the method.

To test whether 2 collections match, compare 2 sets. A set never equals a list, so shelf == scanned below is False even though both name the same SKUs. The symmetric difference reads the list as a set, so the repeated S1 collapses and the result is empty.

Every operation so far built a new set. The last group asks yes-or-no questions about containment, and then changes a set where it stands. a <= b, or a.issubset(b), is True when every element of a is also in b. Read from the other side, b >= a, or b.issuperset(a), says b contains all of a. A data contract check fits this exactly: are all required fields present in a record?

set(record) collects the dict's keys. The record carries an extra coupon field and the check still passes: a subset test ignores whatever the larger set holds beyond the smaller one. When the check fails, required - fields names the missing fields for the error message, which tells the on-call engineer far more than a bare False.

The < operator adds a condition to <=: the sets must also differ, so the larger set holds at least 1 extra element. Every set is a subset of itself and never a proper subset of itself. The empty set is a subset of every set, since it has no element that could be missing.

With numbers, if x <= y is False then x >= y is True. Sets drop that rule. {1, 2} and {3} each hold something the other lacks, so <=, >=, <, > and == all come back False between them. The isdisjoint() method asks a separate question: do the 2 sets share nothing at all?

Each operation in this lesson has an in-place form that rewrites the left set and builds nothing new. The augmented assignment operators work on sets, and each has a method twin. The methods take lists and other iterables, the operators still want a set on the right, and every method returns None.

seen = seen | {3} builds a new set and points seen at it, so report still names the old object holding 1 and 2. seen |= {3} adds 3 to the object seen already names, and report names that same object, so it sees the change and is returns True. When 2 names share a set, choose the form deliberately: an in-place change reaches every name.

update() changed seen and handed back None, as every in-place method does. Writing seen = seen.update(batch) therefore replaces the set with None, and the next method call on seen fails with AttributeError.

When a loop collects batches, call update() as a statement on its own line, or use |= with the batch wrapped in set(). Either way loaded stays the same object and grows with each batch.

Learn to spot the broken version: assigning the result of update() back to the name leaves None after the first batch, and the second pass raises AttributeError: 'NoneType' object has no attribute 'update'.

Containment tests compose with the other operators, since a | b is a set like any other, and issuperset() accepts a list. Comparisons bind looser than | and &, so the second line compares flagged with the whole union. The third line asks for IDs present on both days, and fails.