Sets: Intermediate
Dropbox syncs billions of files across hundreds of millions of devices, and at the core of deciding what needs to be uploaded is a set operation: comparing the set of file hashes already stored in the cloud against the set of file hashes on your local machine. The difference of those two sets tells Dropbox exactly which files are new and need uploading, without scanning file contents or re-transferring anything that is already synced. Set comprehensions, frozensets, and the symmetric difference operator you will learn in this lesson are the same algebraic tools that content deduplication systems use at petabyte scale to move only the data that actually changed.
Union: Combining Sets
Merge sets with union and pipe
The Pipe Operator |
Python provides the | operator as a shorthand for union. This pipe symbol is often preferred for its concise syntax and resemblance to mathematical notation. In many programming contexts, the pipe symbol represents "or", which aligns with the inclusive nature of union: an element is in the result if it is in set A | (or) in set B.
- Method syntax with parentheses
- Works with any iterable (list, tuple)
- a.union([1, 2, 3]) works directly
- More flexible for mixed types
- Operator syntax, more concise
- Requires sets on both sides
- a | [1, 2, 3] raises TypeError
- Must convert to set first
The method form is more flexible because it accepts any iterable as an argument. If you have a list, tuple, or generator, you can pass it directly to the .union() method without first converting it to a set. The operator form requires both operands to be sets, so you must explicitly convert other types before using the | operator.
Chaining Multiple Unions
Merging Permissions
Union with Empty Sets
> Two teams have overlapping members: admins are {"alice", "bob"} and editors are {"bob", "charlie"}. Pick a set operation to produce the combined staff list.
admins = {{"alice", "bob"}} editors = {{"bob", "charlie"}} all_staff = admins.(editors) print(all_staff)
Intersection: Finding Common Elements
Find shared elements with intersection
The Ampersand Operator &
Python provides the & operator as a shorthand for intersection. The ampersand symbol is borrowed from the logical AND operation, which is fitting because intersection returns elements that are in A AND in B. Just like the and keyword requires both conditions to be true, the intersection requires an element to be in both sets.
As with union, the .intersection() method form accepts any iterable while the & operator requires both sides to be sets. Choose the method when working with lists or other iterables, and the operator for concise code when both operands are already sets.
Overlap Across Sets
Finding Common Customers
Intersection: Empty Sets
Difference: Elements Unique to One Set
Isolate unique elements per set
Order: A - B vs B - A
- Elements unique to A
- What A has that B lacks
- What to add to B to include A
- Order: first minus second
- Elements unique to B
- What B has that A lacks
- What to add to A to include B
- Order: first minus second
Chaining Set Differences
New vs Churned Users
Symmetric Difference: In One Set but Not Both
Use symmetric difference to find elements in exactly one of two sets and choose between the ^ operator and the method form.
Symmetric Difference
The Caret Operator ^
Python uses ^ (caret) for symmetric difference. This operator is borrowed from the bitwise XOR (exclusive or) operation. In boolean logic, XOR returns true when exactly one of two inputs is true, but not when both are true. This perfectly matches symmetric difference: an element is included when it is in exactly one set, but not when it is in both.
- Elements in BOTH sets
- Operator: &
- Like logical AND
- Finds shared elements
- Elements in EITHER but not BOTH
- Operator: ^
- Like logical XOR
- Finds unique elements
The relationship between intersection and symmetric difference is complementary. Together they partition the union: every element in (A | B) is in either (A & B) or (A ^ B), but never both. If you know intersection and union, you can compute symmetric difference, and vice versa.
Detecting Changes Example
Symmetric Diff: Commutative
Unlike regular difference, symmetric difference is commutative: A ^ B always equals B ^ A. This makes sense because we are finding elements unique to either side, which is the same regardless of which set we consider "first".
> Two sets a = {1, 2, 3, 4} and b = {3, 4, 5, 6} overlap on some elements. Pick a set operator to see how union, intersection, difference, and symmetric difference each produce a different result.
a = {1, 2, 3, 4} b = {3, 4, 5, 6} print(a b)
Operation Summary
Methods vs Operators
- Use the operator (|, &, -, ^) when both sides are already sets
- Use the method (.union(), .intersection()) when one side is a list or tuple
- Methods accept multiple arguments: a.union(b, c, d) works in one call
- Operators chain naturally: a | b | c reads like mathematical notation
- Methods are more explicit; operators are more concise
> This code computes last_month - this_month, which finds churned users instead of new users. The set difference operands are reversed.
Logic error: shows churned users {'alice'} instead of new users {'charlie', 'diana'}
Set difference is directional: A - B and B - A produce different results. Always read it as "what is in the first set that is NOT in the second set." Getting the operand order right is the most common source of set difference bugs.
The |, &, -, and ^ operators map directly to union, intersection, difference, and symmetric difference. Using these single-character operators makes set algebra in code read closely to the mathematical notation you would write on paper.
Subset and Superset
Validate containment and modify in place
The Comparison Operators
Python provides comparison operators for subset and superset checks: <= for subset (less than or equal to) and >= for superset (greater than or equal to). The intuition is that a "smaller" set is one contained within a "larger" set.
A proper subset or superset means the sets are not equal. Set a is a subset of c (since they are equal), but not a proper subset. The strict operators (< and >) exclude the case where sets are equal, while the non-strict operators (<= and >=) include equality.
Validation with Subsets
Checking for Disjoint Sets
Two sets are disjoint if they have no elements in common. The .isdisjoint() method returns True if the sets share no elements. This is equivalent to checking if the intersection is empty, but .isdisjoint() is more efficient because it can stop early as soon as it finds any common element.
Odd and even numbers are disjoint by definition. Odd numbers and primes share 3, 5, and 7. Primes and composites are disjoint because no number can be both prime and composite. The isdisjoint() method efficiently tells you whether any overlap exists.
- .issubset() or <= -- every element of A is also in B
- .issuperset() or >= -- A contains every element of B
- < proper subset -- A is inside B and they are not equal
- > proper superset -- A contains B and has extra elements
- .isdisjoint() -- A and B share zero common elements
In-Place Operations
Accumulating Data: Update
The .update() method (or |= operator) is particularly useful for accumulating data from multiple sources into a single set. This is common when processing files, API responses, or database queries where data arrives in batches.
Intersection Update Filter
Intersection update (&=) keeps only elements that are in both sets. This is useful for progressively narrowing down a set based on multiple criteria.
In-Place vs Regular Ops
Understanding the difference between in-place and regular operations is crucial. Regular operations leave originals unchanged and return a new set. In-place operations modify the original and return None.
After the union() call, original is still {1, 2, 3}. After the update() call, original has been modified to {1, 2, 3, 4, 5}. Note that update() returns None, not the modified set, so you cannot chain it like new = original.update(addition).
> This code uses the ^= augmented assignment operator inside an expression, which is a syntax error. The regular ^ operator should be used instead.
SyntaxError: invalid syntax with ^= in expression
In-place set operators (|=, &=, -=, ^=) modify the set they are called on. They cannot be used in the middle of a larger expression or on the right-hand side of an assignment, because they return None rather than a new set value.
Regular set operators (|, &, -, ^) always return a new set and leave both operands unchanged. Use them whenever you need the result as a value or want to preserve the originals for further comparisons.
> You are a data engineer at Spotify comparing listener sets across three regional platforms to find shared audiences for cross-promotion, identify platform-exclusive subscribers, and efficiently update running audience sets in place as new subscriber data streams in.
Combining and comparing collections
- Category
- Python
- Difficulty
- intermediate
- Duration
- 44 minutes
- Challenges
- 3 hands-on challenges
Topics covered: Union: Combining Sets, Intersection: Finding Common Elements, Difference: Elements Unique to One Set, Symmetric Difference: In One Set but Not Both, Subset and Superset
Lesson Sections
- Union: Combining Sets (concepts: pySetOperations)
A union combines all elements from two or more sets into a single set. If an element appears in any of the input sets, it appears in the union exactly once. The union operation automatically handles duplicates because the result is still a set, which by definition contains only unique elements. This makes union perfect for merging data from multiple sources. The mathematical notation for union is A ∪ B, read as "A union B". The union of sets A and B contains every element that is in A, in B, or
- Intersection: Finding Common Elements (concepts: pySetOperations)
An intersection finds elements that exist in all specified sets. If an element is in set A AND in set B, it appears in the intersection. Elements that are in only one set are excluded. The intersection operation answers the question "what do these sets have in common?" This is fundamental for finding overlaps, shared characteristics, or common attributes. The mathematical notation for intersection is A ∩ B, read as "A intersect B". The intersection of sets A and B contains only elements that are
- Difference: Elements Unique to One Set (concepts: pySetOperations)
The difference of two sets returns elements that are in the first set but not in the second. This operation answers the question "what is in A that is not in B?" Unlike union and intersection, difference is not symmetric: A - B gives different results than B - A. The order matters because you are asking a directional question. Think of difference as starting with all elements of the first set, then removing any element that also appears in the second set. What remains are elements unique to the
- Symmetric Difference: In One Set but Not Both (concepts: pySetOperations)
Symmetric Difference The symmetric difference contains elements that are in either set but NOT in both. Think of it as the opposite of intersection: instead of finding what sets share, you find what makes each set unique. If an element appears in both sets, it is excluded from the symmetric difference. Mathematically, symmetric difference is equivalent to two other expressions: it equals (A - B) union (B - A), which is the elements unique to A combined with elements unique to B. It also equals (
- Subset and Superset (concepts: pySetOperations)
Beyond combining sets, you often need to check if one set is contained within another. These containment relationships are called subset and superset. A subset is a set where every element exists in another larger set. A superset is the opposite: it contains all elements of a smaller set plus possibly more. Subset and superset checks are fundamental for validation, permission checking, and hierarchical data. For example, checking if a user has required permissions (user permissions should be a s