Choosing the Right Type

Concepts covered: Data Types

Choosing a collection comes down to 4 questions. Order and change were covered in the earlier sections, as were duplicates, and the question about lookup by label is what a dict's keys are for. Ask them in the order below and the answer usually falls out by the second or third.

Does position matter, or will you ask for the first or the nth item? Then a list or a tuple.
Will the contents change after it is built? List, dict or set. Fixed after creation? Tuple.
Do you look things up by a label such as a name or an id? Dict.
Do you only need to know whether a value has been seen, with no repeats? Set.

The questions narrow the field one at a time. Take a version number such as 3.9.7, stored as 3 integers. Position matters, since 3 means the major version only in slot 0, so question 1 leaves a list or a tuple. The parts of a released version never change, so question 2 leaves the tuple. Now take the error count for each pipeline stage. No stage is first or third in any meaningful sense, so question 1 rules out list and tuple. The counts rise as errors arrive, so question 2 allows a dict or a set, and question 3 settles it: you read a count by the stage's name, which makes it a dict. The table pairs more needs with the collection that fits.

Need	Pick	Because
Steps of a pipeline that run in sequence and may grow	list	order matters and steps get added
Latitude and longitude of one warehouse	tuple	2 fixed values, usable as a dict key
Stock level per product name	dict	lookup by label
User ids already emailed today	set	membership test, no repeats
The 30 tracks of a playlist in play order	list	order matters and tracks move
Column names a header row must match	tuple	a fixed fact nothing should edit

Reach for a list when

Arrival order carries meaning
Repeated values are real data
You read items by slot number
Items are appended at the end

Reach for a set when

The only question is whether a value is present
Repeated values should vanish
Order is irrelevant
The items are numbers or strings (tuples work too)

Putting 3 types together

Real tasks mix types, and the mix follows the same questions. The animation walks a list of click events, uses a set to remember which users have already appeared, and builds a list of each user's first event. The list preserves arrival order, the set answers not in without scanning, and the dict inside each event holds that event's fields by name.

events = [
    {"user": 1, "action": "click"},
    {"user": 2, "action": "view"},
    {"user": 1, "action": "buy"},
    {"user": 3, "action": "click"},
    {"user": 2, "action": "click"},
]

seen_users = set()
first_events = []

for event in events:
    user = event["user"]
    if user not in seen_users:
        seen_users.add(user)
        first_events.append(event)

print("events:", len(events))
print("users:", len(seen_users))
print("first:", first_events)

events: 5
users: 3
first: [{'user': 1, 'action': 'click'}, {'user': 2, 'action': 'view'}, {'user': 3, 'action': 'click'}]

Swap the set for a list and the program still works but scans every earlier user on each event. Swap the first-events list for a set and the arrival order is gone.

Do

Run through the 4 questions before writing the first bracket.
Use a tuple for a record whose shape never changes.
Keep a set beside a list when you need both arrival order and a fast membership test.
Count repeats with a dict: the item is the key and the count is the value.

Avoid

Rely on the order a set prints in.
Index a dict with a position number.
Scan a list with in when a set would answer the same question without a scan.
Repeat a key in a dict literal and expect both values to survive.

Capstones

These mix every idea so far: order and change, plus conversion and duplicates. Trace the code by hand first; the explanations assume you did.

What does this program print?

batches = ["b1", "b2", "b1", "b3"]
done = set()
order = []
for batch in batches:
    if batch not in done:
        done.add(batch)
        order.append(batch)
print(order, len(done))

['b1', 'b2', 'b1', 'b3'] 3
['b1', 'b2', 'b3'] 3
['b1', 'b2', 'b3'] 4
['b3', 'b2', 'b1'] 3

That capstone kept a set beside a list so each answered its own question. The next one hands you a need and no code. Run the 4 questions on it in order, and when 2 types both satisfy every answer, prefer the one that refuses what the program never does: a type that blocks an unwanted edit turns a silent mistake into an error on the line that made it.

A program stores the 7 day names and later reads the name for day number 3 with names[3]. The program never edits the day names after they are stored. Which collection fits best?

A list of the 7 names
A dict keyed by day number
A set of the 7 names
A tuple of the 7 names

The third capstone builds 2 collections from the same list of readings, and each answers a different count. len of a set counts distinct values, while len of a tuple counts every reading, repeats included. max returns the same highest value from either one, because dropping repeats never removes the largest value, only extra copies of it.

The last capstone depends on what square brackets mean on a dict built from pairs. dict() makes the first item of each pair a key, so a dict built from (name, quantity) pairs is read with a name. A position number between the brackets asks for a key that is that number, and when no such key exists the lookup stops with a KeyError naming the missing key.

About this lesson

This section is part of the Data Structures: Beginner lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.