sorted() vs .sort()

Concepts covered: Lambda, Sorting, Modules

Python ships 2 sorting entry points over the same Timsort engine, and they differ in 1 contract: what happens to the data passed in. sorted() accepts any iterable, builds a new list, sorts that new list and returns it. list.sort() exists only on lists, rearranges the list it is called on, and returns None. Interviewers probe the difference because it decides whether a function has a side effect on its caller.

sorted(iterable)
  • Accepts lists, tuples, dicts, sets, strings and generators
  • Returns a new list; the input keeps its order
  • Allocates a second list of references, 8 bytes per element
  • Fits when the caller or another function still reads the original
list.sort()
  • Defined only on list objects
  • Reorders the list in place and returns None
  • Allocates no second list
  • Fits when the list was built inside the function and nobody needs its old order

The None trap

The most frequent bug here is assigning the result of .sort(). Read the program below and decide what each of its 3 print lines produces before reading on.

scores = [42, 7, 19]
ranked = scores.sort()
print(ranked)
print(scores)
print(ranked[0])

The first print shows None, because .sort() returns nothing. The second shows [7, 19, 42]: the sort happened, on the list under its old name. The third raises TypeError: 'NoneType' object is not subscriptable. In a pipeline that error tends to surface far from its cause, inside whatever function later received ranked as an argument, which makes it slow to trace.

What key= does, and how often

Both functions take key= and reverse=. The key function maps each element to the value that gets compared. Python calls it exactly once per element, stores the results, and then compares only those stored keys, however many comparisons the sort needs. A key that costs 1 millisecond therefore adds about 17 minutes on 1,000,000 rows, and that figure stays fixed whether the input is random or nearly sorted. The run below logs every key call so you can count them.

calls = []

def by_streams(row):
    calls.append(row[0])
    return row[1]

tracks = [('Halo', 30), ('Drift', 50), ('Ember', 10)]
ranked = sorted(tracks, key=by_streams)
print(calls)
print(ranked)
print(tracks)
result = tracks.sort(key=by_streams)
print(result)
print(tracks)
print(len(calls))
['Halo', 'Drift', 'Ember']
[('Ember', 10), ('Halo', 30), ('Drift', 50)]
[('Halo', 30), ('Drift', 50), ('Ember', 10)]
None
[('Ember', 10), ('Halo', 30), ('Drift', 50)]
6

sorted() logs 3 key calls for 3 rows, returns the reordered list, and leaves tracks exactly as it was. .sort() logs 3 more calls, returns None, and reorders tracks itself. The final count of 6 confirms one call per element per sort, never one per comparison.

itemgetter instead of lambda

Pipeline rows usually arrive as dicts or tuples, and the key is usually one field. operator.itemgetter('streams') builds a callable that performs the same lookup as lambda r: r['streams'], but the lookup runs in C instead of executing a Python function body for every row. Given several field names, itemgetter('artist', 'streams') returns a tuple per row, which is the building block for multi-field ordering. For objects with attributes, operator.attrgetter('streams') plays the same role.

from operator import itemgetter

plays = [
    {'track': 'Halo', 'artist': 'Nova', 'streams': 30},
    {'track': 'Drift', 'artist': 'Kite', 'streams': 50},
    {'track': 'Ember', 'artist': 'Nova', 'streams': 10},
]

by_streams = sorted(plays, key=itemgetter('streams'), reverse=True)
print([p['track'] for p in by_streams])

get_pair = itemgetter('artist', 'streams')
print(get_pair(plays[0]))

The first line prints ['Drift', 'Halo', 'Ember'], highest count first. The second prints ('Nova', 30): a tuple built from the first row, in the order the field names were given. plays itself still starts with Halo, because sorted() never touched it.

Sorting dicts and other iterables

Iterating a dict yields its keys, so sorted(counts) returns the key names alphabetically and drops the counts. To rank by value, sort the (key, value) pairs from .items() with itemgetter(1). Dicts have no .sort() method, and neither do tuples or generators, so for them sorted() is the only choice.

from operator import itemgetter

counts = {'Nova': 12, 'Kite': 40, 'Arlo': 25}
print(sorted(counts))
print(sorted(counts.items(), key=itemgetter(1), reverse=True))
counts.sort()

This prints ['Arlo', 'Kite', 'Nova'], then [('Kite', 40), ('Arlo', 25), ('Nova', 12)], and the last line raises AttributeError: 'dict' object has no attribute 'sort'. The ranked result is a list of pairs; wrap it in dict() if a dict in that order is what the caller wants.

When comparisons fail

Sorting compares keys with <, and Python 3 refuses to compare unrelated types. A column holding a stray string or a None stops the whole sort with an error, so a bad value never produces a quietly half-sorted result. sorted([3, None, 1]) raises TypeError: '<' not supported between instances of 'NoneType' and 'int'. Missing values are routine in real feeds, so plan the key for them; the next section shows a key that places them first or last on purpose.

Do
  • Use sorted() when the input list belongs to the caller
  • Use .sort() on a list the function built itself
  • Pass itemgetter('field') for dict or tuple rows
  • Return the sorted list explicitly from helper functions
Avoid
  • Assign the result of .sort() to a variable
  • Reorder an argument in place without saying so in the name or docstring
  • Put I/O or heavy parsing inside a key function
  • Sort a dict and expect its values back

About this lesson

This section is part of the Custom Sorting: Beginner lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.