Deep Copies

Concepts covered: Copying Lists

A deep copy duplicates the whole structure. copy.deepcopy(x) from the standard copy module walks every list inside x, however deeply nested, and builds a new one for each, so nothing mutable is shared with the original. Numbers and strings are reused, which is safe because nobody can change them. The same module's copy.copy(x) makes a shallow copy, equivalent to x.copy() for a list.

import copy

config = [["us", "eu"], 30]
shallow = copy.copy(config)
deep = copy.deepcopy(config)
config[0].append("apac")
print("shallow:", shallow)
print("deep:", deep)
print(config[0] is shallow[0], config[0] is deep[0])

shallow: [['us', 'eu', 'apac'], 30]
deep: [['us', 'eu'], 30]
True False

Appending "apac" to the original's inner list reaches shallow, which shares that inner list, and misses deep, which got its own. The final line prints the identity checks: True for the shallow copy, False for the deep one.

Equal contents, separate objects

== compares lists item by item, descending into nested lists, so a deep copy is equal to its source. is compares identity, and a deep copy shares no list with its source at any level.

import copy

orig = [[1, 2], [3]]
deep = copy.deepcopy(orig)
print(deep == orig, deep is orig, deep[0] is orig[0])

Output:

True False False

Shared references survive the copy

If the original holds the same inner list twice, deepcopy copies it once and puts that copy in both places. During the call it records every object it has already copied and reuses the copy when it meets the object again. The result keeps the source's pattern of sharing internally while sharing nothing with the source. The same record lets it copy a list that contains itself without looping forever.

import copy

shared = ["x"]
pair = [shared, shared]
clone = copy.deepcopy(pair)

clone[0].append("y")
print(clone)
print(pair)
print(clone[0] is clone[1], clone[0] is shared)

Output:

[['x', 'y'], ['x', 'y']]
[['x'], ['x']]
True False

Objects inside lists

deepcopy also copies instances of your own classes, attribute by attribute. A template job definition can be deep-copied into a run and the run's step list changed, with the template left as it was.

import copy

class Job:
    def __init__(self, name, steps):
        self.name = name
        self.steps = steps

template = Job("nightly", ["extract", "load"])
job = copy.deepcopy(template)
job.steps.insert(1, "validate")
print(template.steps)
print(job.steps)

Output:

['extract', 'load']
['extract', 'validate', 'load']

With copy.copy(template) instead, the new Job would share template.steps, and the insert would change both.

What a deep copy costs

A deep copy duplicates every container inside the structure. The time and memory it needs grow with the total number of nested lists and objects. On a batch of 200,000 rows, each a list of plain values, it builds 200,001 new lists. Some objects can't be copied at all: a generator or an open file raises a TypeError, because a half-consumed stream has no meaningful second copy.

When you know the shape, copy only the levels you will change. Rows stored as flat lists need 2 levels, and a comprehension handles them: [row[:] for row in rows]. It copies each row without inspecting the values inside, and it runs faster than deepcopy because it skips the per-object record keeping.

A shallow copy is enough when

• The items are values you only replace, like numbers and strings
• You will change which items the list holds or their order, never edit an inner list
• Sharing the inner rows is intended because nobody edits them

Reach for a deep copy when

• You'll edit inner lists or object attributes in the copy
• The nesting depth varies or is unknown
• A template must stay unchanged across many runs

What does this program print?

import copy

grid = [[0, 0], [0, 0]]
a = grid[:]
b = [row[:] for row in grid]
c = copy.deepcopy(grid)
grid[1][1] = 7
print(a[1][1], b[1][1], c[1][1])

• 7 0 0
• 7 7 0
• 0 0 0
• 7 7 7

The copy functions live in a module, so a program needs import copy at the top and the module name in front of each call, as in copy.deepcopy(x). A bare deepcopy(x) raises NameError: name 'deepcopy' is not defined unless the program imported it by name with from copy import deepcopy.

About this lesson

This section is part of the Lists: Advanced lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.