The Shape They All Share
A medium Python interview practice problem on DataDriven. Write the Python and run it against test cases, with instant feedback.
- Domain
- Python
- Difficulty
- medium
- Seniority
- mid
- Asked in
- Citi
Problem
A nightly job loads dozens of regional CSV exports into memory as tables, each one a list of row dicts. Over the years each region added its own extra columns, so the records no longer line up: combine every row into a single list, keeping only the fields present in every record across all tables and dropping the rest. With no records at all, the result is empty.
A pile of exports, no two quite alike. Keep only what every record agrees on.
Example 1
Input:
tables = [[{"city":"Austin","temp":71,"region":"TX"},{"city":"Dallas","temp":78,"region":"TX"}],[{"city":"Reno","temp":65,"region":"NV","humidity":20}]]Output:
[{"city":"Austin","temp":71,"region":"TX"},{"city":"Dallas","temp":78,"region":"TX"},{"city":"Reno","temp":65,"region":"NV"}]Example 2
Input:
tables = [[{"qty":5,"sku":"A1","warehouse":"east"}],[{"qty":3,"sku":"B2","warehouse":"west"},{"qty":9,"sku":"C3"}]]Output:
[{"qty":5,"sku":"A1"},{"qty":3,"sku":"B2"},{"qty":9,"sku":"C3"}]Worked solution and explanation
Why this problem exists in real interviews
This is a schema-intersection projection wearing a 'merge a pile of CSVs' costume. The phrase that decides the whole problem is 'common data': you are not concatenating tables, you are concatenating them AFTER pinning every row to the set of fields that all rows share. Almost everyone flattens the rows correctly. The thing that separates people is realizing you cannot decide which columns survive until you have seen every record, so the answer needs one pass to intersect the field sets and a second pass to project. Skip the intersection and you ship ragged rows where one region's bonus column rides along on half the records and is missing from the other half.
Break down the requirements
Step 1: Flatten first, but do not emit yet
Walk every table and every row into one flat list. This is the easy half. The mistake is returning here: at this point you still know nothing about which columns are universal, so anything you emit now is ragged by construction.
Step 2: Intersect the field sets across all rows
Seed a 'common' set with the keys of the first row, then intersect it with each remaining row's keys. A single short row, one record missing a column, drags that column out of the result for everyone. That is the whole point of 'common'.
Step 3: Project every row to the common set
Now make a second pass and rebuild each row keeping only keys in 'common'. Guard the empty case up front: if there are no rows at all, the intersection is undefined, so return an empty list before you ever touch rows[0].
The solution
def merge_common(tables):
rows = [row for table in tables for row in table]
if not rows:
return []
common = set(rows[0])
for row in rows[1:]:
common &= row.keys()
return [{k: v for k, v in row.items() if k in common} for row in rows]- Naive: flatten and return
- out = []; for t in tables: out.extend(t); return out. Reads clean, passes a test where every file happens to share columns, and silently fails the moment one region added a field. The extra column rides along on the rows that have it.
- Correct: intersect then project
- Compute the shared field set across every row, then keep only those keys on each row. Every output row has the identical schema, which is exactly the contract 'merge the common data' was asking for.
Common follow-up questions
- What if instead of dropping non-common columns you must keep the union, filling missing fields with None? (Flips intersection to union and forces a per-row backfill; tests whether the candidate sees the two requirements as the same skeleton with a different reducer.)
- The files no longer fit in memory. How would you compute the common columns across hundreds of files without loading them all at once? (Tests a streaming two-pass design: one pass over headers to intersect, a second to project; the natural bridge to Spark or a chunked pandas read.)
- Two files describe the same entity with the same key but conflicting values. How do you resolve duplicates instead of just concatenating? (Moves from concat to a keyed merge with a conflict policy (last-wins, newest-timestamp), which is the real shape of most ETL consolidation jobs.)