# The Word Map

> Language, quantified.

Canonical URL: <https://datadriven.io/problems/the_word_map>

Domain: Python · Difficulty: easy · Seniority: L4 · Asked in: phData

## Problem

We're building a keyword index over a batch of user-submitted `text`, and the first pass just needs raw word counts. Return a dict mapping each word to how many times it appears, comparing words case-insensitively and ignoring any punctuation stuck to the outer edges of a word.

## Example 1

Input:

```
text = "the cat and the dog"
```

Output:

```
{"the":2,"cat":1,"and":1,"dog":1}
```

## Example 2

Input:

```
text = "Hello, hello! HELLO."
```

Output:

```
{"hello":3}
```

## Worked solution and explanation

### What this really tests

Underneath the keyword-index framing, this is a normalize-then-tally: can you fold every surface form of a word ('The', 'the', 'the,') onto one canonical key before you count? Anyone can split on whitespace and bump a dict. The catch is doing the normalization FIRST: lowercase the token and peel the punctuation off its edges, so 'cat' and 'cat,' hit the same bucket. Skip that and you count 'The' and 'the' as two different words, and every frequency in your map is quietly wrong.

---

### The build, step by step

#### Step 1: Normalize case up front

Call `text.lower()` on the whole string once, before you split. Fold case at the source and 'HELLO', 'Hello', and 'hello' become one key for free. Lowercasing per-token later works too, but it invites the bug where one branch forgets to do it.

#### Step 2: Strip only the outer punctuation

For each token, `token.strip(string.punctuation)` shaves leading and trailing marks ('cat,' becomes 'cat', 'wait...' becomes 'wait') while leaving anything interior alone. That last part is deliberate: 'don't' keeps its apostrophe and stays one word, which is exactly why edge-stripping beats deleting every punctuation character.

#### Step 3: Tally with dict.get

Accumulate into a plain dict: `counts[word] = counts.get(word, 0) + 1`. One pass, O(n), and the default-of-zero pattern means you never special-case the first sighting of a word. Guard against the empty string so a token that was pure punctuation doesn't create a '' key.

---

### The solution

**Lowercase, edge-strip, tally**

```python
import string

def the_word_map(text):
    counts = {}
    for token in text.lower().split():
        word = token.strip(string.punctuation)
        if not word:
            continue
        counts[word] = counts.get(word, 0) + 1
    return counts
```

> **Cost analysis**
>
> One pass over the tokens, O(n) in the character length of the input. Space is O(k) for the output dict where k is the number of distinct words, which is bounded by the vocabulary, not the corpus size.

> **Interviewers watch for**
>
> Whether you say the tokenization rule out loud (letters only? edge punctuation only? what about digits?), whether you normalize BEFORE counting rather than after, and whether you notice that stripping ALL punctuation versus only the edges changes how 'don't' is counted.

> **Common pitfall**
>
> Two failure modes on this shape. Counting the raw split tokens leaves 'cat,' and 'cat' as separate keys. Over-correcting by deleting every punctuation character shatters 'don't' into 'dont' or into 'don' plus 't', depending on how you do it. Edge-strip is the sweet spot the visible tests point at.

**Manual dict.get**

Explicit and dependency-free: `counts.get(word, 0) + 1` reads as exactly what it does, and it is the version to reach for when the interviewer wants to watch you handle the accumulation by hand.

**collections.Counter**

`Counter(words)` is the idiomatic one-liner and hands you `most_common(k)` for the follow-up. Same O(n). Cast to `dict` if the contract insists on a plain dict. Reach for it once you have shown you know what it is doing underneath.

---

## Common follow-up questions

- How would you return only the 5 most frequent words? _(Reach for Counter(words).most_common(5). Shows you do not need to sort the entire dict to answer top-k.)_
- How would you count words across a 50 GB file that does not fit in memory? _(Iterate line by line and feed each tokenized batch into a single persistent counter. Discuss the memory bound: it scales with vocabulary, not corpus size.)_
- How would you make 'don't' and 'dont' count as the same word? _(Decide the policy explicitly: strip apostrophes before tallying, or keep them and normalize afterward. Surface the tradeoff with possessives like 'Alice's'.)_

## Related

- [All practice problems](https://datadriven.io/problems)
- [Mock interview mode](https://datadriven.io/interview/the_word_map)
- [Python Interview Questions](https://datadriven.io/python-interview-questions)
- [Data Engineering Interview Prep Guide](https://datadriven.io/data-engineer-interview-prep)
- [Daily Challenge](https://datadriven.io/daily)

---

Source: DataDriven (https://datadriven.io). DataDriven is the data engineering interview community. Live code execution in SQL, Python, and Spark sandboxes. Every feature is open to every member.