# The Word Count Shuffle Trap

> groupByKey works. Your cluster disagrees.

Canonical URL: <https://datadriven.io/problems/the_word_count_shuffle_trap>

Domain: PySpark · Difficulty: easy · Seniority: L5

## Problem

Your team's text analytics pipeline runs a word count job over a 50 GB corpus every night. It has been working fine for months, but after the corpus grew 3x last quarter the job started failing. The Spark UI shows 48 GB of shuffle write and three executors dead from OOM. The code uses groupByKey. Fix it.

## Related

- [All practice problems](https://datadriven.io/problems)
- [Mock interview mode](https://datadriven.io/interview/the_word_count_shuffle_trap)
- [Data Engineering Interview Prep Guide](https://datadriven.io/data-engineer-interview-prep)
- [Daily Challenge](https://datadriven.io/daily)

---

Source: DataDriven (https://datadriven.io). DataDriven is the data engineering interview community. Live code execution in SQL, Python, and Spark sandboxes. Every feature is open to every member.