The Manifest: Week 2
The Weekly · Week 2 · Stream
The Manifest
Next drop Monday Sep 28, 16:00 UTC9 inFirst in coral_mesa_9943
The challenge
You're a data engineer at a video streaming service. Every title is encoded into several quality levels, called renditions, and a feed has logged 1 JSON line per rendition since 2020. Before a launch weekend, the delivery team needs 1 clean row per rendition to decide what to cache near viewers, and the feed is the only record of what was made. Write the code that turns it into that table.
The mess
The feed is 6 years old and nobody owned it. 3 different encoding tools wrote to it in turn, each using its own names for things. When a tool changed, titles were encoded again and the old lines stayed. Quality levels were named by whoever set them up. Lines are written by a logger, not a serializer. A batch goes out again whenever a job is replayed.
1,000,000 messages
Click a column for its requirements, or click the bottom of the sheet to see the 10 sample rows to produce.
Notes from the delivery team
- 1 row per rendition of a title.
- Latest packaged_at wins for a rendition.
- A time with no zone is UTC.
- packager, ingested_at and other feed extras are bookkeeping.
- Each output logs under its job; the job's close line comes last.
How you're scored
- +1the right value. A right null counts.
- 0null where the source held a value. Silence is free.
- −1a wrong value. Silent corruption costs.
- −2a row that should not exist: no matching key, or a second row for the same key. The last row for a key is the one scored.