The Weekly · Thread
A launch weekend is 10 days out and the delivery team needs to know what is ready to play. The only record is a feed nobody has read in 6 years.
The brief is at https://datadriven.io/community/week-2. Ask about the brief, the rules, the schema. Talk about what you are seeing in the sample.
The results are in. This thread is now the place for approaches, write-ups, and what beat you.
Why reading resolution off height was the wrong safe bet
The rendition tag hazard, present in about 6% of the stream, divided finishers as much as anything in the manifest: the best attempts read profile pairs correctly nearly every time, while the median attempt recovered 14.4% of them. The misreads came from a habit that's normally exactly right: a profile name like 1080p is read by convention off the height. Most finishers carried that convention into the width-and-height pairs and the bare numbers scattered through the feed, and got burned by a fact particular to this catalog: scope titles run wider than a standard 16:9 frame, so the height in a resolution pair no longer lines up with the rung. The number that actually identifies the rung there is the width, and only a small share of finishers built the override for it.
Identity resolution, by contrast, is something nearly every finisher got right. Duplicate sends and a stale backfill re-delivering an older packaging run both got the same answer: hold what you've seen under its natural key, keep whichever copy carries the newest packaging time, drop the rest. That move recovered the overwhelming majority of both hazards, with little spread between finishers. The packager's job-keyed lines, at just 2.5% of the stream, asked for a related kind of patience that few attempts built: the row shows up naming only a job, no title at all, and the title only lands several lines later on that job's own closing line. The holding pattern that solved duplicates was scoped to a key already in hand; this one calls for holding a row against a key that hasn't arrived yet, and reconciling once it does. So where duplicates needed a lookup, the job-keyed lines need a join across time inside a single stream.
2 designs for handling identity showed up in the attempts: some held every row until the very end and flushed once, others emitted rows the moment they were resolved. Both are legitimate on their own, but at least one strong attempt ran both at once, resolving rows on the way through and then flushing the held set again at the close, sending the same records out twice despite a handler that was otherwise doing real work underneath. The emit strategy has to be picked once for the whole handler.
Everything else mechanical, the padding on ids, the quoted numbers, the several spellings of a missing value, the fields arriving in a different order every time, got cleared at a similarly high rate by almost every finisher, which is the baseline this kind of feed now expects going in. And one of the strongest attempts on the whole feed used no pattern matching at all: it read the field contract closely and applied it literally, line by line.
Hello, could you explain why last night the result show me got silver #2.
But now it updated and my result is #3 ? What happened ?
Apologies, an infra error on our side. Your solution scoring didn't change. Some users that had heavy solutions were being incorrectly disqualified. When counting those users solutions into the scoring, the podium shifted. Going forward, the grading accuracy+infra issues have been resolved so that this won't happen again!
Hi guys, how you guys doing ?
how do you feel about this week challenge compared to the previous week ?
This challenge is much more polished. This challenge requires some interesting structural repairs to salvage records, and we are excited to see who's going to anticipate them. We anticipate a much wider spread; a great answer this week is going to be much further from the average - a big spread!
