Skip to main content

21 posts tagged with "Devlog"

View All Tags

The Benchmark Grows Up

Trevor McCormick
Data Product @ Disney+

That clip up top: watch the test set grow, tile by tile, from four matches to seven — the point counter climbing from 135 to 491 human-charted points. Then its self-reported trust score recalculates live on screen — down, then back up — as the reason for the dip gets found and fixed. It closes on a real point from the Sabalenka–Pegula U.S. Open final: a serve into the body, a backhand up the middle, a forehand rally ending wide and deep — flagged correctly trustworthy at a 0.902 score by a version of the tool that had never watched that match before.

Would you trust a player's world ranking more if it came from one dazzling week at one tournament, or from a whole season played across different surfaces and different opponents?

The Draft Grades Itself

Trevor McCormick
Data Product @ Disney+

That clip up top: the machine's actual draft of the Roland-Garros final, point by point — each gets a chart string, a confidence score, and a flag, HIGH rows glowing green. Three flagged points then play back with their draft on screen, from a clean ace (s4*: serve out wide, unreturned, 0.969 confidence) to a messy point full of "not sure" marks (0.933 confidence). The running tally: 59 points drafted, 16 flagged "start from this draft."

Say a machine hands a volunteer chart-corrector a stack of drafted points — how's she supposed to know, before she's watched a single one, which drafts are worth trusting and which she should chart from scratch?

Within One Token of a Human

Trevor McCormick
Data Product @ Disney+

That clip up top: an ad-point rally from the Djokovic–Ruud Roland Garros final, just under six seconds of video, with the machine's chart building itself token by token above the line a human volunteer wrote for the same point. Four pieces light up green, one after another, because the machine and the human agree on every one of them: a second serve landing in the body of the box, then three backhands in a row — two to the same spot, then one that breaks the other way. Then the last ball dies in the net. The human's shorthand for that is the letter n; the machine's chart shows a different mark instead, *, so that last piece glows amber — one disagreement. Under the rule this post is about, one disagreement is still close enough to count: the point is accepted. The video's closing screen turns the whole story into four bars.

Here's the question this post is finally built to answer: if you handed the machine's chart of a point to the volunteer who charted that same point by hand, how many things would she have to fix before she'd sign her name to it?

The Score Bug Is the Point ID

Trevor McCormick
Data Product @ Disney+

That clip up top: a 21-second, sixteen-shot Djokovic–Ruud rally from the Roland Garros final. The old segment-finder had chopped it into four broken pieces — the tick marks on the timeline are where those old cuts fell, and each piece charted only 1 to 3 shots alone. This post's method proved all four belong to one point, glued them back together, and the same shot-detector read the reunited rally as 12 of 16 shots correctly.

Every TV director cuts away sometimes — a reaction shot, a replay, a different angle. So how do you prove, from footage alone, that two clips are really the same point? Two test matches weren't enough, so this session added harder ones: the 2023 Roland Garros final on clay, and the 2024 Wimbledon final on grass. Four matches now, three surfaces, both tours, about 108 new points checked against a human's chart, for zero dollars. What broke wasn't one narrow piece — it broke the assumption under every number on the board, and the fix had been on screen all along.

The $0 Specialist and the Load-Bearing Bug

Trevor McCormick
Data Product @ Disney+

That clip up top: the same Federer rally, tracked two different ways. Left, the old general-purpose tracker's trail of yellow dots — a comet's tail — thins out and dies partway through; it wrote down only six of twelve shots. Right, a tiny tennis-only specialist's trail stays solid to the last shot. Same clip, same notation software underneath both — the only thing that changed is who's watching the ball. The specialist charted all twelve shots, and got every one of the seven shot symbols it committed to right.

Before spending another dollar chasing a tennis ball across the screen, I stopped and asked a simpler question: has somebody already solved this?