T1: Right Hardware, Wrong Sign
That clip up top is match point of Shapovalov's win over Nadal,
Montréal 2017 — the point that made his career. Our software's chart
assembles on the top line; a human volunteer's runs below it. The
human's line ends f+1* — forehand (f), closing in on the net (+), to
a corner (1), a clean winner (). The software's ends with Shapovalov
flat on his back, most labels still ?, its symbol for "couldn't
tell." That gap is what this post closes.*
Could software, rules locked exactly as last tuned, watch a match it had never seen and write a chart a tennis fan would call the same match?
Previously
Two posts back, the software learned to guess who hit each shot by weighing three signals — did the ball reach that side, the serve call, where the shot landed. Last post, a human volunteer's chart of a real match — Nadal against Shapovalov, at night, in Montréal — became the project's first answer key, ending a stretch of calling its own lines: fine for practice, worthless as proof. (Why that matters: Court Vision in Plain English.) Today's rule, set before a single clip played: nothing changes before the scorecard.
Ends swap. The software had to learn that too.
Every tennis fan carries this rule without thinking: players swap ends after the first game, then after every odd-numbered game total, and every six points in a tiebreak. Tell a fan the score is four games to three and they'll say, without looking, which end each player stands at — arithmetic, not a guess. The software had to learn that same arithmetic: no coin toss to watch, just a ball arriving from the near or far side of the screen, so it computes which end a player stands at from the score alone.
That arithmetic matters beyond serve calls. When software is wrong in a way that flips on one fact it never checked, the fault usually isn't what it's measuring — it's a label it assumed instead of computed. Call this the mirrored error: every miss points the same direction, so the tool's eyes work fine and only a naming convention flipped — a line judge with perfect eyesight calling "out" every time she means "in." You wouldn't trust her calls, but you wouldn't send her for glasses.
What carried over from the practice match
Stabilizing the footage, building the empty-court reference photo (the median-frame trick that erases players and ball so anything left over stands out), and tracking players ran on the Montréal footage unchanged. Far-side coverage improved: 100% here versus 96% on the practice reel, because floodlights kill the shadows that had tricked the daytime footage.
A new problem showed up — the fourth time something that isn't a player has fooled the player-finder. This court's ball kids crouch at the net posts, and the code assuming "the biggest moving blob is the player" grabbed them, opening serve-toss windows above a ball kid's head, once above a chair umpire's. The fix is geometry: a server stands at the baseline, so the bootstrap checks each candidate's foot position in real court coordinates — the screen-to-court conversion the project runs on — and rejects anyone standing where a server never would.
Who served? Ask the score, not just the video
The serve detector called a direction confidently on 11 of 25 clips here, down from 40 of 60 on the practice match — a harder test: a different match, at night.
The human chart already knows who served every point; changeover parity turns that into "which end that server stood at," once you know where the match started. Run it both ways: "Nadal started far" explains 11 of 14 confident calls, "near" explains only 3. Nadal started far — a real accuracy figure of 11 of 14, about 79%, up from 6 of 8 on the practice match, checked by eye instead of a chart.
Paying only for tracks worth paying for
Ball tracking still runs behind a spending gate: a human eyeballs a proposed box before the project pays to send it to the paid, prompt-it-yourself tracker. Of 25 proposed boxes, 13 were approved — nine on the first look, three more after the ball-kid fix, one via a backup rule that just follows whatever's moving fastest. Twelve were sent, $0.85 total, eleven usable; the twelfth, a shaky approval, died twelve frames in. That check is 6 for 6 across both matches at picking winners. It stays.
The scorecard
Eleven points, chart locked, graded against the human's:
| What was checked | Result | What it means |
|---|---|---|
| Got the server right | 7 of 11 | Whose racket started the point |
| Rally length, within one shot | 7 of 11 | How many shots the point took |
| Serve landing spot in the box | 1 of 2 | Only two points had enough signal to grade |
| Forehand/backhand, exact match | 2 of 11 — about 1 in 5 | The weakest reading on the sheet |
| Forehand/backhand, allowing a mirror | 9 of 11 — better than 4 in 5 | Flip the label and it's the strongest reading |
That last pair is the headline — the best bad result this project has produced. Nadal and Shapovalov are both left-handed; the software still assumes every player is right-handed, a risk flagged and frozen on purpose so a lefty-heavy match would expose it. A lefty's forehand lands where a righty's backhand does, and vice versa — assume the wrong hand and the label flips, though the geometry stays right. Nine mirrored calls out of eleven means the software found the right side of the body at contact almost every time — just the wrong name for it. One line of match configuration — telling it who's left-handed — turns today's 2 of 11 into roughly 9 of 11 at the next freeze: the cheapest seven-point improvement this project will ever buy.
The rest of the misses hold up too. A tie-break rule letting a strong serve call override a shaky forehand/backhand vote had rescued one confusing point on the practice match; here it pushed two calls wrong instead. And rally length runs long because some points include a fault — a missed first serve — before the point that counts; the human chart logs both separately, but the software counted the fault as part of the rally. Starting the count from the first confident serve fixes both.
When every miss points the same direction, you haven't found a broken tool — you've found a fact about the match it never bothered to check.
For the technical reader
- The tie-break rule that wronged two calls here fixed point_59 on the dev reel: thin far-end tracks make touch votes lie harder than serve priors, so its vote weights are a flagged dev-reel artifact.
- Handedness was flagged as a known liability in the who-hit-it milestone (M3) and frozen on purpose for this test.
- The ballkid impostor triggered the largest-connected-component player pick; the paid tracker throughout is SAM 3.
The verdict
An automated system charted eleven points of a tennis match from broadcast video and got graded against a human's chart of those points — the actual thing this project set out to test, now with a number instead of an impression. The number is bad in the most useful way a number can be: the biggest error has a direction, a known cause, and a one-line fix.
Test-set discipline held — nothing tuned mid-grade. Session cost $0.85; project total about $3.65 of the $9 budget. Next: handedness goes into the match configuration, then a control match — Federer against Haase, two right-handers — answers whether this was ever really about left-handedness at all.