The Score Bug Is the Point ID
That clip up top: a 21-second, sixteen-shot Djokovic–Ruud rally from the Roland Garros final. The old segment-finder had chopped it into four broken pieces — the tick marks on the timeline are where those old cuts fell, and each piece charted only 1 to 3 shots alone. This post's method proved all four belong to one point, glued them back together, and the same shot-detector read the reunited rally as 12 of 16 shots correctly.
Every TV director cuts away sometimes — a reaction shot, a replay, a different angle. So how do you prove, from footage alone, that two clips are really the same point? Two test matches weren't enough, so this session added harder ones: the 2023 Roland Garros final on clay, and the 2024 Wimbledon final on grass. Four matches now, three surfaces, both tours, about 108 new points checked against a human's chart, for zero dollars. What broke wasn't one narrow piece — it broke the assumption under every number on the board, and the fix had been on screen all along.
Previously: cv-10 gave the project its first honest report card, grading its draft charts against a match a human had already charted by hand. cv-12 then swapped in cheaper, more reliable ball-tracking software — and uncovered an accident: the old tracker's habit of losing the ball when a rally ended had secretly been flagging "this point is over." The new tracker never loses the ball, so that free signal vanished, leaving the project needing a real point-boundary detector. cv-06 had already learned a broadcast is mostly not rally footage — finding the stretches that even show the court took its own detective work.
The one graphic that never lies
Picture a broadcast with the sound off. Almost everything keeps changing — players move, the camera pans, the crowd shifts — except the scoreboard graphic in the corner. It isn't part of the court; it's stamped on top of the picture like a watermark, so a pan can't nudge it. And it obeys a rule nothing else on screen does: frozen, pixel for pixel, for the whole length of a point, changing at exactly one moment — when the point ends.
That discipline is what this series calls the score bug as metadata (more in the primer): the graphic, just by sitting there frozen, quietly carries the match's structure. Two things fall out of it for free. It's a point ID — two chunks of video showing the identical frozen scoreboard are fragments of the same point, however the editor cut between them, and can be safely stitched together. And it's a join key — a shared label matching this clip to its point in the human charter's own record.
Clay and grass fight back
The clay final (Djokovic vs. Ruud) broke the court-mapping recipe that worked fine on hard courts. Turning the TV picture into a court diagram needs the homography — the screen-to-court conversion recipe — and its usual tricks assume a blue hard court with crisp white lines. Clay has neither: the "find the blue court" trick found nothing, and the trick that erases players by blending frames together smeared the faint lines into the dirt instead. A clay-specific rebuild (appendix has the method) got it working again, accurate to a fraction of a pixel.
The grass final (Krejcikova vs. Paolini) broke the check on the recipe instead: the software found a court mapping whose lines lined up neatly with each other — and it was wrong, like an umpire confidently announcing a plausible score that isn't the real one. That's an impostor fit. The fix: instead of "do the few lines this mapping used line up?", ask "do all nine of the court's real lines land where this mapping says they should?" A wrong mapping explains a few lines and leaves the rest floating; the real one explains all nine. (One more check earned its keep on this match too, following cv-11 — the appendix has it.)
A "segment" is not a "point"
Both matches mapped, the draft charts still came back mediocre — for a reason underneath every number, not inside any one of them. The pipeline assumed one unbroken stretch of court-view video (a segment) equals one point. That breaks in opposite directions, depending on who's holding the editing scissors.
Roland Garros cuts away mid-rally, chopping one point across several segments: the draft charted just 3.9 shots per point here against the human charter's real 8.9. The rally was there; the video just told it in pieces. Wimbledon does the opposite, almost never cutting between points, so one fifteen-second stretch can hold two or three back to back — the draft overcounted there, 8.1 shots per point against the real 7.2, stringing separate points into one fake rally.
The two earlier test matches had simply gotten lucky: their broadcaster cut almost exactly once per point, splitting the match for free — the same kind of accident as the old tracker's habit of losing the ball, doing a job nobody built it to do.
Reading the scoreboard settles it
This is the gap the score bug closes. Two chunks of video with the identical frozen scoreboard, however the editor cut between them, are the same point — merge them. One chunk where it changes partway through is secretly two points — split it there. Read the sequence of changes and you get the real point-by-point structure, regardless of where the editor cut. It held with a wide, safe margin between "same point" and "different point" readings, and a handful of broadcaster quirks that tried to break it — a scoreboard that grows after every set, a tiebreak banner that shoves it sideways, a fade-in that could fake a change that never happened — all got caught by looking closely at the pixels (appendix has the numbers).
One theory got tested and killed outright, worth saying since it contradicts what I'd guessed going in: I suspected slow-motion replays were sneaking duplicate points into the count. They aren't. Every suspicious duplicate score turned out to be fragments of one rally, or a genuinely different point that passed through the same score — deuce recurs, and the scoreboard alone can't tell those visits apart, only their order can. The last suspect was cleared by physics: the ball's own falling motion matched real-time gravity, and a slow-motion replay would fall roughly four times slower.
With segments stitched into points correctly, the clips lined up against the human record almost perfectly.
The scorecard
Matched row for row against the human-charted record — the independent answer key this project grades itself against:
| What was measured | Before | After |
|---|---|---|
| Clips matched to their correct point, clay | 59 of 68 | 59 of 59 — every one |
| Clips matched to their correct point, grass | 36 of 48 | 49 of 49 — every one |
| Rally length guessed within one shot, clay | 34% | 41% |
| Rally length guessed within one shot, grass | 15 of 36 (about 42%) | 21 of 49 (about 43%) |
| Serve zone guessed correctly, grass | 1 of 4 | 8 of 10 |
| Which player served, clay (honest step back) | 10 of 59 | 7 of 58 |
| How the point ended, clay (honest step back) | 11 of 34 | 9 of 30 |
The two rows that went down are real, worth naming rather than hiding: merged clips can start mid-rally, so the software falls back to a weaker clue (simple alternation) for who served and how it ended. A fair trade for getting the boundaries right, but a real cost. Where the merges had footage to work with, the payoff showed: the 16-shot rally up top went from four broken pieces to one point charted 12 of 16 correctly, and another clay point came back a perfect 12 of 12.
Two problems stay open, neither a scoreboard problem anymore. On clay, the editor sometimes cuts into a rally already in progress, so some early shots were never filmed — the scoreboard proves the clip is one point, but can't invent missing footage. On grass, 19 clips still count as two points when they're really one, because Wimbledon's dead time sits inside that same frozen reading (mechanism in the appendix). Both are chart-reading problems now, not video-editing ones — and they're next.
The least glamorous graphic on the screen — a tiny scoreboard nobody watches on purpose — is the one witness every broadcaster in the world agrees to keep rolling at all times, which makes it the most trustworthy thing in the picture.
For the technical reader
t3, clay (Djokovic–Ruud, RG world feed, 34 min highlights): M1
homography died three ways — "blue court hull" has no blue, and the
clay-hue mask swallows the frame (crowd skin ≈ clay hue); median plate
smears lines into dirt (16 px pan, white-on-orange loses to
white-on-blue); far baseline buried (V200 on V~195 clay). Rebuild:
ECC-stabilized plates, thin-bright tophat filter, verticals labeled by
symmetry (courtside Perrier crates give 20 Hough segments inside the
hull; junk is asymmetric, lines aren't), fit basis swapping the buried
baseline for the far service line. Result: sidelines 0.4/0.5 px,
center line 0.2 px.
t4, grass/WTA (Krejcikova–Paolini, Wimbledon feed): a wrong-but-self-consistent line assignment fit its own intersections to rms 1.7 px. Fix: score candidates by how well all nine model lines land on the observed mask — impostors strand the lines they didn't use.
Handedness: MCP's metadata listed Vondrousova, a well-known left-hander, as right-handed. Post-cv-11 policy: never trust the chart's "hand" column, look it up independently.
Segment note: some RG fragments in the 3.9-vs-8.9 split were written off by the staging pass as replays.
Score bug validation: same-point crops differ by ≤0.005 changed-pixel fraction, different-score crops by ≥0.033 — a 6x margin. Traps: the RG bug grows a column per set (digits slide ~27 px right); the set-1 tiebreak banner drops it 14 px and drifts; a reference frozen on a fade-in frame lets compression noise fake a change 100 frames later. Fix: frozen reference per plateau, anchored on the median of a settling window.
Replay hypothesis, refuted: every duplicate-score group was fragments of one rally or distinct deuce points (a 40-40 plateau sits between "duplicate" 40-AD clips: three deuce cycles; order carries what value alone can't). Cadence suspect acquitted: WASB-track ball gravity is unimodal across all 116 clips; 2x slow-mo would sit ~4x low.
Remaining gaps, mechanism: t4's 19 over-counted clips fail because Wimbledon's dead time lives inside the point's own plateau, the dead ball stays inside the court-y envelope, and the freeze-#3 gate can't touch it — the t1 dead-ball coda in a new costume.
Session cost: $0.00. Project total: about $4.15 of a $9 budget. The benchmark is now four matches, three surfaces, both tours — and every point ID in it comes from the one witness every broadcaster agrees to keep on screen.