M3: The Static Camera Isn't

That clip up top: every serve the pipeline felt sure about, clip after clip — which player served, and which side of the court, deuce or ad, flashed the instant the toss goes up.

Sixty broadcast clips are sitting there ready to be charted, and every one starts with the same problem: telling an actual tennis player apart from everyone else standing near a court who isn't one.

Previously: the last post sliced a highlights reel into sixty complete-point clips using screen-to-court math as a camera fingerprint. That math, the homography — matching the court's painted lines to its real dimensions to turn screen position into court position, from Pixels to Meters — carries a free trick: median-blend many frames of the same court and the players, who never stand still for long, vanish into a photo of an empty court. Two posts later, that trick already found both players in one rally well enough to call every forehand and backhand. Doing it across sixty new clips would otherwise mean paying SAM 3 — the pay-per-prompt "find that object" AI that tracked a ball for about sixty cents in the second post — roughly $0.15 a clip, $9 for the batch, every rerun.

Comparing two photos of the court

This session tried to cash in that free trick everywhere: build the empty-court photo per clip, then flag whatever pixel changed as a player. One catch — both photos need the same camera position. A broadcast camera looks fixed, but any camera operator worth their job nudges the shot to keep the rally centered, the way your own eyes would tracking the ball from courtside. A pixel-by-pixel comparison notices every nudge: the baseline in one photo no longer sits on the baseline in the other. It's like tracing a picture on an Etch A Sketch, then shaking the toy before checking your work — nothing lines up.

The fix: stabilization, nudging every frame back into line with the others before comparing anything, so the same pixel always means the same spot on the court. More on why the empty-court trick needs a still camera: Court Vision in Plain English.

The camera that wasn't sitting still

That math, first fit two posts back on a sixteen-second rally, found almost no camera movement — so the code kept one screen-to-court conversion for the whole clip. A full point can run a minute or more, and these sixty clips run that long. Aligning every frame of a clip to its own first frame and measuring the gap told a different story:

The broadcast camera, measured, over one point: a sharp jump — as much as 32 pixels — mid-point, on top of slow drift throughout.

The camera operator was following the play — invisible to a viewer, fatal to the comparison. It blurs the empty-court photo, and near the far baseline, thirty-some pixels of drift is worth meters of court-position error. Every frame now gets stabilized first.

The far player who turned into a ghost

With frames stabilized, the near player — big, always moving — showed up in essentially 100% of frames; the far player, barely 35%. The empty-court photo had swallowed him.

He's small, and barely moves between points, so a photo built from one clip bakes in a faint standing copy of him — the same trick that erased a stationary line judge two posts ago.

The fix: build the photo from five clips instead of one — this clip plus its four nearest neighbors, aligned and blended by taking the middle value at every pixel again. Players stand somewhere different every point, so five ghostly copies never land on the same pixel twice, and the blend votes them all out. Far-player coverage: 35% to 96%. Near player: still ~100%. Cost: nothing.

Everyone near the court looks like a player

The first serve-detector run flagged several clips with the far player standing 13.5 meters from the court's center, on a court only 10.97 meters wide. Nobody serves from outside their own court.

Those weren't players. They were ballkids, crouched past the doubles sidelines inside the patch of court scanned for the far player, shifting weight just enough to out-move a motionless server. Third case of mistaken identity: the very first session swept spare-ball-holding ballkids into "the tennis ball," and two posts ago a stationary line judge got swept into "the empty court." To a pixel-by-pixel comparison, anyone standing near a court looks like they belong on it. Fix: the far-side scan region is now cut tight to the court; the near side stays generous.

Guessing who served, from posture alone

With players tracked, the serve should be readable from posture: the server hugs the center mark, the returner waits wide — which side of the mark gives away deuce or ad, for free — and the toss stretches the server's outline to full reach, so blob height should spike right at the toss.

Version one checked only distance to the center mark, and failed fast: some clips open mid-rally with no toss to find, some end on a leftover zoom fragment, and — the costly lesson — returners often stand nearly as far from the mark as a server does, so a return swing can read as a toss.

Right: a clean toss, full stretch, on the flagged frame.

Wrong: the flagged "server" is the returner, off the mark, mid-swing — every check in version one said yes.

Version two clears several checks before it commits — tracked early and reliably, right distance from center mark and baseline, a genuine height spike — or it refuses, with reasons attached, instead of guessing.

Result: 40 of 60 clips get a confident serve call; the other 20 are refused, on the record — clips that open mid-rally or rejoin mid-point, plus two long clips too thin on far-player tracking to trust, including the marathon 49-shot rally from the last post. Spot-checking a sample of the confident calls by eye: 6 of 8 named the right server, 5 of 8 found a usable toss moment — honest, not perfect. Geometry alone tops out here; a serve is, definitionally, the ball leaving the hand and crossing the net, and once the ball itself is tracked over these clips, it settles what posture can only guess at.

The scoreboard

What got checked	Result
Far player, before the fix	seen in 35% of frames — mostly a ghost
Far player, after the fix	seen in 96% of frames — essentially solved
Near player, throughout	seen in ~100% of frames — never a problem
Clips confident enough for a serve call	40 of 60
Clips that honestly refused to guess	20 of 60
Spot-check: correct server named	6 of 8
Spot-check: usable serve moment found	5 of 8

The takeaway

A camera that looks perfectly still to a viewer at home can still wreck a pixel-by-pixel comparison — so before comparing two photos of the court, make sure they were taken from exactly the same seat.

For the technical reader

Alignment: each frame is registered to its clip's first frame using ECC (Enhanced Correlation Coefficient) — an alignment method — restricted to pure translation (x/y shift, no rotation/scale). point_29 is the 32px-spike example, now the mandatory first step before plate-building, background subtraction, or homography projection. The free trick cashed in this session: frame-minus-median already IS a player detector, left over from M1's clean-plate trick. Ghost-plate fix: a 5-clip window (target + 4 nearest temporal neighbors), each ECC-aligned before median-merging; zero added API cost, pure numpy.

Serve detector v1: single heuristic, distance-to-center-mark only. point_33: clean, correctly-flagged case. point_40: failure case, flagged "server" is the returner, mid-swing, off-hash, passing every v1 check anyway. v2: four required gates — (1) early-clip track coverage, (2) distance to center mark, (3) distance to own baseline, (4) a genuine local peak in blob height. Any failed gate produces an explicit no_confident_serve result with reasons attached, not a forced guess. Refusal log flags cold-opens, mid-point rejoins, and two long clips with sparse early far-tracks — including the 49-shot rally from the cutting-room post (cv-06).

Also queued: fps as a real parameter — this reel runs 25fps against the original clip's 30fps, and velocity thresholds are secretly per-frame numbers.

What's next

Session tally: player positions across all sixty clips, serve details on two of three, three wrong assumptions caught — for $0.00. Project total: still about seventy-five cents, all spent in the very first tracked-rally session. SAM 3, the paid tracker, has exactly one job left: finding the ball.

Next: a ball detector on these same sixty clips. Then the serve call, hits, bounces, and shot letters should click into a complete point, written in the Match Charting Project's own notation — the human-made answer key this project is chasing. The one working chart string so far, from two posts ago, was just the middle of a rally. The next one should have a beginning, a middle, and an end.