Twinned from https://github.com/cfsh/ninefold/issues/474 by tools/beads/import_github.py's reconcile pass.
Six reviewer passes across #453/#455 caught the same class of defect repeatedly: assertions that passed for reasons unrelated to what they claimed to test. Not one of them was found by the sweep; every one was found by a human-shaped reading of the code. Concrete instances from one session:
| assertion | why it passed | what it claimed |
|---|---|---|
| new-channel.json focus legs | every re-entry forced focus with {"focus": …}, which masks a dead focus by definition | that focus survived the flow |
| ModalFitsViewport in scale-200.json | ran after a *live* scale change, where the floors it measures are stale (#454), so it never saw the scaled width | that the modal fits at 200% |
| RollShown (D11 leg) | neither the L1/R1 taps nor the shoulder chord reached an empty bar, so the fill panel never appeared | that a layout flip keeps the roll |
| RingFit v1 | asked "do labels clear the inner disc" when the collision is *horizontal* | that ring text doesn't collide |
| single-screen.json's △ legs | pressed △ from wherever the menu left focus, so the dive-in branch never ran | that △ dives correctly |
Same shape every time, and the cost is worse than no check: a green gate on the regression under repair reads as coverage.
Mid-session I started re-running each new assertion against the broken build before trusting it — revert the fix, confirm the assertion goes red with a message that names the real symptom, restore. It caught RingFit v1 and the RollShown leg immediately, and it turned CellSize from a guess into the thing that located a second root cause (64x112 → the options row never raised Changed).
It costs one extra build + probe run, which is minutes, against review rounds that cost hours.
Add to CLAUDE.md, next to the existing "UX-behaviour changes: probe tests, not just screenshots" rule:
> A new assertion must be observed failing before it is trusted. Revert the fix (or stub the value), run the probe, confirm it goes red *and that the failure message names the real symptom*, then restore. An assertion that has only ever been green is not evidence — it is a guess with a checkmark.
And a corollary for the forced-focus trap specifically:
> {"focus": …} before an expect_focus proves nothing about focus. Reach the control the way a player does, or assert focus *before* forcing it.
The screenshot half of this is the same failure with a different instrument — twice I posted a shot as proof of a fix and it was showing the bug (one-screen-12-bars-fit.png, one-screen-200-naming-ring.png), and in one case I had noticed the anomaly while looking and discounted it. Where an invariant is expressible, the visual-change rule (#120) probably wants "a shot AND an assertion", not a shot alone.
Filed unclaimed.
None.
No comments.