probe.sh silently runs the last-built DLL — build (or refuse) first nf-zr6d ← Beads

closed priority 2 task unassigned

Twinned from https://github.com/cfsh/ninefold/issues/380 by tools/beads/import_github.py's reconcile pass.

Captured from a real misdiagnosis today, not a hypothetical.

probe.sh launches Godot against whatever dotnet build last produced. It has no build step and no staleness check — grep -n 'dotnet\|build' probe.sh → nothing. CLAUDE.md already warns about this in bold ("⚠ the probe runs the last-built DLLdotnet build after any checkout, or the shots silently show the wrong code"), and it still bit.

What it cost, 2026-08-01. While investigating the permanently-red probe_all (#371) I measured two probes as failing with unknown step: {"expect_hint": …} and concluded another session had landed probes ahead of their harness steps. That was wrong — main's harness had the step; my DLL predated a checkout. I reported the wrong root cause, then rebuilt and re-measured before it reached a PR body. The real baseline was 14/5, not 12/7.

The failure is quiet in the worst way: the probe *runs*, prints plausible output, and exits with a status that looks like a verdict about the tree you're looking at. Nothing in the output says "this is code from a different commit". A stale-DLL run and a genuine regression are indistinguishable from the log.

Options

I'd take A + C: A removes the trap, C means any future log carries enough to spot it.

Filed rather than fixed because it touches the probe entry point every UI PR depends on, and the build-once/skip plumbing for probe_all.sh is a design call worth your steer.

Dependencies

None.

Comments

No comments.

Add a comment