probe_gate reports all 34 probes as regressions when Godot is absent, and offers to file it as an issue nf-u77q ← Beads

closed priority 2 task unassigned

Twinned from https://github.com/cfsh/ninefold/issues/584 by tools/beads/import_github.py's reconcile pass.

On a box with no Godot, probe_gate.py reports every probe as a regression and hands the operator a pre-filled command to file that as a public issue. Measured on the WSL office box just now, --dry-run at 3a936a9:

``` ──── 0 passed, 34 failed ──── ████ FAILING HERE, PASSING IN MAIN'S BASELINE ████ butterbar-gate … tonapse-home (all 34)

Most likely this branch carries the regression, or main has moved on. File it (not filed automatically — outward-facing): gh issue create -R cfsh/ninefold \ --title "feat/office-spawn-script: 3a936a9 regressed butterbar-gate chit-budget … tonapse-home" \ --body "probe_gate on … Full sweep: 0 passed, 34 failed." ```

Nothing regressed. Every one of the 34 failed with the same line, which the sweep log *does* contain:

``` Godot console exe not found at C:/Tools/Godot/…_console.exe (set GODOT_EXE) ```

So the information needed to tell "the environment cannot run this" from "the product broke" is present in the sweep and discarded by the summary.

Why this is newly reachable, and why it is not #566's already-fixed half

#566/#567 closed the dangerous half — a Linux sweep silently overwriting the shared, platform-specific baseline. record_ok is now cleared on a platform mismatch, and probe_gate.py:85-110 documents the measurement behind it. That guard works and this issue does not reopen it.

What is left is the *reporting* half. After a merge, HEAD is origin/main, so the branch guard above does not fire and the platform guard only stops the write — the prose still prints. And a merge from a Godot-less box is newly possible: dotnet was installed here and #567 made --all pass, so merge_pr.py now gets past its gate, merges, and then runs probe_gate post-merge and non-fatal (merge_pr.py:464). The operator's last screenful is therefore *"main @ <sha>: 0 passed, 34 failed"* plus a ready-to-paste command to announce a 34-probe regression that did not happen.

⚠️ Loud-but-wrong, not silent — which is the better failure and worth saying. But it invites an action that is wrong, and probe_gate's own header says the gate exists so that *"a human is present at this exact moment and knows which merge did it."* Handing that human a false answer is the specific thing it is for.

Shape of a fix
Operational note for anyone merging from Linux today

Until this is fixed, merge with SKIP_PROBE_GATE=1 from a Godot-less box. The merge itself is unaffected — the gate is post-merge and non-fatal either way — and the baseline is protected by #567 regardless. What you avoid is the false regression report and the invitation to file it.

Filed unclaimed by nf-dev-office-spawn while working #557. Not on my chunk — found preparing to answer @aedanpope's open question on #558 about whether to merge from this box. Related: #566, #567, #389.

Dependencies

None.

Comments

No comments.

Add a comment