probe_gate: skip the sweep when a merge changes nothing the Godot build can see nf-56wj ← Beads

closed priority 2 task unassigned activesession:probe-gate

Twinned from https://github.com/cfsh/ninefold/issues/637 by tools/beads/import_github.py's reconcile pass.

From @aedanpope, in chat during #604:

> "BTW I noticed lots of discussion that running probes is expensive for submitting PRs - by similar logic, do we only need to run probes on PRs that touch the godot build?"

Filed unclaimed, with the design worked out. #604 is the f5 half of the same idea and is where the predicate would come from.

The answer is yes, but not by #604's argument

probe_gate.py is post-merge, on main. It is not "the probes for this PR" — it measures *main* and records a baseline stamped with main's sha. Its header is explicit that a PR-side gate would not have caught #387:

> "#359's suite was green on its own branch; the regression existed only in the INTERACTION with a main those probes were pinned to."

So "this branch contributes nothing to the build" — #604's reasoning — does not transfer. A per-PR skip framed that way would quietly re-open #387.

The route that does work

If a merge changes nothing the Godot build can see, main's game tree after it is byte-identical to before — and that tree was already swept by the previous code merge. The sweep re-derives a known answer.

probe_gate.py already makes precisely this argument for the sha-equality case, at :576-597, and it says the re-sweep is *harmful* rather than merely wasteful:

> "That costs 3-4 minutes of Godot launches to re-derive a known answer — and, because probes flake (#471), re-rolling the dice can turn a recorded green into a reported NEW failure attributed to an innocent merge. Not sweeping is both cheaper and more accurate."

⚠️ The shape: generalise the existing skip, do NOT add a new one

The existing early-out is baseline.get("sha") == sha. The change is to widen it to "the baseline's sha and this sha have the same Godot-visible tree", and then carry the baseline forward re-stamped with the new sha.

That last clause is the load-bearing half. Skipping outright would leave the baseline describing an older sha than main's HEAD, so every later report would compare against a stamp that no longer matches — a different and worse bug than the one being fixed. Re-stamping keeps the bookkeeping exact.

Two properties to preserve from the existing skip, both already documented there:

Where the predicate comes from

#604 lands affects_godot_build() in tools/pr/rebuild_f5.py, derived from Ninefold.csproj's <Compile Remove> list and the .gdignore files, with CsprojPremiseTest pinning the premise against the real files.

This is the second caller, so the predicate should move to nflib as part of this issue — one definition of "the Godot build" or the two will drift, and they would drift in the direction that skips a sweep on a merge that did change the game. One caller did not earn the extraction; two does.

Worth it?

On the queue as of 2026-08-06: 7 of 9 open PRs are tooling or docs. Each of those merges currently pays 3-4 minutes of ~24 Godot launches to re-derive a known answer, and gives a flake 24 more chances to be attributed to an innocent merge.

Not urgent. Filed so the reasoning is written down rather than re-derived.

Refs #599, #604, #387, #471

Dependencies

None.

Comments

No comments.

Add a comment