menu-focus probe is flaky, and a flaky probe makes the UI gate cry wolf nf-4zcc ← Beads

closed priority 2 task unassigned activesession:probe-gate

Twinned from https://github.com/cfsh/ninefold/issues/471 by tools/beads/import_github.py's reconcile pass.

Unclaimed and unlabelled — inert backlog, per CLAUDE.md.

Measured

Same tree (9a380fe), no edits between runs:

``` probe_gate (python) -> 28 passed, 0 failed probe_gate (shell) -> 27 passed, 1 failed <- menu-focus

./probe.sh tools/probes/menu-focus.json run 1: PASS run 2: FAIL run 3: FAIL ```

Found while porting probe_gate.sh (#412 item 5 / design 001 piece 5), where the two implementations disagreed on the sweep result and the disagreement turned out to be the *probe*, not the port.

Why this matters more than one red probe

The UI gate's entire premise (#389) is that a NEW failure means "this merge regressed main". That inference is only as good as the probes' determinism. A flaky probe produces:

The second one is the quiet failure and matches the shape #389 exists to end: a gate that reports something reassuring while the thing it guards is broken.

What is not yet known
Options, uncosted

The middle option composes best with the gate as built: probe_gate already distinguishes NEW / STANDING / FIXED / GONE, and FLAKY is the missing fourth state that stops a coin-flip being recorded as a fact.

Dependencies

None.

Comments

No comments.

Add a comment