Twinned from https://github.com/cfsh/ninefold/issues/541 by tools/beads/import_github.py's reconcile pass.
Filed unclaimed and unlabelled from ninefold-a at the close of design-001 pieces 6–11. Backlog, for a future triage round.
Sibling to #482, and deliberately not folded into it. #482 is about the mutation harness being rebuilt from scratch each time. This is about the characterization harness, which design 001 §5.5 step 4 *requires* be destroyed — and about the environment knowledge that dies with it.
§5.5 step 4: *"delete the .sh, delete the throwaway harness (it has done its job; keeping it means maintaining two harnesses)."*
That reasoning is right about the harness and wrong about what the harness learned. Five pieces built five independent characterization harnesses, and each one rediscovered the same Windows-specific traps — not by reading the previous one, because the previous one had been deleted:
| trap | pieces that hit it | symptom while it was undiagnosed |
|---|---|---|
| chmod +x silently no-ops on .bat here, so [ -x ] is always false | 9 | all 7 probe scenarios took the no-Godot path; the matrix looked fully populated and exercised one error branch seven times |
| CreateProcess appends .exe before PATHEXT, so Python cannot exec an extensionless script while bash can | 6, 7, 9, 10 | the differential ran the shell against a stub and Python against the real binary — "byte-identical" reported an agreement it had not tested |
| MSYS rewrites POSIX-looking env values when handing the environment to a native process | 9 | bash saw /nonexistent/godot.exe, Python saw C:/Program Files/Git/nonexistent/…; a fixture difference reported as a port difference |
| set -E propagates the harness's own ERR trap into the scenario subshell | 6, 8 | aborted at line N written into the stderr being measured — the harness contaminating its own golden |
| temp filenames / pids / commit shas differ between the two recording runs | 6, 8, 9, 10 | guaranteed false divergence on every scenario that names one |
Five traps, seventeen rediscoveries. Each cost a debugging cycle, and three of them (rows 1–3) produced a harness that passed while testing nothing — the failure mode this whole tier is supposed to be deleting.
Keeping five bespoke harnesses alive would mean maintaining test infrastructure for scripts that no longer exist. §5.5's argument holds. The mistake is treating "delete the instrument" and "discard the findings" as the same act — they are not, and only the first one was intended.
Cheapest first; they are not exclusive.
tools/tests/legacy/README.md (or a section in CLAUDE.md's testing notes) listing the five above with their symptoms. Costs nothing, survives every deletion, and each row is already written — they are in the commit messages of #492, #500, #509, #520 and #522.mkrepo, stub, assert_unchanged and a scrub helper, kept under tools/tests/. #414's harness already proposed exactly these three primitives and design 001 §5.5 adopted them "wholesale" — but as prose to re-implement, not as code to import. That is why five sessions wrote five versions.art_dump/sync.sh (120 lines) is explicitly migrate-on-touch, .cmd is now in scope, and #416/#417 could reopen this territory. The next porter starts from zero.A is the whole win for almost no cost. B is worth it only if another port is actually queued.
The specific traps are Windows-and-bash trivia. The transferable part is that §5.5 step 4 has a cost the doc does not price: a throwaway harness is the only artefact that ever *executed* the environment's edge cases, and the rule that destroys it destroys the only written record of what it found. Design 001's own amendment A2 records two places its prescriptions did not survive contact; this is a third-order version — a prescription that survives contact and quietly bills the next session for it.
Related: #482 (mutation-harness rebuilds), #450 (nothing verifies the verifier), #414 (the harness this pattern came from).
None.
No comments.