reviewer logs grow unbounded: 81 MB in logs/, ~16 MB/day, no rotation nf-jmzb ← Beads

open priority 2 task unassigned

Twinned from https://github.com/cfsh/ninefold/issues/483 by tools/beads/import_github.py's reconcile pass.

Filed unclaimed and unlabelled from ninefold-a. Backlog, for a future triage round.

Raised by aedanpope in chat after noticing dispatch.log while reviewing the design-001 stack:

> "file an issue for the log growth, but I think this would be fine to cap the max line-count of the file and use FILO for scripts that write to it"

That approach is right, and measuring it first changes which file it should be pointed at.

Measured on the sandbox box, 2026-08-04

| path | size | count | rate | |---|---|---|---| | ~/.ninefold-reviewer/dispatch.log | 88 KB | 1,078 lines | ~18 KB/day | | ~/.ninefold-reviewer/logs/pr-N.<stamp>.log | 81 MB | 260 files | ~16 MB/day |

Span: 2026-07-30 → 2026-08-04, five days, nothing pruned. Largest single transcript 2.1 MB (pr-332).

dispatch.log is not the problem. At 18 KB/day it reaches 1 MB in about two months. The 81 MB is logs/, which is ~900× larger and growing ~900× faster — one file per review, written by spawn() with --output-format stream-json --verbose, so each holds the reviewer's full transcript including every tool call.

Left alone this is ~6 GB/year on a box that also hosts the coder clones.

Two mechanisms, because they are two different shapes

1. dispatch.log — exactly aedanpope's suggestion. One append-only stream, so a line cap with oldest-line eviction fits directly. Cheap, and worth doing even though it is the smaller problem: it bounds the file for free and the writer already owns it.

⚠️ One trap, and it is the #461 class. poll.py reads this file's mtime as the liveness signal that distinguishes *"the dispatcher is dead"* from *"it is alive and mid-sweep"*. So:

Both are satisfied by "the script that writes it also caps it", which is what the suggestion says — worth writing down so a later logrotate-shaped fix does not quietly undo #462.

2. logs/ — the actual 81 MB, and a line cap does not apply. These are separate files, not one stream. The analogous move is a retention policy in spawn(): keep the newest N transcripts (or N days) and delete the rest, same FILO idea one level up.

Worth deciding what they are *for* first, since that sets N. Today they are forensic — the charter says the transcript exists so *"review behavior can be audited from the log"* — and in practice the useful window is the current review round, not five days. --output-format stream-json --verbose is also what makes them 2 MB each; whether that verbosity earns its cost is the same question and probably has the same answer.

Related: #482 mentions spawn buffering the transcript rather than streaming it to the file — same call site, so whoever picks this up should look at both together.

Dependencies

None.

Comments

No comments.

Add a comment