perf: window drift no longer forces a full rebuild (#99) - #113

Follow-up to #109, #111 and #112, from the same profiling session: with those merged, the extension host was still pegged, and the incremental loader still re-read the whole corpus on almost every refresh.

What happens

On a long history the rolling window is always drifting past something. Some file's oldest event has just left it, so either the file arrives as a 'cutoff' rebuild, or its stored aggregate is rebased in place. The analysis fast path demanded "appends and nothing else", so either case rejected it and the refresh fell back to re-reading every body.

Measured on a 650-file, 1.4 GB history with several agents appending concurrently — the extension's own diagnostics:

files(discovered=650 changed=1 reused=649)
io(bytes=1 406 040 801 lines=452 447)  incremental(bodies=650)
ms(read-parse=49 596  total=49 797)

One file changed, 650 bodies read, ~50 s of parse time — repeated on every watch tick, about once a minute. 136 of 140 refreshes in one session took that path; the fast path was effectively unreachable on this machine.

Why this is safe

Neither case invalidates the other files:

• A 'cutoff' rebuild is assigned only when the body on disk is unchanged (bodyUnchanged, where the reason is chosen) — the file is re-read solely to recompute its aggregate under the new cutoff. It therefore owns exactly the UUIDs it owned before and cannot preempt a later owner.
• A rebase is the same story without the read: it only narrows an aggregate that is already stored, and the UUIDs it drops are handed to the ownership pass below, which re-plans precisely those files that could claim them.

What genuinely invalidates every stored contribution is a changed analysis context — no prior runtime, a different time zone, a different window length — and that still forces the rebuild, as do deletions, moves and a cutoff moving backward.

The guards that matter keep their tests, and I confirmed they still fail a broader version of this change: a rebuilt earlier file preempting a later content UUID owner, and an append that gives a zero-timestamp file its first timestamp (which reorders files, and with them ownership). Both still fall back to the full rebuild.

Test

the window drifting past an old event keeps the refresh incremental — one file's oldest event leaves the window while another session appends. It fails on main with "window drift re-read the whole corpus: 32 bodies" and passes here, and it asserts contentAnalysis equals what the full loader produces, so the faster path is checked for agreement, not just for speed.

Effect in practice

Same machine and history, before and after, counted from the extension's log:

Full test run is unchanged against main on my machine: the same 8 pre-existing failures (Windows symlink and file-locking tests), plus the new test.

There is a second, narrower cause of the remaining full rebuilds — a file that is appended to and has an event leave the window in the same refresh is planned as rebuild:source, which still rejects the fast path. That one needs care around UUID ownership for the appended lines, so I left it out of this PR rather than widen the change.

🤖 Generated with Claude Code