Date: 2026-05-23 · Branch: architectural-improvements

A headless agent (Claude Opus, --permission-mode bypassPermissions) answers one canonical flow question per repo — twice: with the codegraph MCP server, and without any MCP (built-in Read/Grep/Glob/Bash only). Same model, same prompt; codegraph is the only variable. Each cell was re-indexed fresh first, so the "with" arm reflects the current resolvers.

Across 37 cells, codegraph cut total file reads from 158 → 40 — 75% fewer. It never increased reads in any cell. The mechanism: a few sub-millisecond codegraph calls replace a read-and-grep exploration. Token cost stays roughly flat (codegraph calls trade for reads) — the win is fewer tool calls + lower wall-clock, which is the design target.

The gap widens with repo size and flow complexity: on medium/large repos the without-codegraph arm often thrashes — many greps/globs, shell find/grep (Bash), and occasionally spawning a sub-agent — while the with-codegraph arm answers in 2–6 calls. On tiny repos (a handful of files) the two arms tie or codegraph is marginally slower (MCP/index overhead doesn't pay off when the whole flow fits in one or two files) — but reads still drop.

• R / G / Gl / B / Ag = Read / Grep / Glob / Bash / sub-agent (Task) tool calls.
• cg-calls = codegraph MCP calls in the "with" arm (the trade for reads/greps).
• dur = wall-clock seconds. files = indexed file count (the size proxy).
• reads saved = without-reads − with-reads.
• One run per arm (a snapshot — run-to-run variance is real; treat ±1–2 reads and ±10s as noise, look at the pattern across cells). 2-runs/arm headline numbers for several of these flows live in docs/design/dynamic-dispatch-coverage-playbook.md §7.

Totals (37 cells): with codegraph 40 reads / 21 greps, without 158 reads / 71 greps — 75% fewer reads, ~70% fewer greps. Codegraph never increased reads in any cell, and the without-arm additionally ran shell find/grep (Bash) and a sub-agent that the with-arm never needed. (74 agent runs, ~$29 total.)

• Biggest wins are medium/large backends with a real route→handler→service flow: excalidraw (0R vs 9R/7G), spring-halo (0R vs 9R + 8 Bash), spring-realworld (0R vs 8R), django-realworld (1R vs 8R), aspnet-jellyfin (4R vs 13R + 21 Bash + a spawned sub-agent), aspnet-eshop (0R vs 6R).
• Without codegraph, large repos make the agent thrash: it falls back to shell find/grep (Bash) and on jellyfin even spawned a sub-agent — exactly the behavior codegraph is meant to prevent. The with-arm answers those in 2–6 codegraph calls.
• Tie zone = tiny repos (Dart books 6 files, Kotlin Jetcaster, Ruby forem, Swift spi): the whole flow fits in 1–2 files, so reading is already cheap; codegraph ties on reads and is sometimes a few seconds slower (MCP + index overhead). This matches the design note that codegraph's value scales with repo size.
• Duration tracks reads on the big repos (jellyfin 61s vs 132s, spring-halo 75s vs 148s, excalidraw 53s vs 98s) and is noise on small ones.
• Some "with" cells still read 2–4 files (jellyfin, gitness, laravel-firefly, forem) — the residual is the documented frontier (anonymous handlers, deep service chains, dynamic finders); codegraph gets the agent to the right file, then it reads one to confirm a detail.

All 14 README frameworks and every flow-relevant language are validated (see the playbook). The sizes here are by indexed file count; a few languages lack a clean third size in the corpus (Dart/Kotlin = S/M, Scala/Luau = S only, C = L only, C++ = M only) — those cells are omitted rather than faked.

Driver + parser: /tmp/ab-matrix/run.sh (matrix of lang|size|repo|question) and /tmp/ab-matrix/parse-matrix.mjs. Each cell: rm -rf .codegraph && codegraph init -i, then scripts/agent-eval/run-all.sh <repo> "<question>" headless (with = codegraph-only MCP, without = empty MCP), parsed from the stream-json logs.