Skip to content

feat(opencode): cache-friendly compaction via primary loop request path - #42506

Open
rekram1-node wants to merge 1 commit into
devfrom
cache-friendly-compaction
Open

feat(opencode): cache-friendly compaction via primary loop request path#42506
rekram1-node wants to merge 1 commit into
devfrom
cache-friendly-compaction

Conversation

@rekram1-node

Copy link
Copy Markdown
Collaborator

Summary

Default compaction currently sends a request that shares no prefix with the conversation it is compacting: no system prompt, no tool definitions, and the history flattened into a serialized [User]: ... transcript inside a single user message. Every compaction pays full prompt-processing cost for the entire session at exactly the moment the context is largest, and providers cannot reuse any of the warm prompt/KV cache from the preceding turn.

This PR makes default compaction a prefix-extension of the previous request instead:

before                                after
------                                -----
(no system)                           same system prompt + instructions + skills + MCP context
(no tools)                            same tool definitions
[User]: ... flattened transcript      same typed message history
summary instructions up front         summary instruction appended as final user message

Everything before the trailing instruction is byte-identical to the prior turn's request, so the provider reuses the cached prefix and only the instruction plus the summary output are uncached.

Changes

  • Compaction agent is no longer registered by default. Default compaction inherits the session's last active agent and the model recorded on the user message. An explicit agent.compaction config seeds the old hidden deny-all agent and preserves the legacy behavior exactly (dedicated agent, optional model override, serialized-transcript summary request), so existing customizations are unaffected.
  • Request assembly stays in the primary loop. SessionPrompt gains a single shared prepare helper (tools + system + model messages) used by both normal turns and compaction. SessionCompaction.process accepts an optional run callback; when present (and no custom compaction agent is configured), the loop executes the summary request through the shared envelope. SessionCompaction keeps owning selection, prompt construction, and persistence — no new service dependencies.
  • New buildReplayPrompt in core builds the trailing instruction for the replayed-history path, including prior-summary merge instructions for repeated compactions.
  • Plugin hooks (experimental.session.compacting, experimental.chat.messages.transform) run unconditionally at their original sites.

Not in scope (follow-ups)

  • Preventing tool invocation during the summary request (tools must stay in the request for prefix identity; toolChoice: "none" is not usable because some providers omit the definitions entirely).
  • The post-compaction request still diverges at the checkpoint message — that is inherent to history replacement.

Tests

  • New wire-level test asserts the compaction request carries identical system + tools to the normal turn, replays typed history (no [User]: flattening), and ends with the summary instruction.
  • New tests pin agent inheritance (including that an inherited agent's configured default model does not override the session model) and exact legacy behavior for a configured agent.compaction.
  • Full compaction.test.ts, prompt.test.ts, and agent.test.ts suites pass; repo-wide typecheck passes.

Default compaction no longer uses a dedicated compaction agent. It inherits
the session's active agent and delegates request assembly to the primary
loop's shared envelope (same system prompt, instructions, skills, MCP
context, and tool definitions), replaying the typed message head and
appending the summary instruction as the final user message. This keeps the
compaction request prefix-identical to the preceding turn so provider
prompt/KV caches are reused instead of fully re-prefilled.

An explicitly configured agent.compaction preserves the legacy behavior
exactly: dedicated hidden agent, optional model override, and the
serialized-transcript summary request.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant