fix(core): recover v2 context overflow#31003
Open
kitlangton wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
V2 automatic compaction estimates request size before provider dispatch, but estimates and configured model limits can be wrong. A provider may still reject the real request after local preflight says it fits.
This PR recognizes provider context-overflow failures, performs one forced compaction, rebuilds the request from the completed checkpoint, and retries exactly once.
Invariant: one logical provider turn gets at most one overflow recovery.
Stage 1: Normalize provider overflow
Provider protocols report the same condition differently. The LLM package now adds an optional provider-neutral classification to existing error surfaces:
classification?: "context-overflow"The shared classifier covers the existing V1 provider patterns and is reused by:
InvalidRequestReasonOrdinary invalid requests retain their existing behavior and are not compacted.
Compression line: provider-specific detection lives below Session orchestration.
Stage 2: Recover before publishing a failed assistant
The runner defers only a first context-overflow event that arrives before durable assistant content. It then asks compaction to bypass the local estimate while preserving every real safety check.
Overflow-triggered compaction bypasses:
compaction.auto: false, because this path recovers an otherwise terminal provider failure.It still requires:
If compaction cannot complete, the original overflow is published normally and the identical request is not retried.
Stage 3: Bound the retry
The existing safe-boundary retry reloads projected history and reconstructs the request after compaction. An internal retry marker consumes the overflow budget while unrelated epoch or model-selection retries preserve it.
sequenceDiagram participant Attempt1 participant Checkpoint participant Attempt2 Attempt1->>Checkpoint: complete overflow compaction Checkpoint-->>Attempt2: rebuild from durable context alt second attempt succeeds Attempt2-->>Attempt1: completed answer else second attempt overflows Attempt2-->>Attempt1: durable terminal failure endThe second overflow is never compacted again. This prevents loops caused by undocumented provider limits, incorrect catalog limits, or a checkpoint that still cannot fit.
Invariant: successful forced compaction consumes the only overflow retry.
Stage 4: Preserve partial-output safety
Provider
step-startis now lazy: it does not create a durable assistant until text, reasoning, tool activity, or terminal completion requires one.sequenceDiagram participant Provider participant Publisher participant History Provider->>Publisher: step-start Note over Publisher: no durable assistant yet alt overflow before output Provider-->>Publisher: context overflow Publisher-->>History: no failed assistant persisted else text or tool activity begins Publisher->>History: start durable assistant Provider-->>Publisher: context overflow Publisher->>History: persist terminal failure, do not retry endOnce assistant output or tool activity is durable, recovery is disabled to avoid duplicate content or side effects.
Verification
git diff --checkScope
Included:
Not included: