feat(opencode): support per-variant limit overrides#34815
Open
michaelofengend wants to merge 2 commits into
Open
feat(opencode): support per-variant limit overrides#34815michaelofengend wants to merge 2 commits into
michaelofengend wants to merge 2 commits into
Conversation
Model variants in provider config can now override limit (context/input/output). The override is applied once when the session resolves its model, so max output tokens and the compaction trigger follow the active variant, and the raw limit object no longer leaks into provider request options. Fixes anomalyco#34544
Contributor
There was a problem hiding this comment.
Pull request overview
Adds first-class support for per-variant token limit overrides in model configuration so that selecting a variant can change effective context/output limits without changing downstream call signatures.
Changes:
- Introduces
ProviderTransform.withVariantLimit(model, variantID)to merge a variant’s partiallimitover the base model limit. - Applies variant-resolved limits in session model resolution and in
LLMRequestPrep.prepare, and preventsvariant.limitfrom being merged into provider request options (avoids leakinglimitinto request bodies). - Extends the v1 config schema (and regenerated SDK schema/types) to include
variants.*.limit, and adds a regression test coveringmax_tokensoverride + nolimitleak.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| packages/sdk/openapi.json | Exposes variants.*.limit in the OpenAPI schema for config typing/SDK consumers. |
| packages/sdk/js/src/v2/gen/types.gen.ts | Regenerates SDK TS types to include variants.*.limit. |
| packages/opencode/test/session/llm.test.ts | Adds coverage ensuring variant limit.output drives max_tokens and limit doesn’t leak into the request body. |
| packages/opencode/src/session/prompt.ts | Resolves the session model through withVariantLimit(...) so downstream consumers see the variant-adjusted limits. |
| packages/opencode/src/session/llm/request.ts | Applies variant limit override when computing maxOutputTokens and strips limit from merged variant request options. |
| packages/opencode/src/provider/transform.ts | Implements withVariantLimit(...) helper used by session/request prep. |
| packages/core/src/v1/config/provider.ts | Updates v1 config schema so variant limit is validated/typed and appears in generated schemas. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This was referenced Jul 2, 2026
6 tasks
# Conflicts: # packages/opencode/test/session/llm.test.ts
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes #34544
Type of change
What does this PR do?
Variants in a model's config can now carry a
limitoverride, so the same model can have e.g. a 200k-context preset next to its full 1M default:Previously a variant's
limitdid nothing useful:maxOutputTokensand the compaction check always read the basemodel.limit, and the rawlimitobject was merged into provider request options along with the rest of the variant (so it could end up in the request body).The fix is a small
ProviderTransform.withVariantLimit(model, variantID)helper that returns the model with the active variant'slimitmerged over the base (partial — override justcontextif that's all you set). It's applied once where the session resolves its model inprompt.ts, so everything downstream (max output tokens viaLLMRequestPrep.prepare, the overflow/compaction trigger inoverflow.ts) follows the active variant without any signature changes.preparealso applies it itself so direct callers get the same behavior, and now stripslimitfrom the variant before merging request options. The variantlimitfield is typed in the v1 config schema so it shows up in the config JSON schema (SDK types regenerated).Not covered here: the context-% display in the TUI/web sidebar still reads the base model limit (it resolves models from the provider list, not the session), and the v2 runner has the same gap — happy to follow up on either if you want.
How did you verify your code works?
packages/opencode/test/session/llm.test.ts: a config-defined variant withlimit: { output: 4096 }producesmax_tokens: 4096in the outgoing request, and nolimitkey leaks into the request body.bun test test/session/llm.test.ts(27 pass),test/provider(421 pass), andprompt/compaction/processorsuites (123 pass) inpackages/opencode.bun typecheckclean inpackages/core;packages/opencodeshows only the pre-existingsrc/bus/global.tserror that's also present on a cleandevcheckout.Screenshots / recordings
N/A — no UI change.
Checklist