Skip to content

feat(opencode): support per-variant limit overrides#34815

Open
michaelofengend wants to merge 2 commits into
anomalyco:devfrom
michaelofengend:variant-limit-override
Open

feat(opencode): support per-variant limit overrides#34815
michaelofengend wants to merge 2 commits into
anomalyco:devfrom
michaelofengend:variant-limit-override

Conversation

@michaelofengend

Copy link
Copy Markdown

Issue for this PR

Closes #34544

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Variants in a model's config can now carry a limit override, so the same model can have e.g. a 200k-context preset next to its full 1M default:

"glm-5.2": {
  "limit": { "context": 1000000, "output": 131072 },
  "variants": { "200k": { "limit": { "context": 200000 } } }
}

Previously a variant's limit did nothing useful: maxOutputTokens and the compaction check always read the base model.limit, and the raw limit object was merged into provider request options along with the rest of the variant (so it could end up in the request body).

The fix is a small ProviderTransform.withVariantLimit(model, variantID) helper that returns the model with the active variant's limit merged over the base (partial — override just context if that's all you set). It's applied once where the session resolves its model in prompt.ts, so everything downstream (max output tokens via LLMRequestPrep.prepare, the overflow/compaction trigger in overflow.ts) follows the active variant without any signature changes. prepare also applies it itself so direct callers get the same behavior, and now strips limit from the variant before merging request options. The variant limit field is typed in the v1 config schema so it shows up in the config JSON schema (SDK types regenerated).

Not covered here: the context-% display in the TUI/web sidebar still reads the base model limit (it resolves models from the provider list, not the session), and the v2 runner has the same gap — happy to follow up on either if you want.

How did you verify your code works?

  • New test in packages/opencode/test/session/llm.test.ts: a config-defined variant with limit: { output: 4096 } produces max_tokens: 4096 in the outgoing request, and no limit key leaks into the request body.
  • bun test test/session/llm.test.ts (27 pass), test/provider (421 pass), and prompt/compaction/processor suites (123 pass) in packages/opencode.
  • bun typecheck clean in packages/core; packages/opencode shows only the pre-existing src/bus/global.ts error that's also present on a clean dev checkout.

Screenshots / recordings

N/A — no UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Model variants in provider config can now override limit (context/input/output). The override is applied once when the session resolves its model, so max output tokens and the compaction trigger follow the active variant, and the raw limit object no longer leaks into provider request options.

Fixes anomalyco#34544
Copilot AI review requested due to automatic review settings July 2, 2026 00:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class support for per-variant token limit overrides in model configuration so that selecting a variant can change effective context/output limits without changing downstream call signatures.

Changes:

  • Introduces ProviderTransform.withVariantLimit(model, variantID) to merge a variant’s partial limit over the base model limit.
  • Applies variant-resolved limits in session model resolution and in LLMRequestPrep.prepare, and prevents variant.limit from being merged into provider request options (avoids leaking limit into request bodies).
  • Extends the v1 config schema (and regenerated SDK schema/types) to include variants.*.limit, and adds a regression test covering max_tokens override + no limit leak.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
packages/sdk/openapi.json Exposes variants.*.limit in the OpenAPI schema for config typing/SDK consumers.
packages/sdk/js/src/v2/gen/types.gen.ts Regenerates SDK TS types to include variants.*.limit.
packages/opencode/test/session/llm.test.ts Adds coverage ensuring variant limit.output drives max_tokens and limit doesn’t leak into the request body.
packages/opencode/src/session/prompt.ts Resolves the session model through withVariantLimit(...) so downstream consumers see the variant-adjusted limits.
packages/opencode/src/session/llm/request.ts Applies variant limit override when computing maxOutputTokens and strips limit from merged variant request options.
packages/opencode/src/provider/transform.ts Implements withVariantLimit(...) helper used by session/request prep.
packages/core/src/v1/config/provider.ts Updates v1 config schema so variant limit is validated/typed and appears in generated schemas.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

# Conflicts:
#	packages/opencode/test/session/llm.test.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Variants of models config does not allow the change of the context size.

2 participants