feat(opencode): respect provider/model streaming: false to disable response streaming#31357
Open
sebdanielsson wants to merge 2 commits into
Open
feat(opencode): respect provider/model streaming: false to disable response streaming#31357sebdanielsson wants to merge 2 commits into
streaming: false to disable response streaming#31357sebdanielsson wants to merge 2 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR adds a configuration-based way to disable true provider streaming while keeping the rest of the pipeline streaming-compatible by simulating a stream.
Changes:
- Add
simulateStreamingMiddleware()to opt out of on-the-wire streaming whenoptions.streaming === false. - Add a test asserting that provider requests are not made with
stream: truewhen streaming is disabled.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| packages/opencode/src/session/llm.ts | Adds streaming opt-out logic and conditionally injects simulated streaming middleware. |
| packages/opencode/test/session/llm.test.ts | Adds coverage to confirm provider requests are non-streaming when configured. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
…ponse streaming Some OpenAI-compatible backends don't support streaming or return broken streamed output (e.g. self-hosted vLLM corrupting streamed tool-call args). The existing options.streaming config wasn't consumed, so there was no way to opt out. Honor options.streaming:false (per-model or per-provider) by adding the AI SDK's simulateStreamingMiddleware, which calls doGenerate (stream:false on the wire) and replays the result as a simulated stream, leaving the rest of the pipeline unchanged. Defaults to streaming on. Fixes anomalyco#785 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… stream values Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
0da9544 to
fda049c
Compare
Author
|
Just this version in a GitHub Action workflow, and it worked without the stream-stripping proxy we're currently using to get around this limitation. 👍 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes #785
Type of change
What does this PR do?
Some OpenAI-compatible backends either don't support streaming or return broken streamed output. In my case a self-hosted vLLM (Gemma) corrupts streamed tool-call arguments (duplicates characters), so every edit comes back garbled. The existing
options.streamingconfig wasn't actually consumed, so there was no way to opt out.This makes
options.streaming: false(per-model or per-provider) actually work. When set, it adds the AI SDK'ssimulateStreamingMiddleware, which callsdoGenerate(stream: falseon the wire) and replays the result as a simulated stream — so the rest of the pipeline is unchanged. Defaults to streaming on, so existing behavior is untouched.{ "provider": { "vllm": { "options": { "streaming": false } } } }Only covers the default AI SDK path. The experimental
experimentalNativeLlmruntime is a separate path and isn't handled here.How did you verify your code works?
test/session/llm.test.tsthat setsstreaming: falseand has the mock server return a non-streaming JSON completion. It only parses if the request was non-streaming (a streamed request expects SSE and fails), and assertsbody.streamisn'ttrue.bun test test/session/llm.test.ts-> 27 pass, plus typecheck and oxlint clean.Screenshots / recordings
No UI changes.
Checklist