fix(coderd/x/chatd/chatadvisor): textualize advisor prompt tool exchanges - #27059
Merged
Merged
Conversation
…nges The nested advisor call defines no tools, but BuildAdvisorMessages forwarded raw tool_use/tool_result blocks from the parent conversation. Models mimic the forwarded pattern and spend the step committing to a tool call they cannot make, ending it with no text output. Fold each exchange into a plain-text user-role note instead, and include the finish reason and content-part kinds in the no-text-output error so the failure mode is diagnosable from field reports.
ibetitsmike
approved these changes
Jul 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes CODAGT-592.
Problem
The advisor tool sometimes fails with the opaque error
advisor produced no text output. Live reproduction againstclaude-sonnet-4-6showed the cause:BuildAdvisorMessagesforwards the parent conversation's rawtool_use/tool_resultblocks into the nested advisor call, which defines no tools. The nested model imitates the forwarded pattern and spends its turn committing to a tool call it cannot make (captured reasoning from a failing run: "The user wants me to make another tool call to the advisor about writing a poem about cucumbers."), so the step ends with reasoning-only or empty content and no advice. Because each chat step currently rebuilds the advisor runtime and snapshot (CODAGT-593), the second advisor call in a run reliably sees the first call's exchange, which is why the first call succeeds and later ones fail.Fix
BuildAdvisorMessagesrewrites tool activity as plain-text notes: assistant tool-call parts are removed and folded, together with their matching result, into a single user-role note of the form[The parent agent ran the X tool with input {...}. Result: ...]. No raw tool blocks and no bare call lines reach the tool-less nested request. This also removes the provider requirement thattool_resultblocks pair with atool_use, so results orphaned by window truncation are kept as notes instead of dropped.advisor produced no text outputerror now appends the finish reason and content-part kinds, e.g.advisor produced no text output (finish_reason=stop; parts: reasoning=1), so field reports distinguish tool-call mimicry, reasoning-only turns, and truncation.Validated live by driving the production
RunAdvisorpath againstclaude-sonnet-4-6through the dev.coder.com AI gateway: the failing scenario went from 3/3 errors to 6/6 genuine advice (with and without extended thinking), with the control scenario unaffected.Related: CODAGT-593 (per-step advisor runtime recreation, addressed separately) and CODAGT-742 (advisor tool call design).
Investigation and validation details
Reproduction
A CLI prototype constructed the exact conversation snapshot the generation preparer hands the advisor tool and called the real
chatadvisor.NewRuntime/Runtime.RunAdvisor/BuildAdvisorMessages/chatloop.GenerateAssistantchain against liveclaude-sonnet-4-6, with a stream-teeing model wrapper capturing whatrunner.godiscards (finish reason, part kinds, reasoning text).tool_use/tool_resultpair forwardedadvisor produced no text outputEvery failing response was a tiny thinking block, zero text, zero tool-call stream parts, finish reason
stop; the model's own reasoning text showed it deciding to "make the second tool call" in a request withtools=0. The refundedremaining_uses: 1200in the failing tool-result JSON matches the original issue screenshot.Decision log
[tool call: name(input)]text lines plus separate result messages; live runs then returned the literal[tool call: advisor(...)]line as the advice 6/6 times. The bare assistant call line is itself an imitable pattern, so no assistant-authored tool artifact may survive the handoff. The folded user-role note produced 6/6 genuine advice.dropOrphanToolMessageswas removed: without raw tool blocks there is no provider pairing constraint, and an orphaned result note retains context value.MaxOutputTokens) did not reproduce onclaude-sonnet-4-6; the model adapts thinking length to the cap. The enriched error would identify such cases on other models viafinish_reason=length.This PR was generated by Coder Agents on behalf of @ThomasK33 (Linear agent session for CODAGT-592).