Skip to content

fix(coderd/x/chatd/chattool): make edit_files schema and errors actionable for models - #28121

Open
ibetitsmike wants to merge 2 commits into
mainfrom
mike/chatd-edit-files-tool-errors
Open

fix(coderd/x/chatd/chattool): make edit_files schema and errors actionable for models#28121
ibetitsmike wants to merge 2 commits into
mainfrom
mike/chatd-edit-files-tool-errors

Conversation

@ibetitsmike

Copy link
Copy Markdown
Collaborator

Problem

Chat 45b87e40-ffe7-49e5-8932-5fd0bdb9e542 on dev.coder.com failed 57 of 75 edit_files tool calls. Every failure was the same: the model omitted files[].path (it batched edits per file but only filled in edits), and the error relayed back to the model was:

POST http://[fd7a:115c:...]:4/api/v0/edit-files: unexpected status code 400: "path" is required

The model retried the identical malformed call dozens of times. Two gaps made this sticky:

  1. The edit_files input schema had no field descriptions, so path was only a bare required property.
  2. The agent API error reached the model wrapped in HTTP transport noise (method, internal tailnet URL, status code) with no indication of which files entry was broken.

Changes

  • Add description tags to every edit_files schema field and state the path requirement in the tool description.
  • Validate files entries in the tool before plan-turn checks and the workspace connection lookup, returning entry-indexed errors such as files[1].path is required; provide the absolute path of the file to edit; no files in this batch were applied.
  • Relay agent API failures with Message, Helper, Detail, and Validations from codersdk.Error instead of the raw transport-prefixed string.

Validation

  • go test ./coderd/x/chatd/chattool passes; new tests cover the schema description, entry-indexed validation errors, and transport-noise stripping (each verified red-green by toggling the fix off).
  • go build ./..., go vet, and pre-commit (fmt + lint) pass.

Mux created this PR on Mike's behalf.

…nable for models

A dev.coder.com chat failed 57 consecutive edit_files calls because the
model omitted files[].path and the relayed error was an opaque agent
API transport error. Describe every schema field, validate entries
before calling the agent with entry-indexed messages, and strip HTTP
method/URL/status noise from agent API errors.
…t_files

Validate entries before plan-turn checks and connection lookup, keep
Helper and Validations in relayed agent API errors, and deduplicate
the schema and malformed-input tests.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 25f1eb405e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@DanielleMaywood DanielleMaywood left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm slightly hesitant with some of the changes because Mathias spent a lot of time tweaking this but as long as we track the before & after I'm fine

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants