Skip to content

test(coderd/x/chatd): pin the debug runs that turns, titles and status labels record - #30933

Draft
ibetitsmike wants to merge 5 commits into
mike/chatdebug-stale-after-optionfrom
mike/chatdebug-debug-run-tests
Draft

ibetitsmike wants to merge 5 commits into
mike/chatdebug-stale-after-optionfrom
mike/chatdebug-debug-run-tests

Conversation

@ibetitsmike

@ibetitsmike ibetitsmike commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Adds tests that pin what four call sites record when they create and finalize a debug run: chat turns, manual titles, automatic titles and turn status labels. #30934 moves all four onto one helper. Automatic titles and status labels had no debug-run test, and the chat turn and manual title tests checked only some fields.

  • TestRunnerDebugTurnEnsureCreatesOnce now checks every insert parameter, the full RunContext the model call sees, and that the insert runs on a context that ignores the caller's cancellation but has a deadline.
  • New TestRunnerDebugTurnEnsureCreateFailureDisablesTurn: a failed insert disables debug for the turn, and the turn does not retry it.
  • TestProposeChatTitle_DebugRun also checks that the run has no trigger message, its model config and model, the seed label in its summary, and that it recorded steps.
  • New TestQuickgenDebugRun covers automatic titles and status labels when the model succeeds, when it fails, and when the insert fails. It checks the insert, the run context and the finalize call. Titles record the configured provider and model; status labels record the resolved ones.
  • New TestPrepareManualTitleDebugRunCreateFailure: when the manual title insert fails, the title call gets no run and finishing does nothing.

Why it is safe: the PR changes only tests, and they pass on main's production code. To show they can fail, I ran 34 mutants of the four call sites, and 31 fail the new tests. Two survivors change nothing these tests can observe: the seed summary on the manual title insert (finalize rewrites the summary) and the 10 second finalize timeout for titles and labels. This layer does not pin the third: removing context.WithoutCancel from the title and label insert in quickgen.go. #30934 moves that detach into the shared BeginRun, which the chat turn test pins.

Validation: go test -race -count=1 -p 4 -parallel 4 ./coderd/x/chatd/ -run '^(TestRunnerDebugTurn.*|TestProposeChatTitle_DebugRun|TestQuickgenDebugRun|TestPrepareManualTitleDebugRunCreateFailure)$' passes on this branch and with main's production code, and the pre-commit hook passed.

Stack

This stack simplifies the chatdebug heartbeat, debug-run and stream wrapper code. Each refactor sits on a test-only layer that pins the behavior it keeps.

  1. test(coderd/x/chatd/chatdebug): cover both debug heartbeat launchers #30931 Cover both debug heartbeat launchers
  2. refactor(coderd/x/chatd): set the chatdebug stale threshold at construction #30932 Set the chatdebug stale threshold at construction
  3. test(coderd/x/chatd): pin the debug runs that turns, titles and status labels record #30933 Pin the debug runs that turns, titles and status labels record
  4. refactor(coderd/x/chatd): create and finalize debug runs through one helper #30934 Create and finalize debug runs through one helper
  5. test(coderd/x/chatd/chatdebug): table the stream wrapper tests and cover their status rules #30980 Table the stream wrapper tests and cover their status rules
  6. refactor(coderd/x/chatd/chatdebug): share one finalize-once lifecycle between the stream wrappers #30981 Share one finalize-once lifecycle between the stream wrappers

Xum, an AI agent, opened this PR for @ibetitsmike and wrote this description.

…s labels record

Four call sites create and finalize debug runs by hand. Before they share
one helper, pin what each records: every create parameter, the run context
the model call sees, the finalize status and seed summary, and what happens
when the insert fails. Title generation and status labels had no debug-run
test at all; the chat turn test checked only a few fields.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-18T16:47:10.587592Z 9c018c1 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 584e4fb5d2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/quickgen_debug_internal_test.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 9c018c1b70

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant