Repository navigation
test(coderd/x/chatd): pin the debug runs that turns, titles and status labels record - #30933
ibetitsmike wants to merge 5 commits into
Conversation
…s labels record Four call sites create and finalize debug runs by hand. Before they share one helper, pin what each records: every create parameter, the run context the model call sees, the finalize status and seed summary, and what happens when the insert fails. Title generation and status labels had no debug-run test at all; the chat turn test checked only a few fields.
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 584e4fb5d2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. 🎉 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Adds tests that pin what four call sites record when they create and finalize a debug run: chat turns, manual titles, automatic titles and turn status labels. #30934 moves all four onto one helper. Automatic titles and status labels had no debug-run test, and the chat turn and manual title tests checked only some fields.
TestRunnerDebugTurnEnsureCreatesOncenow checks every insert parameter, the fullRunContextthe model call sees, and that the insert runs on a context that ignores the caller's cancellation but has a deadline.TestRunnerDebugTurnEnsureCreateFailureDisablesTurn: a failed insert disables debug for the turn, and the turn does not retry it.TestProposeChatTitle_DebugRunalso checks that the run has no trigger message, its model config and model, the seed label in its summary, and that it recorded steps.TestQuickgenDebugRuncovers automatic titles and status labels when the model succeeds, when it fails, and when the insert fails. It checks the insert, the run context and the finalize call. Titles record the configured provider and model; status labels record the resolved ones.TestPrepareManualTitleDebugRunCreateFailure: when the manual title insert fails, the title call gets no run and finishing does nothing.Why it is safe: the PR changes only tests, and they pass on
main's production code. To show they can fail, I ran 34 mutants of the four call sites, and 31 fail the new tests. Two survivors change nothing these tests can observe: the seed summary on the manual title insert (finalize rewrites the summary) and the 10 second finalize timeout for titles and labels. This layer does not pin the third: removingcontext.WithoutCancelfrom the title and label insert inquickgen.go. #30934 moves that detach into the sharedBeginRun, which the chat turn test pins.Validation:
go test -race -count=1 -p 4 -parallel 4 ./coderd/x/chatd/ -run '^(TestRunnerDebugTurn.*|TestProposeChatTitle_DebugRun|TestQuickgenDebugRun|TestPrepareManualTitleDebugRunCreateFailure)$'passes on this branch and withmain's production code, and the pre-commit hook passed.Stack
This stack simplifies the chatdebug heartbeat, debug-run and stream wrapper code. Each refactor sits on a test-only layer that pins the behavior it keeps.