Skip to content

ci(.github/workflows): report doc-check reviews that outlast the wait as running - #31707

Closed
nickvigilante wants to merge 4 commits into
mainfrom
vigilante/docs-1478-doc-check-stop-failing-the-workflow-when-a-review-outlasts
Closed

nickvigilante wants to merge 4 commits into
mainfrom
vigilante/docs-1478-doc-check-stop-failing-the-workflow-when-a-review-outlasts

Conversation

@nickvigilante

@nickvigilante nickvigilante commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Since doc-check was turned back on (Oct 30), 28 of the 45 runs that started a review failed.
Every one failed the same way: the job stopped waiting for the review chat after 40 minutes.
The chat keeps running in Coder after the job stops watching, so the red check reported a review that was still working.

This PR reports a review that outlasts the wait as still running, with a notice that names the chat, instead of failing the job.

Linear: DOCS-1478

Changes

  • The chat step sets continue-on-error: true.
  • A new last step, Fail unless the review is still running, runs when the chat step failed.
    It passes in one case only: the action reports a timeout and the chat status is running.
    It then writes a notice and a step summary with the chat ID.
    The summary says the review posts only if it has findings, and that the review workspace stays up until its autostop deadline.
  • Every other failure of the chat step still fails the job, with the error kind, last chat status, and chat ID.
    That covers a chat error, a spend limit, an API failure, missing outputs, and a timeout on a chat that is not running.
  • The comment on wait-timeout-seconds names both things the wait does and that it must stay below timeout-minutes.

A review that times out and later ends in an error now shows a green check and posts nothing, so only the chat shows that result.
Before this PR its check was red, but only because of the timeout, the same as a review that later finished clean.

Testing

  • make lint/actions/actionlint and make lint/actions/zizmor pass, with no zizmor findings on doc-check.yaml.

  • I ran the new step's script under bash with each status the action can report on a failure:

    chat-error-kind chat-status Result
    timeout running Exit 0, notice, summary with chat ID
    timeout waiting, requires_action, paused, or error Exit 1
    api_error running Exit 1
    spend_exceeded error Exit 1
    (empty) (empty) Exit 1, unknown

    The action's status schema has no interrupting, so an interrupting chat arrives as api_error.

  • I read coder/agents-chat-action at the pinned SHA. On a timeout it sets chat-error-kind, chat-status, and chat-id before it fails the step.
    It does not set workspace-id, so the workspace stop step skips a timed-out review, as it did before this PR.

  • The step names the chat by ID because chat-url is built from the DOC_CHECK_CODER_URL secret, and a recent run log prints it as ***/agents/<id>.

  • Not run end to end: that needs a real review longer than 40 minutes, which would post to a PR.

Decision log

Failure data (runs since Oct 30 17:54 UTC): 200 runs.
119 skipped every job (draft PRs, labels other than doc-check, and closed PRs), 34 succeeded (17 of those ran a review), 28 failed, 17 were cancelled, and 2 were still running or awaiting approval.
All 28 failures ended with Timed out after 2400s waiting for chat <id> to reach a terminal status.
Of the 10 timed-out chats I could trace, 7 were still running (some after 4.5 hours) and 3 had finished with a clean result after the job went red.
The 17 cancellations are pending runs that GitHub replaced with a newer run for the same PR, and this PR does not change them.

Why not wait: none: it would end the job in a minute, but two things depend on the wait.

  • The step that stops the review workspace runs only for a review that finishes within the wait, which was 17 of the 45 review runs.
    The other 28 already leave their workspace up until the coder-workflow-bot template's 24-hour autostop, before and after this PR.
    With wait: none, all 45 would, which is the overload ci(.github/workflows): stop doc-check workspaces after each review #29851 fixed.
  • While a run waits, it holds the per-PR concurrency group, so at most one more run queues behind it.
    Without the wait, every push would post another message to the PR's chat, and Coder queues messages to a busy chat, so reviews would pile up on the shared model.
    After a timeout, the next run can still queue one message on a chat that is still running; the wait limits this to one per 40 minutes per PR.

Why not a longer wait: GitHub-hosted jobs stop at 6 hours, some reviews have run longer than 4.5 hours, and a longer wait holds the concurrency queue longer.

Left as is: an action annotation for the timeout still appears on the run page, because the action calls core.setFailed.
The job and the PR check are green.
The doc-check job is not a required check (gh pr checks --required lists only required), so a green still-running review doesn't let anything merge that couldn't before.

Review rounds (coder-agents-review): accepted every in-scope finding in rounds 1 and 2.
Out-of-scope findings for a follow-up, not changed here:

  • The stop step never runs for a timed-out or errored chat, because the action sets workspace-id only on success.
  • The stop step's status list includes completed, which the server no longer returns.
  • The action's status schema lacks interrupting, so polling an interrupting chat ends as api_error.
  • The delete-workspace job on merge can delete a workspace that a still-running chat uses.
  • triage-via-chat-api.yaml prints a chat URL built from a secret, so it renders masked.
  • A review chat has no upper bound on runtime, so a review that loops holds its workspace and a share of the model for hours, and later runs queue behind it.
  • output.md says a later step reads the doc-check-trace block to set the commit status, but no step in doc-check.yaml reads it.
  • The action has no input to report a timeout on a running chat as a non-failure, so each caller needs continue-on-error and a gate step.

Tradeoff: the gate passes any chat that is running at 40 minutes, so a stuck review also shows green.
chat-status cannot tell a stuck review from a slow one; the red check before this PR fired for both.

Related: #31649 makes reviews more thorough and slower, so it raises the share of reviews that outlast the wait.
This PR is what keeps those from failing the job.

CI flakes
  • test-go-pg (windows-2022) failed on the first run with no failing test (DONE 3922 tests, 134 skipped, then make exit 1).
    This PR changes only a workflow file, and the job passed on rerun.
  • gen failed on d7f1ec2297 before any step ran, with "The self-hosted runner lost communication with the server".
    It passed on rerun.

Generated by Coder Agents on behalf of @nickvigilante.

… as running

Since doc-check was turned back on, 28 of 45 runs that started a review
failed, every one because the job stopped waiting for the review chat
after 40 minutes. The chat keeps running in Coder and posts its own
results, so the red check reported a review that was still working.

The chat step now may fail without failing the job, and a final step
fails the job unless the failure is a wait timeout on a chat that is
still running. The wait stays, so a review that finishes in time still
stops its workspace, and the per-PR concurrency group still caps each PR
at one running and one queued review.

Refs DOCS-1478
@linear-code

linear-code Bot commented Nov 3, 2026

Copy link
Copy Markdown

DOCS-1478

@nickvigilante

Copy link
Copy Markdown
Contributor Author

CI note: lint-actions failed on the first run before any step ran, with "The self-hosted runner lost communication with the server". It passed on rerun, along with actionlint and zizmor locally, so I read it as a runner flake rather than a lint finding on doc-check.yaml.


Generated by Coder Agents on behalf of @nickvigilante.

@nickvigilante

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

@coder-agents-review

coder-agents-review Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Chat: Review posted | View chat
Requested: 2026-11-03 20:01 UTC by @nickvigilante
Spend: $21.38 / $100.00

Review history
  • R1 (2026-11-03): 10 reviewers, 1 Nit, 3 P2, 4 P3, 4 P4, COMMENT. Review
  • R2 (2026-11-03): 11 reviewers, 1 Nit, 3 P2, 5 P3, 7 P4, COMMENT. Review
  • R3 (2026-11-03): 7 reviewers, 1 Nit, 3 P2, 5 P3, 8 P4, COMMENT. Review
  • R4 (2026-11-03): 4 reviewers, 1 Nit, 3 P2, 5 P3, 8 P4, APPROVE. Review

deep-review v0.13.0 | Round 4 | 731f0c6..d7f1ec2

Last posted: Round 4, 17 findings (3 P2, 5 P3, 8 P4, 1 Nit), APPROVE. Review

Finding inventory

Findings

# Sev Status Location Summary Round Reviewer Posted
CRF-1 P4 Author fixed (1025355) doc-check.yaml:513 pending branch can never match; server has no pending chat status R1 Netero Yes
CRF-2 P4 Author fixed (1025355) doc-check.yaml:530 ::error:: repeats the action's own error annotation R1 Netero Yes
CRF-3 Note Note (body) doc-check.yaml:512 Timeout on paused or requires_action fails the job R1 Netero No
CRF-4 Note Note (body) doc-check.yaml:502 New step has no automated test; no harness for workflow scripts R1 Netero No
CRF-5 P2 Author fixed (1025355) doc-check.yaml:514 Notice and summary promise PR results, but clean and late-errored reviews post nothing R1 Mafu-san P2, Mafuuu P3, Hisoka P3, Chopper P3, Leorio P3, Razor P3 Yes
CRF-6 P2 Author fixed (1025355) doc-check.yaml:514 Green timeout path hides that the workspace is not stopped; wait rationale applies to a minority of runs R1 Pariston P2, Mafu-san P3 Yes
CRF-7 P2 Author fixed (1025355) doc-check.yaml:465 wait-timeout-seconds comment says "only" workspace stop; drops concurrency reason and timeout-minutes coupling R1 Gon P2, Mafu-san P3, Leorio P3, Razor P4 Yes
CRF-8 P3 Author fixed (1025355) doc-check.yaml:442 continue-on-error comment narrows it to timeouts and misstates the last step's rule R1 Gon P2, Leorio P4 Yes
CRF-9 P3 Author fixed (1025355) doc-check.yaml:498 Step comment lists failure cases incompletely (poll api_error on a running chat, requires_action) R1 Gon P2, Razor P4 Yes
CRF-10 P4 Author fixed (1025355) doc-check.yaml:527 Newline comment does not name the workflow-command consequence R1 Gon P2 Yes
CRF-11 P4 Author fixed (1025355) doc-check.yaml:502 Step name "Report the review status" does not match a pass/fail gate that skips on success R1 Gon Yes
CRF-12 Nit Author fixed (1025355) doc-check.yaml:505 ERROR_KIND/ERROR_MESSAGE drop the chat- prefix the other env names keep R1 Gon Yes
CRF-13 P3 Author fixed (1025355) doc-check.yaml:530 Failure annotation omits chat status and chat link R1 Leorio Yes
CRF-14 P3 Author fixed (1025355) doc-check.yaml:521 Chat link is built from a secret URL, so the summary masks it R1 Chopper Yes
CRF-15 Note Note (body) doc-check.yaml:466 Wait limits but does not prevent queued follow-ups on a still-running chat R1 Pariston No
CRF-16 Note Note (body) PR description "doc-check is not a required check" unverified R1 Mafu-san No
CRF-17 Note Dropped by orchestrator (covered by CRF-5, CRF-8, CRF-9) doc-check.yaml:498 Same fact repeated in four comments R1 Gon No
CRF-18 Note Dropped by orchestrator (confirmation, asks no change) doc-check.yaml:503 Action outputs match the step's env R1 Melody No
CRF-19 Note Dropped by orchestrator (confirmation, asks no change) doc-check.yaml:512 Action failure-output claims verified at pinned SHA R1 Mafu-san, Razor No
CRF-20 Note Dropped by orchestrator (confirmation, asks no change) PR description Decision log quality R1 Leorio No
CRF-21 OOS Out of scope (body) doc-check.yaml:478 Stop step never runs on timeout or error: action sets workspace-id only on success R1 Netero, Melody, Mafuuu, Hisoka, Pariston No
CRF-22 OOS Out of scope (body) doc-check.yaml:482 Stop step list includes completed, which the server no longer returns R1 Netero No
CRF-23 OOS Out of scope (body) agents-chat-action src/codersdk.gen.ts:100 ChatStatusSchema lacks interrupting; polls fail as api_error R1 Netero, Melody No
CRF-24 OOS Out of scope (body) doc-check.yaml:56 delete-workspace on merge can delete a workspace a still-running chat uses R1 Hisoka No
CRF-25 OOS Out of scope (body) triage-via-chat-api.yaml:296 Summary prints a chat URL built from a secret, so it renders masked R1 Chopper No
CRF-26 P3 Author fixed (43a29e6) doc-check.yaml:475 "failures surface in the workflow run log" is false for a review that fails after the wait R2 Mafu-san P3, Leorio P4, Gon OOS Yes
CRF-27 P4 Author fixed (43a29e6) doc-check.yaml:466 Wait comment does not say why 2400 and not longer (longer wait holds the concurrency group longer) R2 Gon P3 Yes
CRF-28 P4 Author fixed (43a29e6) doc-check.yaml:529 "The review chat failed" is false for api_error/org_not_found/spend_exceeded; api_error status is the at-creation snapshot R2 Chopper P4, Leorio P4, Gon Nit Yes
CRF-29 P4 Author fixed (PR description edit) PR description Testing table claims every server status but omits paused and lists interrupting, which arrives as api_error R2 Mafu-san P3, Leorio P4 Yes
CRF-30 P2 Dropped by orchestrator (reverses the R1 CRF-8 wording the author applied; repetition within one step block) doc-check.yaml:442 continue-on-error comment repeats the gate's pass rule R2 Gon No
CRF-31 P2 Dropped by orchestrator (reverses the R1 CRF-9 request the author applied; repetition within one step block) doc-check.yaml:503 Gate step comment restates step name and condition R2 Gon No
CRF-32 P4 Dropped by orchestrator ("could not" is accurate: the job has no workspace ID on this path) doc-check.yaml:523 "could not stop" implies an attempted stop R2 Leorio No
CRF-33 Note Note (body) doc-check.yaml:516 A stuck review still running at 40 minutes now shows green R2 Pariston No
CRF-34 Note Dropped by orchestrator (confirmation, asks no change) doc-check.yaml:508 outcome vs conclusion; queued follow-up does not misread; notice text works; fixes match replies R2 Meruem, Leorio, Mafu-san No
CRF-35 OOS Out of scope (body) doc-check path (chat layer) No upper bound on review chat runtime R2 Pariston No
CRF-36 OOS Out of scope (body) .claude/skills/doc-check/output.md:127 Skill says a later step reads the trace block; no step does R2 Melody No
CRF-37 OOS Out of scope (body) agents-chat-action src/index.ts:49 Action has no input to report a running-chat timeout as non-failure R2 Meruem No
CRF-38 P4 Author fixed (d7f1ec2) doc-check.yaml:471 "A longer wait would not cover them" overstates; the PR data shows a longer wait covers some R3 Mafu-san P4, Leorio P4 Yes
CRF-39 Note Dropped by orchestrator (confirmation, asks no change) doc-check.yaml:529 Round 2 fixes match their threads and the description R3 Mafu-san, Leorio No
CRF-40 Note Dropped by orchestrator (confirmation, asks no change) doc-check.yaml:471 CRF-38 fix matches the decision log R4 Leorio, Mafuuu No

Contested and acknowledged

Round log

Round 1

Netero first pass: 2 P4, 2 Note, no P0, so the panel ran. Law not run (effective +42).
Round type: round 1, full panel. Focused change (one workflow file, YAML and bash), size 9 plus wildcard because the step writes action-controlled text into workflow commands (input handling). Specialists by trigger: Kurapika (workflow-command input, secrets), Melody (action outputs to step consumer pairing). Core: Pariston, Mafuuu. Pool: Hisoka, Chopper, Mafu-san (agent-authored), Gon, Leorio (once-per-PR floor). Wildcard: Razor (random from Meruem, Kite, Zoro, Knov, Razor).
Cross-check decisions:

  • CRF-5 set to P2 (Mafu-san) over five P3 ratings: a chat that errors after the wait now yields a green check and no comment, which contradicts the PR description's claim that a chat error still fails the job.
  • CRF-6 merges Pariston P2 (notice hides unstopped workspace) and Mafu-san P3 (decision log premise overstated); kept P2 because 28 of 45 runs leave a workspace up for a day and this PR turns those runs green. Root cause is CRF-21 (out of scope).
  • CRF-7 kept at P2 (Gon): the removed comment tied 2400s to timeout-minutes 50; raising the wait without the job cap makes the job cancelled, the outcome this PR removes.
  • CRF-8 and CRF-9 downgraded from Gon P2 to P3. Keep-argument: a maintainer who believes the comment could delete or weaken the gate step. Downgrade reason: the step name and script state the rule next to the comment; Leorio P4 and Razor P4 traced a smaller consequence.
  • CRF-10 downgraded from Gon P2 to P4. Keep-argument: removing the substitution would let a :: line in an error message run as a workflow command. Downgrade reason: the current comment already says workflow commands are read per line; CRF-2's fix removes lines 527-529.
  • CRF-14 is inferred from GitHub docs ("Summaries automatically mask any secrets") and the action's URL construction; not observed on a run.
  • Kurapika: no findings.
    Reviewed against 731f0c6..67bd9bc.

Round 2

Churn guard PROCEED: CRF-1, 2, 5 to 14 addressed in 1025355 (author claims; panel verified at head). Netero: no findings.
Round type: fix round (no files added, no exported signatures, only fixes to round 1 findings; +41 effective). Panel: personas that raised findings whose code changed (Pariston, Mafuuu, Hisoka, Chopper, Mafu-san, Gon, Leorio, Razor), specialists for the changed file (Kurapika, Melody), wildcard Meruem (random from Meruem, Kite, Zoro, Knov). Gon and Leorio also required by the floor (comments touched).
Fix verification: panel confirmed every round 1 fix at head; Kurapika, Razor, Hisoka, Mafuuu, Melody, Meruem no new findings.
Cross-check decisions:

  • CRF-26: Gon listed it out of scope; it is in scope because the PR changes the behavior the unchanged line 475 describes. P3 (Mafu-san) over Leorio P4. Mafu-san raised it in round 1 inside CRF-5, but the posted CRF-5 comment named only lines 465 and 498.
  • CRF-27 downgraded from Gon P3 to P4. Keep-argument: a maintainer who sees most runs time out raises both limits and holds each PR's queue for hours. Downgrade reason: the comment now says the wait holds the concurrency group, from which the cost of a longer wait follows, and the author applied exactly what CRF-7 asked.
  • CRF-28 merges Chopper P4, Leorio P4, and Gon's Nit on the same string. Verified in /tmp/agents-chat-action at 9b3ac42: the poll api_error is thrown with no chat object and pollWithContext fills chat-status from the at-creation snapshot (Chopper's reading; Leorio's "unknown" holds only when no snapshot exists).
  • CRF-29 settled P4 (Leorio) over Mafu-san P3. Keep-argument: the table is the PR's only test evidence and claims every status. Downgrade reason: the script's result for paused is correct (Mafu-san reproduced exit 1), so the gap is in the PR text, not the code.
  • CRF-30, CRF-31 dropped. Keep-argument: three copies of the pass rule can drift when the condition changes. Drop reason: round 1 asked for this wording, the author applied it, and asking to remove it now reverses the review; the copies sit within one 60-line block.
  • CRF-32 dropped. Keep-argument: a reader may look for a failed stop. Drop reason: the job lacks the workspace ID, so "could not" is literal.
    Reviewed against 731f0c6..1025355.

Round 3

Churn guard PROCEED: CRF-26, 27, 28 fixed in 43a29e6; CRF-29 fixed by a PR description edit (no thread reply). Netero: no findings; actionlint exit 0; zizmor could not run (GLIBC_2.39 missing).
Round type: fix round (comment and one string change, +9 -6 since round 2). Panel: Mafu-san, Leorio, Gon, Chopper (raised the round 2 findings), Kurapika and Melody (the changed ::error:: line prints action outputs into a workflow command), wildcard Zoro (random from Mafuuu, Meruem, Kite, Zoro, Knov, Razor). Gon and Leorio also required by the floor (comments touched).
Fix verification: all round 2 fixes confirmed at head by Netero, Chopper, Melody, Leorio, Mafu-san.
Cross-check decisions:

  • CRF-38 is text added by the CRF-27 fix. Mafu-san and Leorio both P4. Leorio's suggested wording says some reviews run longer than the 6-hour job limit; the decision log only supports "longer than 4.5 hours", so the posted fix uses Mafu-san's wording plus the 6-hour cap as a fact.
    Reviewed against 731f0c6..43a29e6.

Round 4

Churn guard PROCEED: CRF-38 fixed in d7f1ec2 (one comment sentence). Netero: no findings; actionlint exit 0.
Round type: fix round (one comment reworded). Panel: Mafu-san and Leorio (raised CRF-38), Gon (floor, comment touched), wildcard Mafuuu (random from Mafuuu, Meruem, Chopper, Kite, Zoro, Knov, Razor). All four: no findings.
No open in-scope findings remain. Out-of-scope items CRF-21 to CRF-25 and CRF-35 to CRF-37 still need a human to file or accept.
Reviewed against 731f0c6..d7f1ec2.

About deep-review

CRF = Coder Review Finding (P0-P4, Nit, Note)

Reviewer Focus
Bisky tests
Chopper ops/errors
Churn-guard change verification
Ging language modernization
Gon naming
Hisoka edge cases
Killua perf
Kite change integrity
Knov contracts
Knuckle SQL
Komugi flake/determinism
Kurapika security
Law decomposition
Leorio docs
Luffy product
Mafu-san process
Mafuuu contracts
Melody dispatch/pairing
Meruem structural
Nami frontend
Netero mechanical checks
Pariston premise testing
Pen-botter product gaps
Razor verification
Robin duplication
Ryosuke Go arch
Takumi concurrency
Zoro shape

🤖 Managed by Coder Agents.

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR makes a doc-check wait timeout on a running chat pass the job instead of failing it. Findings: 3 P2, 4 P3, 4 P4, 1 Nit.

Out of scope (needs a ticket or explicit acceptance by a human):

  • .github/workflows/doc-check.yaml:478: the action sets workspace-id only on success (setFailureOutputs in src/outputs.ts at the pinned SHA omits it), so the stop step never runs for a timed-out or errored chat, and those workspaces stay up until the coder-workflow-bot template's 24-hour autostop.
  • .github/workflows/doc-check.yaml:482: the stop step's status list includes completed, which the server no longer returns (migration 000543), so that entry never matches.
  • coder/agents-chat-action src/codersdk.gen.ts:100: ChatStatusSchema has no interrupting, so polling an interrupting chat fails schema parsing, and three such polls end the step as api_error, which fails the job.
  • .github/workflows/doc-check.yaml:56: after a timed-out run ends, the merge-time delete-workspace job can delete the workspace while the chat is still working in it, because the concurrency group no longer covers the chat.
  • .github/workflows/triage-via-chat-api.yaml:296: the step summary prints a chat URL built from secrets.TRAIAGE_CODER_URL, so the URL renders masked.

Notes:

  • A timeout on a chat in paused or requires_action fails the job. That is correct for requires_action; paused needs a human edit on the bot's chat, so it should not occur in normal runs.
  • The new step has no automated test, and the repository has no harness for workflow scripts; the PR's bash table is the only coverage.
  • The action reuses the PR's chat, so after a timed-out run ends the next push queues a follow-up on the still-running chat. The wait limits this to one queued message per 40 minutes per PR; it does not prevent it.
  • The decision log's statement that doc-check is not a required check could not be checked here (branch rules API returned 401).

🤖 This review was automatically generated with Coder Agents.

Comment thread .github/workflows/doc-check.yaml Resolved
Comment thread .github/workflows/doc-check.yaml Resolved
Comment thread .github/workflows/doc-check.yaml Resolved
Comment thread .github/workflows/doc-check.yaml Resolved
Comment thread .github/workflows/doc-check.yaml Resolved
Comment thread .github/workflows/doc-check.yaml Resolved
Comment thread .github/workflows/doc-check.yaml Resolved
Comment thread .github/workflows/doc-check.yaml Resolved
Comment thread .github/workflows/doc-check.yaml Resolved
Comment thread .github/workflows/doc-check.yaml Resolved
…step

Rename the step to "Fail unless the review is still running" and state
its single pass case in the comments. Drop the pending status, which the
server no longer returns. Name the chat by ID, because the chat URL is
built from a secret and prints masked. Stop repeating the action's error
message, and print the error kind, chat status, and chat ID instead. Say
that the review posts only when it has findings, and that the workspace
stays up until its autostop deadline. Restore the wait's coupling to
timeout-minutes in its comment.

Refs DOCS-1478
@nickvigilante

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review of the round 1 fixes in 1025355. Every round 1 finding is fixed at this head, and the gate script gives the exit codes in the testing table. Findings: 1 P3, 3 P4.

Out of scope (needs a ticket or explicit acceptance by a human):

  • doc-check review chats have no upper bound on runtime: a review that loops keeps its workspace and a share of the shared model for hours, and later runs queue messages behind it, so the PR gets no review in that time.
  • .claude/skills/doc-check/output.md:127: the skill says a later step reads the doc-check-trace block to set the commit status, but no step in doc-check.yaml reads it, so a review that ends with outcome: error after the wait never changes the check.
  • coder/agents-chat-action src/index.ts:49: the action calls core.setFailed on a timeout on a running chat and has no input to report that case as a non-failure, so each caller needs continue-on-error plus a gate script, and the run page keeps a failure annotation on a green job.

Notes:

  • The gate passes any chat that is running at 40 minutes, so a stuck review now shows a green check and the still-running notice. chat-status cannot tell a stuck review from a slow one, and the red check before this PR fired for both.

codersdk/chats.go:131

P4 [CRF-29] The PR description's testing table says it covers each status the server can return, but it omits paused and lists interrupting, which the action cannot report on a timeout. (Mafu-san, Leorio)

codersdk/chats.go:131 defines ChatStatusPaused, and the action's schema (src/codersdk.gen.ts:100-108 at the pinned SHA) accepts it and does not treat it as terminal, so a timeout on a paused chat reaches the gate step; the script exits 1 for it. The same schema has no interrupting, so an interrupting chat ends as api_error, not as timeout/interrupting. Add paused to the timeout exit-1 row and drop interrupting, or say that interrupting arrives as api_error.

🤖

🤖 This review was automatically generated with Coder Agents.

Comment thread .github/workflows/doc-check.yaml Resolved
Comment thread .github/workflows/doc-check.yaml Resolved
Comment thread .github/workflows/doc-check.yaml Resolved
Name the chat step, not the chat, in the failure annotation, and label
the status as the last one seen, because a polling failure does not mean
the chat failed. Give the reason the wait is not longer, and say that a
failure after the wait shows only in the chat.

Refs DOCS-1478
@nickvigilante

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

1 similar comment
@nickvigilante

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review of the round 2 fixes in 43a29e6 and the PR description edit. Every round 2 finding is fixed at this head. Findings: 1 P4, in text the wait-comment fix added.

🤖 This review was automatically generated with Coder Agents.

Comment thread .github/workflows/doc-check.yaml Resolved
@nickvigilante

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review of d7f1ec2, which rewords the wait comment. The longer-wait sentence is fixed, and no in-scope findings remain open. The out-of-scope items listed in the earlier reviews still need a human to file a ticket or accept them.

🤖 This review was automatically generated with Coder Agents.

@nickvigilante

Copy link
Copy Markdown
Contributor Author

Closing this. It assumes a review that outlasts the wait still posts its results later, and the production chats show most of them can't: 159 of 244 review turns since Oct 30 hit a GitHub 401, because the review workspace's installation token expires an hour after the workspace starts.
The real problem is load: doc-check keeps a median of 40 review chats active on the local model, and model steps slow from about 20 seconds to over 200 seconds once more than about 30 are active.
DOCS-1478 now tracks stopping that wasted load instead, in a new PR.


Generated by Coder Agents on behalf of @nickvigilante.

@github-actions github-actions Bot locked and limited conversation to collaborators Nov 3, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant