Skip to content

feat: limit concurrent chat agents with pooled admission - #27902

Open
ibetitsmike wants to merge 24 commits into
mainfrom
mike/chatd-pool-admission
Open

feat: limit concurrent chat agents with pooled admission#27902
ibetitsmike wants to merge 24 commits into
mainfrom
mike/chatd-pool-admission

Conversation

@ibetitsmike

@ibetitsmike ibetitsmike commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

Caps how many chat agents can generate concurrently on unlicensed deployments: at most 5 root chats and 10 subagent chats, enforced as two independent deployment-wide pools. Deployments with remaining licensed agent runtime hours (agent_runtime_hours) are uncapped.

Problem

Nothing bounds concurrent chat generation today. A community deployment can start unlimited simultaneous agents, and the cost of that generation has no license-tied limit.

How it works

  • Admission happens when a worker acquires chat ownership, inside the per-chat claim transaction, before any runner exists. The policy lives in enterprise/coderd/x/chatd; OSS carries only an AgentCapacityLimiter seam (Admit plus Limits), with a noop limiter that admits everything when no factory is configured (AGPL builds stay uncapped).
  • A chat counts against its pool while it is unarchived, running/interrupting, owned, and its runner heartbeat is fresh. Slots held by crashed replicas free automatically once heartbeats go stale. requires_action chats keep their runner but do not count.
  • Queued-for-capacity state is derived, not stored: a chat is queued when it is running, unarchived, unowned, and its pool is full. The single-chat GET computes it on read; there is no database marker to keep in sync. Refused chats stay running but unowned: no runner, no parked goroutine, no column.
  • Each worker keeps an in-memory map of chats it refused. The map drives capacity_change watch events (banner shows and clears live) and the wait-time histogram. It is a per-replica approximation; API reads always derive from the database.
  • Parents blocked in wait_agent keep their root slot; children draw from the separate subagent pool. Subagents cannot nest, so the subagent pool cannot deadlock on itself.
  • Interrupting chats bypass the cap and sort first in acquisition, so an over-cap user can always stop a chat.
  • Pool counts serialize across replicas with an advisory lock held through the acquisition transaction; the entitlement check short-circuits before the lock, so uncapped deployments never take it. Slot release rides the runner's ownership nudge, with the periodic acquisition tick as fallback.
  • Metrics: coderd_chatd_agents_active and coderd_chatd_agents_queued_for_capacity (per pool, database-derived), plus the coderd_chatd_agent_capacity_wait_seconds histogram.

Accepted approximations

  • FIFO is keyed on updated_at, so sending a message to a queued chat re-queues it at the back.
  • The wait histogram is per-replica approximate; the exactly-once queue-entry counter from earlier revisions is dropped (its information lives in the histogram count).
  • A lost capacity_change pubsub event leaves a stale banner until the next single-chat refetch.

Validation

  • Unit/integration tests: pool caps, independence, interrupt and requires_action bypass, capacity-neutral takeover, stale heartbeat release, licensed bypass matrix, derived-predicate query (pool full vs not full), FIFO by updated_at including the message re-queue caveat, queued-event dedup, queue pruning on archive, interrupt claim under a full pool, worker paging past refused candidates, and a concurrent-admission race test. New guards verified red-green.
  • Remote dogfood UAT on a dev deployment (unlicensed): 6 concurrent root chats produced exactly 5 running and 1 queued with the banner; freeing a slot cleared the banner live in ~1s without a reload; a queued chat stopped in ~0.6s despite the full pool; FIFO order held; 3 subagents progressed while the root pool was saturated.

Mux created this PR on Mike's behalf.

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

Docs preview

Check off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here.

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@coder-tasks

coder-tasks Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Documentation Check

New Documentation Needed

  • docs/ai-coder/agents/platform-controls/index.md - Document the new license-gated concurrent-agent capacity limit. Unlicensed/Community deployments are capped at 5 root chats and 10 subagent chats generating at once (two independent deployment-wide pools); deployments with remaining licensed agent_runtime_hours are uncapped. Explain the queued-for-capacity behavior (over-cap chats stay running but queue FIFO and surface a banner until a slot frees) and how to lift the cap (obtain a license / runtime hours). A subsection near Spend management is the natural home, since that section already documents license-gated Coder Agents limits.
    • Addressed in f479526: added a Concurrent agents section covering the 5-agent Community limit, the separate 10-slot subtask pool, the queued banner, and Premium Agent Hours removing the concurrency limit.
  • docs/ai-coder/agents/getting-started.md - Short pointer for Community users so they understand why agents may queue.
    • Addressed in f479526: added a Plan for concurrency limits subsection linking to the new Concurrent agents section.

Note

The auto-generated doc changes in this PR (docs/admin/integrations/prometheus.md, docs/reference/api/chats.md, docs/reference/api/schemas.md) look correct.


Automated review via Coder Agents

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 488948e1e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/queries/chats.sql Outdated

@DanielleMaywood DanielleMaywood left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will perform a more thorough review of the backend tomorrow when I'm more awake but the frontend changes are fine with me

Comment thread coderd/database/dbauthz/dbauthz.go Outdated
Comment thread coderd/database/dbauthz/dbauthz.go Outdated
Comment thread coderd/x/chatd/runner.go Outdated
Comment thread coderd/x/chatd/options.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1eac94c228

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/queries/chats.sql
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8959eba6c9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/worker.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 05375b8e7d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread enterprise/coderd/x/chatd/agentadmission.go
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2b5a180a2b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread enterprise/coderd/x/chatd/agentadmission.go
Comment thread site/src/api/queries/chats.ts Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Mux requested this review on Mike's behalf.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 21e7706a55

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread site/src/api/queries/chats.ts Outdated
Comment thread enterprise/coderd/x/chatd/agentadmission.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Mux requested this review on Mike's behalf.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 6b109bfc94

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@@ -0,0 +1,68 @@
ALTER TABLE chats
ADD COLUMN capacity_queued_at TIMESTAMPTZ;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the reason for adding this column is only to be able to show "chat is queued" in the UI, then there is a simpler approach. Assuming max 10 concurrent root chats, you can determine if a root chat is queued because of no capacity by checking that:

  1. The chat is in a runnable state (see ARCHITECTURE.md, it's defined there)
  2. The chat has no owner (either worker_id is NULL or it's not but there's no current heartbeat)
  3. There are 10 root chats with owners (worker_id not NULL and non-expired heartbeat).

That's one SQL query with a join.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only gotcha there is you can't distinguish between "queued because you reached your limit" and "queued because a worker hasn't picked it up yet". Adding some kind of delay that is a multiple of ChatAcquireInterval would help.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can distinguish between those 2. In the first case there are 10 or more claimed and running root chats, in the latter there are fewer.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦 even simpler!

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adopted. The column is gone entirely: no migration for it, no view change, no mark/clear queries. Queued state is now derived exactly as you describe, one SQL query (GetChatQueuedForCapacity) checking runnable + unowned (no live owner heartbeat) + pool full, where pool fullness distinguishes a capacity wait from an ordinary not-yet-picked-up chat (resolving Cian's gotcha per your follow-up). The single-chat GET computes it on read; list responses skip it.

Mux replied on Mike's behalf.

Comment thread coderd/x/chatd/worker.go
}
newRows := 0
progressed := false
for _, row := range rows {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is racy unless you synchronize all acquires on a single lock (e.g. an advisory db lock). I'd recommend only running the gatekeeping logic if the deployment doesn't have a valid license. Otherwise the lock will become a bottleneck for deployments with a high chat volume.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1; the enterprise admitter should be a no-op essentially.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partly adopted. refusedPools is now write-free (it only skips re-admission attempts within one pass), so the worst case of the race is a one-tick (~1s) admission delay corrected by the next acquisition pass.

On the lock: the entitlement check short-circuits before AcquireLock, so licensed deployments with remaining agent runtime hours never take it. Only capped deployments reach the lock, and those are bounded at 5+10 concurrent agents by construction, so the serialized section can't see high volume. Gating on "no valid license" alone would change the PRD policy (CODAGT-845): licensed-without-agent-hours deployments are meant to behave as community, so the gate keys on the agent_runtime_hours feature rather than license presence.

Mux replied on Mike's behalf.

Comment thread coderd/x/chatd/worker.go Outdated
if row.Status == database.ChatStatusRunning && refusedPools[row.ParentChatID.Valid] {
if !row.CapacityQueuedAt.Valid {
progressed = true
w.markCapacityQueued(ctx, row.ID)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd advise against manually keeping capacity_queued_at up to date with MarkChatCapacityQueued andClearChatCapacityQueued. It seems prone to getting out of sync because of worker crashes etc. This PR would be simpler and the code more straightforward without capacity_queued_at.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adopted. capacity_queued_at, MarkChatCapacityQueued, ClearChatCapacityQueued, and every clear site (UpdateChatExecutionState CASE, archive, auto-archive) are deleted. Queued state is derived from ownership + status + pool fullness, so there is nothing to desync across worker crashes. The only remaining worker-local state is an in-memory map of refused chats that drives capacity_change events and the wait histogram; it is advisory only, and API reads always derive from the database.

Mux replied on Mike's behalf.

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 8d05987999

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/worker.go Outdated
return
}
if w.opts.AgentAdmission != nil {
w.opts.AgentAdmission.RecordQueued()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this not be done inside the AgentAdmission interface?
Then you get a nice single-method Admitter interface.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, and further: RecordQueued is deleted outright (the exactly-once queue-entry counter needed the persisted marker to be exact; its information now lives in the wait histogram's count). AgentAdmission is the single-method Admit interface. Queue metrics moved to the OSS worker, which is the component that actually observes refusals and admissions.

Mux replied on Mike's behalf.

Comment thread coderd/x/chatd/worker.go
}
newRows := 0
progressed := false
for _, row := range rows {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1; the enterprise admitter should be a no-op essentially.

Comment thread coderd/x/chatd/agentadmission.go Outdated
}

// AgentAdmissionFactory builds an admission gate. A nil factory leaves capacity uncapped.
type AgentAdmissionFactory func(AgentAdmissionOptions) AgentAdmission

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This almost looks like functional options. The non-negotiables are LifetimeCtx and Store. Registerer, Logger, and HeartbeatStaleSeconds can get sensible default values.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved by dissolution: with metrics moved to the OSS worker, the enterprise implementation no longer needs Store, Logger, Clock, Registerer, or LifetimeCtx. The options struct is gone; the factory is now NewAgentAdmissionFactory(entitlements) and receives only heartbeatStaleSeconds from chatd at construction.

Mux replied on Mike's behalf.

type AgentAdmission interface {
// Admit reports whether the worker may acquire the chat. Refused chats
// remain unowned and are retried from the capacity queue.
Admit(ctx context.Context, store database.Store, chat database.Chat) (bool, error)

@johnstcn johnstcn Aug 6, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we drop the database.Store from the Admit() signature here? The AgentAdmission should already have one available to use from AgentAdmissionOptions. Thinking down the line when we carve out a dRPC interface.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kept deliberately, with a comment documenting why: the store parameter is the acquisition transaction. AcquireLock is pg_advisory_xact_lock (transaction-scoped), so the count + lock must run inside the same transaction as the ownership write for the serialization to hold until commit; an admission gate holding its own store reopens the double-admit race. For a future dRPC carve-out the seam would need to move the whole claim (admit + acquire) behind the interface rather than just the check.

Mux replied on Mike's behalf.

@@ -0,0 +1,68 @@
ALTER TABLE chats
ADD COLUMN capacity_queued_at TIMESTAMPTZ;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only gotcha there is you can't distinguish between "queued because you reached your limit" and "queued because a worker hasn't picked it up yet". Adding some kind of delay that is a multiple of ChatAcquireInterval would help.

Add a two-pool concurrency cap for chatd agents, enforced when a worker
acquires chat ownership: at most 5 root chats and 10 subagent chats
generate concurrently on unlicensed deployments. Refused chats stay
running but unowned, stamped with capacity_queued_at for FIFO admission
and a queued banner in the UI. Deployments with remaining licensed
agent runtime hours are uncapped.

The admission policy lives in enterprise/coderd/x/chatd; OSS carries
only the AgentAdmission seam invoked inside the acquisition
transaction. Active use is counted from ownership plus fresh
chat_heartbeats, so slots self-free when a replica crashes or a chat
leaves running/interrupting. Interrupting chats bypass the cap so
over-cap users can stop their own chats.
…dmission

Each guard was verified red-green: dropping the capacity_queued_at
ORDER BY key fails the FIFO test, refusing interrupting chats fails the
interrupt claim test, and removing the advisory lock over-admits in the
concurrency test.
… query

Sort interrupting chats ahead of the capacity queue so a full-pool
backlog cannot delay a stop request, make the exclude_ids predicate
nil-safe, and query the capacity callout story by its alert role.
…ests

A full root pool with a deep queued backlog previously buried subagent
candidates behind it: acquisition retried every queued root before
reaching an admittable subagent. Candidates now interleave across the
two pools with FIFO order inside each, and one refusal marks a pool
full for the rest of the pass so later chats in it are queue-marked
without an acquisition transaction.

Also adds the missing dbauthz method-suite tests for the four new
capacity queries and renames capacityCounted to occupiesCapacitySlot
per review.
The acquisition query interleaves the two capacity pools, and the
no-progress pass termination is only sound when every batch can hold
both pools' oldest candidates. A configured batch size of one surfaces
only the tie-break-favored root pool: with two already-marked queued
roots the pass refuses one, re-skips the other, and ends with the
subagent pool unexamined.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e7300dcc2f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/runner.go
Comment thread coderd/x/chatd/worker.go Outdated
Replica A can refuse a chat and publish the queued capacity event while
replica B later admits it. B has no local refusal entry, so gating the
clear event on the local capacity queue left the banner stale until the
next refetch. On capped deployments, admission of a running chat now
always publishes the clear.

Also assert FIFO admission on the serial Admit call order recorded by
fakeAdmission instead of task-start order, which goroutine scheduling
can reorder, and document the requires_action resume overage bound.
Comment thread coderd/database/queries/chats.sql Outdated
Comment thread coderd/x/chatd/agentadmission.go Outdated
Comment thread coderd/x/chatd/chatd.go Outdated
Review feedback from cian: AgentAdmission and AgentCapacityPolicy always
shipped as the same object, so replace them with a single
AgentCapacityLimiter interface (Admit + Limits). Limits returns the caps
plus a capped bool instead of a Capped field. A noop limiter replaces
nil checks; chatd wires it whenever no factory is configured.

Consolidate CountChatCapacityActiveByPool and
CountChatCapacityUnownedByPool into CountChatCapacityByPool so admission
and the metrics loop share one scan, and widen idx_chats_capacity_active
to cover unowned chats so the merged query stays indexed.
@ibetitsmike
ibetitsmike requested a review from johnstcn August 6, 2026 13:53
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 546e293293

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread site/src/api/queries/chats.ts Outdated
Comment thread coderd/x/chatd/worker.go Outdated
…ents

Codex review findings on 546e293:
- An interrupting chat acquired by a replica that never refused it kept
  the queued banner because the clear only covered running acquisitions.
  Interrupting acquisitions now publish the clear too.
- A delayed queued capacity event could arrive after another replica's
  clear and restore the banner. Acquisition bumps updated_at while
  refusals do not, so the frontend merge now applies capacity events
  only when the snapshot is not older than the cache.

Also add the concurrent-agents docs section and point the queued
banner's learn-more links at it.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f479526e34

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/worker.go Outdated
Codex found the cap can flip to uncapped (license update) between one
replica's refusal, which published queued=true, and another replica's
admission, which then read capped=false and skipped the clear, leaving
a stale banner. The clear condition now depends only on whether a real
limiter is configured: the noop limiter never refuses, so no queued
event can exist, while any real limiter may have refused while capped.
No dynamic state can desync the clear from the queued event anymore.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8e7c35ee66

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/worker.go
…pshot

Codex found the skipped-pool path published queued=true from a stale
candidate row: another replica can acquire the chat between the batch
query and the publish, and because the event payload reloaded the chat,
it carried the acquisition's new updated_at and defeated the frontend
freshness guard when delivered after the owner's clear.

enterCapacityQueue now validates and publishes from one snapshot: it
only queues chats that are still running, unarchived, and without a
live owner heartbeat, and the event payload is that same snapshot, so
a stale queued event either never publishes or carries pre-acquisition
updated_at that loses to the clear. Crashed-owner takeovers still queue
because their heartbeat is stale.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d5fcb03a20

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/worker.go
…kipped passes

Codex found prune starvation: with a sustained full-pool backlog every
acquisition pass ends on an all-skipped batch, which returned before
pruneCapacityQueue could run, so an entry for a chat another replica
acquired persisted indefinitely, suppressing the next episode's queued
event and measuring wait time from the previous episode.

An all-skipped pass now reconciles the local entries it did not see
with one targeted FilterChatCapacityWaiting query and drops those that
stopped being capacity-wait candidates. One bounded query per pass,
instead of paging the global candidate stream, which under churn had
no termination bound.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6796c34075

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/worker.go
An all-skipped acquisition pass ended before reaching chats sorted
behind the known queue heads, so a chat arriving on a saturated pool
never got its queued event and open tabs kept showing plain running
state. Reconcile now lists all waiting chats once per all-skipped
pass, pruning departed entries and queueing arrivals in pools the
pass proved full.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 18e4af8d75

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/x/chatd/runner.go
Comment thread site/src/api/queries/chats.ts Outdated
…races

Two races found by review:

Resuming a chat from waiting or error during the runner's abandon gap
preserved worker_id/runner_id, so generation restarted on the retained
runner without capacity admission while the slot release may have
already admitted a queued chat. Idle-state resumes now clear ownership,
routing the resume through worker acquisition and re-admission.

A capacity clear published at acquisition could be rejected by the
frontend freshness guard when a newer status event advanced the cached
updated_at first, leaving the queued banner stuck on a generating chat.
Capacity events now order against a per-chat capacity revision (the
newest capacity event's updated_at) instead of the general guard.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: afd69ab78a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread site/src/api/queries/chats.ts
…ity invalidation

The ordering guards can reject a legitimate capacity event when
delivery reorders against message sends that bump updated_at (for
example a queued event loaded before but delivered after a send to the
same still-unowned chat). The worker keeps its local queue entry and
emits no replacement event, so the banner would stay wrong until an
unrelated refetch. After merging a capacity event into the caches,
invalidate the chat entity when the merged result does not reflect the
event; the single-chat GET derives the authoritative queued state.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 280da2ad2b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants