Skip to content

fix(mothership): serialize headless run admission with chat soft delete - #9107

Merged
waleedlatif1 merged 1 commit into
stagingfrom
fix/run-segment-deleted-chat
Oct 16, 2026
Merged

waleedlatif1 merged 1 commit into
stagingfrom
fix/run-segment-deleted-chat

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Headless run admission (createRunSegment: task wake, inbox, v2 chat, Slack, knowledge chat, benchmarks) neither locked nor checked the chat row. A run inserted after a soft delete's live-run query was never stopped by that delete. Chat-view admission (admitChatTurn) was already safe because it UPDATEs the chat with deletedAt IS NULL.
  • insertRunSegment now reads the chat row FOR SHARE, after the run-admission advisory lock, which is the same order admitChatTurn uses. FOR SHARE conflicts with the soft delete's UPDATE, so either the delete commits first and the run is inserted cancelled with tool admission closed at deletedAt, or the delete waits and its stopLiveRunsTx sees the run and stops it.
  • Callers that use createRunSegment directly already treat a cancelled admission like a Stop that arrived before admission, without calling the worker. Slack was the exception: it ignored the status. It now ends the turn as that Stop would, so it never calls the worker.
  • Hard delete was already serialized, because the run insert's FK KEY SHARE lock conflicts with the delete's FOR UPDATE. Behaviour for live chats is unchanged. The cost is one extra indexed primary-key read per run-segment insert.

Type of Change

  • Bug fix

Testing

  • removal-stops-runs.integration.ts (real Postgres and Redis):
    • a headless run admitted after the delete commits starts stopped;
    • a race between headless admission and the delete, forced in both orders behind a held chat row lock: either the delete's Stop is pushed and recorded, or the run is never live;
    • a headless run on a live chat stays live.
  • assistant.integration.ts: a Slack turn whose chat was deleted before admission never starts the worker.
  • The new tests fail on the previous code and pass with this change.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Oct 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Oct 16, 2026 3:08am UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 5 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Critical impact] The PR appears safe to merge.

Summary

The PR makes headless run admission wait on the chat row so it cannot slip past a soft delete.

  • Headless runs stop when their chat is deleted.
  • Slack skips its worker when admission finds a deleted chat.

No actionable issues found. Tests were inspected but not run.

Diagram

sequenceDiagram
  participant A as Headless admission
  participant C as Chat row
  participant D as Soft delete
  A->>A: Take admission advisory lock
  alt Admission locks chat first
    A->>C: Read FOR SHARE
    A->>A: Insert run and commit
    D->>C: Set deletedAt
    D->>D: Find and stop the run
  else Delete locks chat first
    D->>C: Set deletedAt and commit
    A->>C: Read deletedAt FOR SHARE
    A->>A: Insert cancelled run
  end
Loading

Reviews (1) · Last reviewed commit: "fix(mothership): serialize headless run ..." · Reviewed by Greptile

@waleedlatif1
waleedlatif1 merged commit 0c6c375 into staging Oct 16, 2026
53 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/run-segment-deleted-chat branch October 16, 2026 03:49

This branch was previously deployed

1 inactive deployment
Preview — 1e8e4e78 Deployed Oct 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant