Skip to content

improvement(deployments): shorten lock windows and add tx safety timeouts on the deploy path#5841

Merged
waleedlatif1 merged 1 commit into
stagingfrom
improvement/deploy-lock-friendliness
Jul 22, 2026
Merged

improvement(deployments): shorten lock windows and add tx safety timeouts on the deploy path#5841
waleedlatif1 merged 1 commit into
stagingfrom
improvement/deploy-lock-friendliness

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • checkpointWebhookCandidate is now a single fenced UPDATE (no transaction, no workflow-row lock) — it runs after every external provider call, so it no longer holds webhook locks across client round trips; operation currency is asserted in-statement via exists/notExists
  • Merged the retire + repoint webhook UPDATEs at activation into one generation-conditional statement (candidate promotion stays separate — non-deferrable partial unique index webhook_active_registration_unique forbids vacate+claim in one statement)
  • Collapsed the deactivate-all + activate-one workflow_deployment_version pair into one UPDATE that only touches rows whose is_active actually changes (stops rewriting every version row on every deploy)
  • Dropped no-op transaction wrappers and an ineffective FOR UPDATE from the webhook cleanup snapshot/delete helpers
  • New setDeploymentTxTimeouts (statement 30s / lock 5s / idle-in-tx 30s via set_config(..., true), PgBouncer-safe, one round trip) applied to all 8 deployment-path transactions so a wedged client can never pin webhook/workflow locks indefinitely
  • Bounded retention prune for terminal workflow_deployment_operation rows (>30d) in the stale-executions cron; the newest-generation row per workflow is structurally excluded so generation numbering stays monotonic

Context

Follow-up to the PlanetScale lock-spike alert: idle-in-transaction sessions holding RowExclusiveLocks on webhook + its indexes during concurrent deploys. Root cause was multi-statement transactions idling between round trips, not traffic.

Type of Change

  • Improvement

Testing

All 7 affected suites pass (95 tests); SQL rendering verified against drizzle 0.45.2 via toSQL(); semantics of merged statements audited for bit-identical column values vs the old statement pairs

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 22, 2026 6:24am

Request Review

@cursor

cursor Bot commented Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches core deploy/webhook persistence and generation fencing; incorrect pruning or merged SQL could wedge deployments or corrupt registration state, though tests and explicit invariants mitigate this.

Overview
Reduces Postgres lock hold time and idle-in-transaction risk on the workflow deploy path after lock-spike incidents during concurrent deploys.

setDeploymentTxTimeouts sets transaction-local statement (30s), lock (5s), and idle-in-tx (30s) limits via set_config(..., true) at the start of deployment-path transactions (prepare, activation, outbox schedule/cleanup paths, webhook prepare/list-retired).

Webhook registration store: checkpointWebhookCandidate is now a single fenced UPDATE with no transaction or workflow-row lock; operation currency is enforced in-statement with exists/notExists on workflow_deployment_operation. Activation merges retire + repoint active rows into one generation-conditional UPDATE (candidate promotion stays separate for unique-index ordering). Cleanup snapshot read/delete drop empty transactions and FOR UPDATE.

activateDeploymentOperation flips active deployment version in one UPDATE that only touches rows whose is_active changes, instead of deactivating all versions then activating one.

Stale-executions cron adds batched pruning of terminal workflow_deployment_operation rows older than 30 days, excluding the newest generation per workflow so deploy generation numbering stays monotonic.

Reviewed by Cursor Bugbot for commit c9e9909. Configure here.

@greptile-apps

greptile-apps Bot commented Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR reduces lock time and adds safety limits to the workflow deployment path. The main changes are:

  • Replaces webhook candidate checkpoint transactions with a fenced update.
  • Consolidates webhook and deployment-version activation updates.
  • Adds transaction-scoped statement, lock, and idle timeouts.
  • Removes no-op transaction wrappers from webhook cleanup helpers.
  • Prunes old terminal deployment operations while retaining the latest generation.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the updated deployment and webhook paths.

Important Files Changed

Filename Overview
apps/sim/app/api/cron/cleanup-stale-executions/route.ts Adds bounded retention pruning for terminal deployment operations while preserving each workflow's latest generation.
apps/sim/lib/webhooks/registration-store.ts Shortens webhook lock windows with statement-level fencing, consolidated activation updates, and simpler cleanup queries.
apps/sim/lib/workflows/deployment-outbox.ts Applies deployment transaction timeouts to schedule and workflow-output reconciliation.
apps/sim/lib/workflows/persistence/deployment-operations.ts Adds transaction safety timeouts and combines deployment-version activation into one conditional update.

Reviews (2): Last reviewed commit: "improvement(deployments): shorten lock w..." | Re-trigger Greptile

cursor[bot]
cursor Bot reviewed Jul 22, 2026
View reviewed changes
Comment thread apps/sim/hooks/queries/mcp.ts Outdated
@waleedlatif1
waleedlatif1 force-pushed the improvement/deploy-lock-friendliness branch from 97d245d to c9e9909 Compare July 22, 2026 06:24
@waleedlatif1

waleedlatif1 commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

waleedlatif1 commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator Author

@cursor review

cursor[bot]
cursor Bot reviewed Jul 22, 2026
View reviewed changes

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit c9e9909. Configure here.

@waleedlatif1 waleedlatif1 mentioned this pull request Jul 22, 2026
Merged
@waleedlatif1
waleedlatif1 merged commit e0e6f24 into staging Jul 22, 2026
20 checks passed
@waleedlatif1
waleedlatif1 deleted the improvement/deploy-lock-friendliness branch July 22, 2026 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cursor cursor[bot] cursor[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@waleedlatif1