Skip to content

fix(db): throttle autovacuum on the largest tables so vacuum WAL bursts stop stalling commits - #9087

Merged
waleedlatif1 merged 1 commit into
stagingfrom
fix/vacuum-throttle-large-tables
Oct 15, 2026
Merged

waleedlatif1 merged 1 commit into
stagingfrom
fix/vacuum-throttle-large-tables

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Vacuuming the largest tables (and their TOAST relations) writes WAL, mostly full-page images, faster than the synchronous standbys can flush it. Until the vacuum ends, every write commit on the primary waits for a standby, so the whole app stalls.
  • Adds migration 0431, which sets a per-table autovacuum_vacuum_cost_delay = 10 / autovacuum_vacuum_cost_limit = 200 on embedding, embedding_search and workflow_execution_logs, for each table and its TOAST relation. That caps how fast those vacuums dirty pages, and so their WAL rate.
  • Trade-off: these three tables leave cross-worker I/O balancing and vacuum more slowly. This reverses 0362's choice for these tables only.
  • Lock analysis: ALTER TABLE ... SET (autovacuum params) only updates the catalog and takes SHARE UPDATE EXCLUSIVE. There is no table rewrite. Reads and writes never wait on it, even while it is queued behind a running vacuum or another holder. A running non-wraparound autovacuum is cancelled by the waiting ALTER and restarts later with the new settings. The migrator's existing 5s lock_timeout plus retry covers the case where the ALTER queues behind a long-running conflicting holder.

Type of Change

  • Bug fix

Testing

  • New packages/db/large-table-vacuum-throttle-migration.integration.ts runs against real Postgres 17 and checks that:
    • the statement holds only ShareUpdateExclusiveLock, and relfilenode is unchanged (no rewrite);
    • another session can read and write the table while the migration transaction is open;
    • while the ALTER is queued behind a conflicting holder, a writer with a 300ms lock_timeout still inserts;
    • every table and its TOAST relation carry both settings.
  • Fails before this change (the migration file is missing). Also fails if the migration is swapped for an exclusive-lock DDL.
  • check:migrations passes, and drizzle-kit generate reports no schema changes.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…ts stop stalling commits

Vacuuming these tables writes WAL, mostly full-page images, faster than the
synchronous standbys flush it, so every write commit on the primary waits for
a standby until the vacuum ends. A per-table autovacuum cost delay and limit,
on each table and its TOAST relation, cap that rate.

The ALTERs are catalog-only under SHARE UPDATE EXCLUSIVE: no rewrite, and
reads and writes never wait on them, even while one is queued behind a
running vacuum. An integration test applies the migration and checks the lock
mode, that writes proceed, and that the settings land on both relations.
@vercel

vercel Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 15, 2026 8:03pm UTC

Request Review

@greptile-apps

greptile-apps Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Critical impact] The PR appears safe to merge; no actionable issues were found.

Summary

Adds migration 0431 to slow autovacuum on embedding, embedding_search, and workflow_execution_logs, including their TOAST relations.

  • Autovacuum on three large tables now sends WAL at a slower pace.

No actionable issues were found. The integration tests were reviewed but not run.

waleedlatif1 explicitly acknowledged slower vacuum and loss of cross-worker balancing as the lesser cost of reducing WAL bursts.

Reviews (1) · Last reviewed commit: "fix(db): throttle autovacuum on the larg..." · Reviewed by Greptile

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 9411deb into staging Oct 15, 2026
53 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/vacuum-throttle-large-tables branch October 15, 2026 22:19

This branch was successfully deployed

1 active deployment
Preview — 7c625b8c Deployed Oct 15, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant