Skip to content

improvement(db): route additional staleness-tolerant reads to the read replica#4966

Merged
waleedlatif1 merged 5 commits into
stagingfrom
improvement/replica-phase2
Jun 11, 2026
Merged

improvement(db): route additional staleness-tolerant reads to the read replica#4966
waleedlatif1 merged 5 commits into
stagingfrom
improvement/replica-phase2

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

Second batch of opt-in dbReplica reads, each classified against the routing checklist (plain SELECT, no locks/transactions, no read-your-writes in a user-perceivable window, not an authz/enforcement decision, staleness-tolerant consumer):

  • Workspace context assembly (lib/copilot/chat/workspace-context.ts): the multi-table fan-out that builds the WORKSPACE.md prompt context (workflows, folders, KBs, tables + row counts, MCP servers, schedules, connectors) now reads the replica — it's prompt-assembly at turn start, and mid-turn freshness comes from VFS tools which stay on the primary. The workspace-access authz gate and shared permission/file/credential helpers are untouched
  • Workspace execution-metrics endpoint: time-bucketed dashboard aggregations over execution logs; the permission check stays on the primary
  • KB chunk pagination + tag display reads (queryChunks, tag definitions/usage): chunk rows are written before a document's processingStatus flips to completed, so any replica state showing a completed document already contains its chunks; tag slot allocation and cleanup decision counts stay on the primary
  • Copilot @-mention context reads: mentioned-KB and mentioned-execution lookups — mention targets originate from already-visible lists
  • Workspace-event activity scans: background rule evaluation over hours-scale windows

Deliberately not flipped: log-detail fetch (read directly after execution by id — read-your-writes), v1 list endpoints whose service helpers also serve mutation-refetch UI paths (would need executor threading; deferred), and admin/invite listings (negligible volume).

Type of Change

  • Improvement

Testing

  • Typecheck clean; 654 tests passing across copilot, knowledge, workspace-events, and workspace-API suites; lint clean
  • Verified replica routing end-to-end in a live environment (replica-side scan counters advance on flipped tables; zero writes on replicas; pagination and authz paths confirmed on the primary)

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Jun 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jun 11, 2026 6:06pm

Request Review

@cursor

cursor Bot commented Jun 11, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Replica lag could briefly skew dashboards, WORKSPACE.md, and no-activity scans, but authz and execution-log mention paths stay on the primary; mis-routing would be user-visible rather than a security bypass.

Overview
Routes another batch of read-only, staleness-tolerant queries from the primary db to dbReplica, offloading dashboard metrics, copilot prompt assembly, and background scans while keeping authz and freshness-sensitive paths on the primary.

Workspace execution metrics (metrics/executions): workflow filtering, all-time bounds, and execution-log aggregation now hit the replica; the workspace permission check remains on db.

Copilot workspace context (workspace-context.ts): the multi-table fan-out for WORKSPACE.md (workflows, folders, KBs, tables + row counts, connectors, MCP servers, schedules) reads the replica; workspace access gating and credential/file helpers are unchanged.

Copilot @-mentions (process-contents.ts): mentioned knowledge-base name lookup uses dbReplica; execution-log context still uses db (read-your-writes). Tests mock dbReplica.

No-activity workspace events (no-activity.ts): paged subscription and watched-workflow listing use the replica; the recent-activity execution-log probe stays on the primary.

Reviewed by Cursor Bugbot for commit 293dc79. Configure here.

Comment thread apps/sim/lib/workspace-events/rules.ts Outdated
Comment thread apps/sim/lib/knowledge/tags/service.ts Outdated
@greptile-apps

greptile-apps Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR routes additional staleness-tolerant reads to a read replica across five files. Permission and authz checks are explicitly kept on the primary in every case, and the critical hasRecentActivity function and execution-log fetch (processExecutionLogFromDb) remain on the primary as well.

  • workspace-context.ts: All prompt-assembly queries (workflows, folders, KBs, tables, MCP servers, schedules, connectors) are moved to dbReplica; getUsersWithPermissions stays on the primary.
  • metrics/executions/route.ts: Dashboard aggregation queries (workflow list, bounds, bucketed logs) are moved to dbReplica; the workspace permission check at line 39 stays on db.
  • no-activity.ts: The background-cron subscription and watched-workflow paginator are moved to dbReplica; hasRecentActivity — the decision-making query — remains on the primary.

Confidence Score: 5/5

Safe to merge. Every authz gate stays on the primary, the decision-making query in no-activity stays on the primary, and processExecutionLogFromDb is correctly retained on the primary.

The routing choices are consistent and conservative: permission checks, authz gates, mutation-adjacent reads (hasRecentActivity, execution log fetch), and cooldown state remain on the primary. Replica reads are limited to prompt-assembly fan-outs, dashboard aggregations, and background cron paginators — all explicitly staleness-tolerant.

No files require special attention. The boundary between replica and primary reads is drawn correctly throughout.

Important Files Changed

Filename Overview
apps/sim/app/api/workspaces/[id]/metrics/executions/route.ts Three analytics aggregation queries moved to dbReplica; the workspace permission check stays on primary.
apps/sim/lib/copilot/chat/process-contents.ts KB lookup in processKnowledgeFromDb moved to dbReplica after authz check; processExecutionLogFromDb correctly retains db (primary), with its former lazy import now resolved at module scope.
apps/sim/lib/copilot/chat/workspace-context.ts All workspace prompt-assembly fan-out queries moved to dbReplica; getUsersWithPermissions authz call stays on primary.
apps/sim/lib/workspace-events/no-activity.ts Background-cron subscription and workflow paginators moved to dbReplica; hasRecentActivity (the decision query) correctly remains on primary.
apps/sim/lib/copilot/chat/process-contents.test.ts Test mock updated to expose dbReplica alongside db, keeping the mock consistent with the new module-level import.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Incoming Request / Cron Poll] --> B{Auth / Permission Check}
    B -->|db PRIMARY| C[Permission Granted?]
    C -->|No| D[Return 401/403]
    C -->|Yes| E[Data Reads]
    E --> F[workspace-context.ts]
    E --> G[metrics/executions/route.ts]
    E --> H[no-activity.ts]
    E --> I[process-contents.ts KB lookup]
    F -->|dbReplica| J[(Read Replica)]
    G -->|dbReplica| J
    H -->|dbReplica| J
    I -->|dbReplica| J
    H --> K{hasRecentActivity decision query}
    K -->|db PRIMARY| L[(Primary DB)]
    M[processExecutionLogFromDb] -->|db PRIMARY| L
Loading

Reviews (3): Last reviewed commit: "fix(db): no-activity decision read stays..." | Re-trigger Greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

Re the Greptile review: the P1 (workspace-event rules evaluating on the replica) was reviewed at an older commit — rules.ts was fully reverted to the primary in 48e5ec2 for exactly the consecutive-failure-window reason described. The P2 (@logs mention lookup missing a freshly completed execution) is a fair catch — the live-updating logs list makes run-then-mention a machine-speed flow, so the by-id lookup is back on the primary in 4f92b15 (single-row lookup, negligible load). KB mention resolution stays on the replica (picker lists are primary-fed and human-paced).

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/lib/workspace-events/no-activity.ts Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 293dc79. Configure here.

@waleedlatif1 waleedlatif1 merged commit da8e6ee into staging Jun 11, 2026
15 checks passed
@waleedlatif1 waleedlatif1 deleted the improvement/replica-phase2 branch June 11, 2026 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant