Skip to content

improvement(oauth): coalesce token refresh + cache terminal failures#4706

Merged
waleedlatif1 merged 1 commit into
stagingfrom
waleedlatif1/slack-cred-deselect
May 22, 2026
Merged

improvement(oauth): coalesce token refresh + cache terminal failures#4706
waleedlatif1 merged 1 commit into
stagingfrom
waleedlatif1/slack-cred-deselect

Conversation

@waleedlatif1
Copy link
Copy Markdown
Collaborator

@waleedlatif1 waleedlatif1 commented May 21, 2026

Summary

  • Coalesce OAuth token refreshes per account.id so concurrent workflow runs trigger exactly one provider call instead of racing — fixes 201 silent invalid_refresh_token errors / 30d for Slack alone
  • Detect Slack-style HTTP 200 + {ok: false} body errors in refreshOAuthToken (previously silently dropped to WARN with no providerId)
  • Cache terminal failures (invalid_refresh_token, access_denied, etc.) for 1h to stop futile retry storms; flag clears on reconnect
  • Single performCoalescedRefresh helper consolidates the three refresh call sites; reuses existing acquireLock/releaseLock + redisConfigMock from @sim/testing

Type of Change

  • Bug fix
  • Improvement

Testing

  • 199/199 unit + integration tests passing
  • lib/concurrency/__tests__/singleflight.test.ts, lib/concurrency/__tests__/leader-lock.test.ts, lib/oauth/__tests__/terminal-errors.test.ts added
  • Production audit via CloudWatch isolated the rotation race to specific high-traffic Slack credentials
  • Lint clean, typecheck clean on affected packages (sim, realtime, workflow-persistence)

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel
Copy link
Copy Markdown

vercel Bot commented May 21, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped May 21, 2026 11:56pm

Request Review

@cursor
Copy link
Copy Markdown

cursor Bot commented May 21, 2026

PR Summary

Medium Risk
Touches OAuth token refresh behavior and introduces Redis-backed coordination/caching; failures could impact access-token availability or refresh frequency under concurrency.

Overview
OAuth refreshes are now coordinated and de-duplicated per account. Token refresh logic in app/api/auth/oauth/utils.ts is refactored to a shared performCoalescedRefresh that uses local singleflight plus a Redis-based withLeaderLock so only one concurrent request hits the provider while others poll the DB for the updated token.

Refresh failure handling is hardened. refreshOAuthToken now returns a structured { ok: true|false } result (including Slack-style HTTP 200 { ok: false } bodies and extracted errorCode), and terminal refresh errors are cached in Redis for 1 hour via new lib/oauth/terminal-errors.ts to short-circuit repeated failing refresh attempts; the dead flag is cleared on credential create/reconnect.

New concurrency utilities and tests added. Introduces lib/concurrency/singleflight.ts and lib/concurrency/leader-lock.ts with corresponding unit tests, and updates existing OAuth util tests to mock Redis and the new result shape.

Reviewed by Cursor Bugbot for commit b424129. Configure here.

@waleedlatif1 waleedlatif1 force-pushed the waleedlatif1/slack-cred-deselect branch from 74180a1 to 9a07693 Compare May 21, 2026 23:02
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 21, 2026

Greptile Summary

This PR eliminates the OAuth token-refresh race condition by introducing a two-layer coalescing strategy: coalesceLocally deduplicates concurrent refreshes within a single process, while withLeaderLock coordinates across processes via a Redis distributed lock. It also changes refreshOAuthToken to return a discriminated union (RefreshTokenResult) instead of null, detects Slack-style HTTP 200 + {ok:false} body errors, and caches terminal failures in Redis for 1 hour to stop retry storms.

  • singleflight.ts + leader-lock.ts: New concurrency primitives; in-process deduplication plus Redis leader election with correct fallback when Redis is unavailable.
  • terminal-errors.ts: Redis-backed dead-flag cache with graceful no-op on Redis absence; flag cleared on reconnect via draft-hooks.ts.
  • utils.ts: All four refresh call sites consolidated into performCoalescedRefresh; prior behaviour (proactive-refresh fallback to still-valid token, resolvedCredentialId for DB writes) is preserved.

Confidence Score: 5/5

Safe to merge; the coalescing logic is well-tested and all Redis operations degrade gracefully to pre-PR behaviour when unavailable.

The concurrency primitives are correct, the fallback paths (Redis down → uncoordinated leader, lock release failure → TTL expiry) are exercised by tests, and all four refresh call sites now go through the coalesced path. The one non-blocking concern is that a handful of OAuth app-level error codes in the terminal set could cause user-visible reconnect requirements after a server misconfiguration.

apps/sim/lib/oauth/terminal-errors.ts — the TERMINAL_ERRORS set includes app-level OAuth config errors alongside user-level credential errors.

Important Files Changed

Filename Overview
apps/sim/lib/concurrency/singleflight.ts New in-process singleflight coalescer; cleanly deduplicates concurrent calls per key and removes inflight entry on both success and rejection.
apps/sim/lib/concurrency/leader-lock.ts New Redis-backed leader-election primitive; correctly handles acquireLock throw (falls back to uncoordinated leader) and releaseLock throw (swallowed, lock expires via TTL).
apps/sim/lib/oauth/terminal-errors.ts New terminal-error cache; all Redis ops are gracefully no-ops on failure. Includes app-level error codes (bad_client_secret, invalid_client_id) alongside user-level codes — see review comment.
apps/sim/app/api/auth/oauth/utils.ts All four refresh call sites consolidated into performCoalescedRefresh; fallback-to-stale-token logic preserved; getCredentialsForCredentialSet now fully migrated.
apps/sim/lib/oauth/oauth.ts refreshOAuthToken return type changed from null-or-object to discriminated union; Slack-style ok:false body errors now detected; all call sites updated.
apps/sim/lib/credentials/draft-hooks.ts clearDeadFlag called on both new-credential and reconnect paths, correctly resetting the 1-hour terminal-error block.

Reviews (4): Last reviewed commit: "improvement(oauth): coalesce token refre..." | Re-trigger Greptile

Comment thread packages/workflow-persistence/src/save.ts Outdated
@waleedlatif1 waleedlatif1 force-pushed the waleedlatif1/slack-cred-deselect branch from 9a07693 to 5669c4f Compare May 21, 2026 23:09
@waleedlatif1 waleedlatif1 changed the title improvement(oauth): coalesce token refresh + scrub unknown credential refs improvement(oauth): coalesce token refresh + cache terminal failures May 21, 2026
@waleedlatif1 waleedlatif1 force-pushed the waleedlatif1/slack-cred-deselect branch from 5669c4f to 3554a5d Compare May 21, 2026 23:17
@waleedlatif1 waleedlatif1 force-pushed the waleedlatif1/slack-cred-deselect branch from 3554a5d to d6b870f Compare May 21, 2026 23:23
@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@cursor review

@waleedlatif1 waleedlatif1 force-pushed the waleedlatif1/slack-cred-deselect branch from d6b870f to b6c7967 Compare May 21, 2026 23:32
Comment thread apps/sim/lib/concurrency/leader-lock.ts
Comment thread apps/sim/lib/concurrency/leader-lock.ts
@waleedlatif1 waleedlatif1 force-pushed the waleedlatif1/slack-cred-deselect branch from b6c7967 to 92b42c1 Compare May 21, 2026 23:35
Comment thread apps/sim/app/api/auth/oauth/utils.ts
@waleedlatif1 waleedlatif1 force-pushed the waleedlatif1/slack-cred-deselect branch from 92b42c1 to 99d9265 Compare May 21, 2026 23:38
@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/app/api/auth/oauth/utils.ts Outdated
@waleedlatif1 waleedlatif1 force-pushed the waleedlatif1/slack-cred-deselect branch from 99d9265 to b424129 Compare May 21, 2026 23:56
@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@cursor review

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit b424129. Configure here.

@waleedlatif1 waleedlatif1 merged commit 9276463 into staging May 22, 2026
14 checks passed
@waleedlatif1 waleedlatif1 deleted the waleedlatif1/slack-cred-deselect branch May 22, 2026 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant