improvement(redis): strip idempotency body and cap mothership stream zsets#4624
improvement(redis): strip idempotency body and cap mothership stream zsets#4624waleedlatif1 wants to merge 65 commits into
Conversation
…ership workflow edits via sockets, ui improvements
…ng improvements, posthog, secrets mutations
…ration, signup method feature flags, SSO improvements
…nts, secrets performance, polling refactors, drag resources in mothership
…y invalidation, HITL docs
…endar triggers, docs updates, integrations/models pages improvements
…ions, jira forms endpoints
…mat, logs performance improvements fix(csp): add missing analytics domains, remove unsafe-eval, fix workspace CSP gap (#4179) fix(landing): return 404 for invalid dynamic route slugs (#4182) improvement(seo): optimize sitemaps, robots.txt, and core web vitals across sim and docs (#4170) fix(gemini): support structured output with tools on Gemini 3 models (#4184) feat(brightdata): add Bright Data integration with 8 tools (#4183) fix(mothership): fix superagent credentials (#4185) fix(logs): close sidebar when selected log disappears from filtered list; cleanup (#4186)
v0.6.46: mothership streaming fixes, brightdata integration
…m integration, atlassian triggers
…rizations, mothership positional table row insertion, CI improvements, org-external users, file viewer improvements
v0.6.62: fix new copilot chat creation and selection on refresh
…ixes, db query optimizations, contract boundaries code hygiene, CORS, toast improvements, tables infinite query, executor robustness, reranker support
…tion blocks/connectors updates
…ogs block, parallel-in-loop wall clock, gpt-image-2
…s, logs panel width, tables UI/DB decoupling v0.6.67: VFS upload fix, posthog/copilot correlation, exa date filters, logs panel width, tables UI/DB decoupling
…ering upgrades, data drains, security hardening, paginated dropdowns
…ntegrations, robots.txt update, workday hardening
v0.6.72: billing pool contention fix
…personation fixes, md rendering, doc/pdf/pptx generation improvements
…pentelemetry updates, data drains to snowflake, blob, datadog, bigquery
…ip md polish v0.6.75: scheduler claim-budget drain, helm chart hardening, mothership md polish
v0.6.78: file block get
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
PR SummaryMedium Risk Overview Adds Reviewed by Cursor Bugbot for commit 832b782. Configure here. |
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 29606901 | Triggered | Generic High Entropy Secret | a54dcbe | apps/sim/providers/utils.test.ts | View secret |
| 32763747 | Triggered | Generic Password | 3e9849b | helm/sim/tests/validators_test.yaml | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secrets safely. Learn here the best practices.
- Revoke and rotate these secrets.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
Greptile SummaryThis PR fixes two Redis memory drivers behind the 2026-05-15 OOM incident: it strips the full workflow result from webhook/polling idempotency keys (only a
Confidence Score: 5/5Safe to merge — both changes are targeted, well-reasoned, and address a confirmed production OOM. The ZREMRANGEBYRANK stop index (-eventLimit - 1) is arithmetically correct: when the set is under the limit the resulting rank is negative and Redis returns an empty range (no-op); at and above the limit it evicts the correct surplus. All callers of webhookIdempotency and pollingIdempotency are await-without-capture, so the undefined returned on duplicate calls is silently discarded exactly as intended. The test mock's handling of negative stop indices matches real Redis semantics. No behavioural regressions are visible on any changed path. No files require special attention. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Incoming webhook or polling event] --> B{atomicallyClaim}
B -->|first time| C[execute operation]
C --> D{storeResultBody?}
D -->|true - default| E[store result + body in Redis]
D -->|false - webhook/polling| F[store marker only: success + status]
E --> G[return actual result]
F --> G
B -->|duplicate - completed| I[return existingResult.result - undefined if storeResultBody=false]
B -->|duplicate - in-progress| J[waitForResult polls - returns undefined if storeResultBody=false]
B -->|duplicate - failed| K[throw or retry]
subgraph trim [appendEvents buffer.ts]
L[pipeline.zadd] --> M[pipeline.zremrangebyrank 0 to -eventLimit-1 - keeps newest eventLimit entries]
M --> N[pipeline.expire and set seq]
end
Reviews (1): Last reviewed commit: "chore(redis): trim verbose comments on i..." | Re-trigger Greptile |
Summary
ZREMRANGEBYRANKtrim toappendEventsso mothership stream ZSETs respect the existingCOPILOT_STREAM_EVENT_LIMITconfig (was dead code; biggest observed key was 3,755 members and climbing).Type of Change
Testing
Tested manually — audited dedup-path callers (`processor.ts` passes the return to `jobQueue.completeJob` for logging only; polling sites discard it) to confirm body-stripping is observably safe. Updated `buffer.test.ts` to assert the trim is called.
Checklist