v0.7.14: perf improvements, code hygiene, GitLab private host support#5201
v0.7.14: perf improvements, code hygiene, GitLab private host support#5201waleedlatif1 wants to merge 7 commits into
Conversation
waleedlatif1
commented
Jun 24, 2026
- improvement(pi): minor improvements to docs (improvement(pi): minor improvements to docs #5192)
- improvement(mistral): update OCR pricing to OCR 4 rate ($4/1,000 pages) (improvement(mistral): update OCR pricing to OCR 4 rate ($4/1,000 pages) #5193)
- perf(workspace): server-prefetch home, knowledge, tables, and files list pages (perf(workspace): server-prefetch home, knowledge, tables, and files list pages #5196)
- refactor(stores): model execution and workflow-diff state as status enums (refactor(stores): model execution and workflow-diff state as status enums #5197)
- refactor(sse): consolidate client SSE readers behind a single typed primitive (refactor(sse): consolidate client SSE readers behind a single typed primitive #5195)
- feat(gitlab): support self-managed GitLab host across tools, block, triggers, webhook, and connector (feat(gitlab): support self-managed GitLab host across tools, block, triggers, webhook, and connector #5200)
…s) (#5193) * improvement(mistral): update OCR pricing to OCR 4 rate ($4/1,000 pages) * docs(mistral): document mistral-ocr-latest alias resolves to OCR 4
…rimitive (#5195) Replace four hand-rolled client SSE decode loops with two layered primitives in lib/core/utils/sse.ts: - readSSELines: the single byte-stream decode engine. Splits on \n, strips trailing \r, tolerates data: with/without a leading space, skips the [DONE] sentinel, honors an AbortSignal before each chunk and between events, and releases the reader lock only when it acquired it. - readSSEEvents<T>: a thin JSON layer that parses each payload and routes unparseable lines to onParseError (default: skip). An SSESource union accepts a Response, a ReadableStream, or an already-acquired reader so callers that must stash the reader for external cancellation keep ownership of the lock. Migrates use-execution-stream, chat use-chat-streaming, home use-chat (via readSSELines for schema-validated decode), and the workflow chat panel. Legacy server/wand exports (encodeSSE, SSE_HEADERS, readSSEStream) are untouched. Behavior is preserved across abort, RAF batching, TTS, [DONE], delimiter tolerance, and reader-lock ownership. Tests in sse.test.ts pin the prior behavior: \n and \n\n framing, mid-chunk splits, [DONE], data: with/without leading space, \r\n stripping, sync/async early-stop, pre-aborted and mid-stream abort, lock release/non-release per source, lock release on a throwing handler, and Response/stream/reader sources.
…riggers, webhook, and connector (#5200) * feat(gitlab): support self-managed GitLab host across tools, block, triggers, webhook, and connector Add an optional `host` so the GitLab integration can target a self-managed instance (e.g. gitlab.example.com) instead of gitlab.com. Defaults to gitlab.com everywhere, so existing workflows, blocks, triggers, and stored webhooks are unchanged. - Shared host helper (normalizeGitLabHost/getGitLabApiBase) used by all 19 tools, the block, triggers, the webhook provider, and the connector - SSRF hardening: reject structurally unsafe hosts (userinfo `@`, whitespace, control chars, embedded path/query, empty labels) before the token-bearing request is built; allow self-managed hosts, ports, and IDN punycode - Route the webhook provider's previously-raw fetches through secureFetchWithValidation (DNS + private-IP rejection + IP pinning), matching the tool and connector paths - Add regression tests for the host validator * fix(gitlab): handle unsafe-host errors gracefully in webhook provider Address review feedback: - Validate the optional self-managed host up front in createSubscription and deleteSubscription so a structurally unsafe value surfaces as a clear error (create) or a graceful non-strict skip (delete) instead of an unhandled UnsafeGitLabHostError, mirroring the connector's handling - Document the layered SSRF defense: bare IP literals pass the structural host guard by design and are rejected at the fetch layer (validateUrlWithDNS); add a confirming test group making that intent explicit * fix(slack): drop assistant:write scope pending app review approval Requesting assistant:write before Slack approved it fails the OAuth/install flow for users. Remove it from both request paths until approval lands: - Remove from the user OAuth scope list (oauth.ts), matching the existing users:read.email TODO pattern - Remove the action_assistant capability from the bot manifest generator (capabilities.ts), leaving a TODO to restore it after approval The set_status/set_title/set_suggested_prompts tools remain and surface their existing graceful "reconnect with assistant:write" message until re-enabled.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
PR SummaryMedium Risk Overview Workspace lists now server-prefetch on Home, Knowledge, Tables, and Files via cookie-forwarded internal Client SSE is centralized in Execution and workflow-diff stores gain GitLab adds optional
Mistral OCR cost per page updates to the OCR 4 rate ($0.004/page). Slack Reviewed by Cursor Bugbot for commit e1c3c7f. Bugbot is set up for automated code reviews on this repo. Configure here. |
Greptile SummaryThis release bundles four independent improvements: GitLab self-managed host support wired across tools, blocks, triggers, webhook provider and connector; a consolidated typed SSE primitive (
Confidence Score: 4/5Safe to merge after confirming the tool executor routes GitLab API calls through the SSRF-protected fetch layer; all other changes are clean refactors with good test coverage. Every changed surface — SSE primitive, store enums, prefetches — is well-tested and the logic is correct. The one open question is whether the 20 GitLab tool configs, which now build URLs from a user-supplied hostname, are executed through the same secureFetchWithValidation chokepoint that the webhook provider and connector explicitly use. The structural guard in normalizeGitLabHost prevents authority-confusion attacks, but bare IPs pass through and would reach internal infrastructure carrying the user's PRIVATE-TOKEN header if the tool executor uses plain fetch. apps/sim/tools/gitlab/utils.ts and all 20 tool files under apps/sim/tools/gitlab/ — confirm the tool executor applies DNS/IP SSRF protection when resolving ToolConfig.request URLs.
|
| Filename | Overview |
|---|---|
| apps/sim/lib/core/utils/sse.ts | New typed SSE primitive: readSSELines (raw) + readSSEEvents (JSON). Handles both \n and \n\n framing, \r stripping, [DONE] sentinel, abort signal, and correct lock-ownership semantics. Comprehensive tests accompany it. |
| apps/sim/tools/gitlab/utils.ts | New shared normalizeGitLabHost/getGitLabApiBase utilities with structural host validation (UnsafeGitLabHostError). Correctly rejects userinfo, embedded paths, whitespace; intentionally allows bare IPs delegating DNS/SSRF blocking to the fetch layer — needs confirmation the tool executor honours the same chokepoint. |
| apps/sim/lib/webhooks/providers/gitlab.ts | Host parameter threaded through subscribe/unsubscribe; all outbound GitLab API calls switched to secureFetchWithValidation and UnsafeGitLabHostError handled gracefully in both strict and non-strict modes. |
| apps/sim/stores/execution/types.ts | Introduces ExecutionStatus enum ('idle' |
| apps/sim/stores/workflow-diff/types.ts | Introduces WorkflowDiffStatus ('none' |
| apps/sim/app/workspace/[workspaceId]/lib/prefetch-internal-fetch.ts | Server-side cookie-forwarding prefetch helper; workspaceId comes from Next.js router params, getInternalApiBaseUrl is a fixed value. Safe pattern, well-documented rationale for routing through API routes instead of the data layer directly. |
| apps/sim/app/chat/hooks/use-chat-streaming.ts | Replaced manual SSE decode loop with readSSEEvents; adds terminated flag to gate post-stream TTS/flush only when no final/error event was received. Also fixes a latent double-finalizeMessageStream bug from the old success-final path that didn't return. |
| apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/chat/chat.tsx | Replaced manual SSE decode loop with readSSEEvents. Minor: flushChunks() is called redundantly after readSSEEvents returns in the error-final path (harmless no-op but slightly inconsistent with use-chat-streaming.ts pattern). |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[User supplies optional host] --> B{normalizeGitLabHost}
B -->|empty / undefined| C[default: gitlab.com]
B -->|structurally unsafe\nuserinfo / path / whitespace| D[UnsafeGitLabHostError]
B -->|bare IP / valid hostname| E[getGitLabApiBase]
E --> F[https://host/api/v4]
F --> G{Call surface}
G -->|Webhook provider subscribe / unsubscribe| H[secureFetchWithValidation\nDNS + IP blocklist applied]
G -->|Connector sync| I[secureFetchWithValidation\nDNS + IP blocklist applied]
G -->|Tools ToolConfig.request| J[Tool executor\nSSRF protection unconfirmed]
D --> K[Surfaced as user-facing validation error]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[User supplies optional host] --> B{normalizeGitLabHost}
B -->|empty / undefined| C[default: gitlab.com]
B -->|structurally unsafe\nuserinfo / path / whitespace| D[UnsafeGitLabHostError]
B -->|bare IP / valid hostname| E[getGitLabApiBase]
E --> F[https://host/api/v4]
F --> G{Call surface}
G -->|Webhook provider subscribe / unsubscribe| H[secureFetchWithValidation\nDNS + IP blocklist applied]
G -->|Connector sync| I[secureFetchWithValidation\nDNS + IP blocklist applied]
G -->|Tools ToolConfig.request| J[Tool executor\nSSRF protection unconfirmed]
D --> K[Surfaced as user-facing validation error]
Comments Outside Diff (1)
-
apps/sim/tools/gitlab/get_project.ts, line 31-36 (link)Tool executor SSRF gap for user-controlled host
The webhook provider and connector both explicitly call
secureFetchWithValidationwhen usinggetGitLabApiBase(host), routing the request through the DNS/IP blocklist. The GitLab tools (this file and all 20 sibling tools) build a URL fromgetGitLabApiBase(params.host)but the HTTP call is made by the tool executor framework — which is not visible in the diff and may use plainfetch. If the tool executor doesn't go throughsecureFetchWithValidation, a value like127.0.0.1or169.254.169.254would pass the structural check and the request (carryingPRIVATE-TOKEN) would reach internal infrastructure.The test file explicitly documents this assumption: "SSRF to private/loopback/metadata addresses is the responsibility of
validateUrlWithDNS/secureFetchWithValidationat fetch time, the single SSRF chokepoint" — but unlikecleanupGitLabHookByUrlor the connector, nothing in the diff shows the tool executor enforcing that chokepoint. Please confirm thatToolConfig.requestURLs are processed throughsecureFetchWithValidation(or equivalent) before this merges.
Reviews (1): Last reviewed commit: "feat(gitlab): support self-managed GitLa..." | Re-trigger Greptile
| if (event === 'final' && eventData) { | ||
| if ('success' in eventData && !eventData.success) { | ||
| const errorMessage = eventData.error || 'Workflow execution failed' | ||
| flushChunks() | ||
| finalizeMessageStream(responseMessageId) | ||
| } else if (contentChunk) { | ||
| accumulatedContent += contentChunk | ||
| pendingChunks += contentChunk | ||
| scheduleFlush() | ||
| appendMessageContent( | ||
| responseMessageId, | ||
| `${accumulatedContent ? '\n\n' : ''}Error: ${errorMessage}` | ||
| ) | ||
| } | ||
| } catch (e) { | ||
| logger.error('Error parsing stream data:', e) | ||
| return true | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if (contentChunk) { | ||
| accumulatedContent += contentChunk | ||
| pendingChunks += contentChunk | ||
| scheduleFlush() | ||
| } | ||
| }, | ||
| }) | ||
| flushChunks() | ||
| finalizeMessageStream(responseMessageId) |
There was a problem hiding this comment.
flushChunks() call in error path
When the final event carries a failure, flushChunks() is called inside onEvent (line 575), then return true stops the reader. After readSSEEvents resolves, flushChunks() is called unconditionally again (line 591) — a no-op since pendingChunks is empty, but still confusing. Consider guarding the post-loop flush so it only runs when no final event was processed, similar to how use-chat-streaming.ts uses the terminated flag for this purpose.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
…ning into ECS taskdef (#5189) * feat(secrets): ingest env secrets at container runtime instead of fanning into ECS taskdef The app/socket ECS taskdefs were ~42KB, ~93% of which was the secrets[] array: 268 pointer entries each restating the full ~78-char secret ARN, marching toward the 64KB taskdef limit and growing ~150 bytes per hosted key added. The secret blob itself is only ~18KB/268 keys. Move secret delivery to container boot: new @sim/runtime-secrets loadRuntimeSecrets() reads SIM_ENV_SECRET_ID, fetches the combined secret once, and hydrates process.env (no-clobber, no-op when unset, fail-fast). Bootstrap entrypoints for app + realtime await it before importing the real server (env-flags reads env at module load). The app bootstrap is bun-bundled in the Dockerfile builder stage since it runs outside the Next standalone bundle; realtime keeps full node_modules and runs the TS entry. Backward-compatible: with the current fan-out taskdef the loader no-ops and the app reads the injected env vars unchanged. The matching infra change (empty secrets[] + SIM_ENV_SECRET_ID) ships separately, after this image is live. * fix(runtime-secrets): address review feedback - Move the binary-secret guard outside the retry loop (sendWithRetry) so a missing SecretString throws immediately instead of burning 3 attempts + backoff. - Bound each Secrets Manager request with AbortSignal.timeout(5s) so a stalled response can't hang boot indefinitely. - Drop the redundant @aws-sdk/client-secrets-manager pin from apps/realtime; it resolves transitively via @sim/runtime-secrets. - Add a test for the non-retriable binary-secret path.