v0.8.4: nextjs upgrade, cli updates, jotform triggers, secrets descriptions, helm updates - #6805
v0.8.4: nextjs upgrade, cli updates, jotform triggers, secrets descriptions, helm updates#6805waleedlatif1 wants to merge 21 commits into
Conversation
…live code (#6777) * chore(deps): upgrade next to 16.3.1, its optimizer no longer deletes live code 16.3.0 was reverted in #6242 because its Turbopack optimizer modelled a bare `return <asyncCall>()` tail call inside an async function as returning the promise object, propagated that always-truthy fact through the caller's `await`, and deleted everything after the resulting `if`. That shipped two dead code paths to production: the whole `POST /api/credentials` create path, and the insert inside `upsertAsyncToolCall`. We reported it as vercel/next.js#96595. The fix — "[turbopack] Collapse nested promises in the analyzer" (vercel/next.js#96601) — folds `Promise<Promise<T>>` to `Promise<T>` in the analyzer, and was backported as #96675 and released in 16.3.1. Verified before taking the bump: - The minimal reproduction from the issue no longer reproduces on 16.3.1. All four routes keep their code; on 16.3.0 `/api/broken` lost everything after the `if`. - A production build of `apps/sim` on 16.3.1 still emits the markers whose disappearance was the original signal: `credential_connected` (43 files), `acquireOrganizationUserMutationLocks` (28), and the `upsertAsyncToolCall` insert-path warning (10). The `return await` hardening added to both sites in the revert stays as is, and so does the TypeScript toolchain configuration. 16.3.1 published 2026-08-13, so it is inside the 7-day `minimumReleaseAge` supply-chain window until 2026-08-20 and needs an exclusion to install. The alternative is sitting on 16.2.12, whose successor we already reverted once, so the entries go in dated and come out on the next touch of the file. The mermaid and js-yaml exclusions aged out on 2026-08-11 and 2026-08-07 and are dropped here per that same rule. * fix(deps): keep the musl and win32 SWC binaries in the lockfile The release-age exclusion only listed the four @next/swc platforms that package.json pins, but next declares all eight as its own optionalDependencies, so all eight are normally resolved into bun.lock. A gated optional dependency does not fail the install — bun drops it silently — so the first install stripped both musl variants and both win32 variants from the lockfile. That left the Alpine devcontainer and any Windows machine with no SWC binary to resolve. Adding the remaining four to the exclusion list restores all eight entries at 16.3.1. Worth knowing for the next time this happens: bun.lock is sticky here. Once an optional dependency has been dropped, re-running the install — even with --force, even with the age gate switched off entirely — does not bring it back, because the resolution is not reattempted. The lockfile has to be regenerated from a base that still contains the entries, which is why this restores bun.lock from staging before re-applying the bump.
…elease-age waivers (#6784) js-yaml < 4.3.1 has quadratic CPU consumption in !!omap resolution (GHSA-5p4m-2wfm-xmqj / CVE-2026-59870). sim-cli builds with --packages=bundle, so its dev-scoped js-yaml is bundled into the published CLI. 4.3.1 is already what apps/sim pins, so this collapses sim-cli onto the hoisted copy. The minimumReleaseAge waivers for js-yaml and mermaid were temporary and have both aged past the 7-day window; leaving them behind would disable the supply-chain gate for those packages indefinitely.
…OneDrive (#6785) * fix(connectors): index Office documents and PDFs from SharePoint and OneDrive The SharePoint and OneDrive connectors filtered their listings against a 12-item plain-text extension whitelist, so a document library of .docx, .pdf or .xlsx files synced as "success, 0 documents" — no document, no failed row, and no log line, which is indistinguishable from a wrong folder path. Both whitelists had been unchanged since the connectors shipped, and Sim already parses all of these formats for a manually uploaded knowledge base document. Adds a shared `extractConnectorText` in connectors/utils that routes binary document formats through the same `parseBuffer` the upload path uses, so the OOXML zip-bomb guard and each parser's extraction limits apply. The previously-accepted text formats stay on their exact existing path: sending .csv through CsvParser would silently reformat every already-indexed connector document on its next re-index. Also logs a per-page count of files skipped for an unsupported extension. Unsupported files are counted rather than turned into failed document rows, so a library full of images does not fill the knowledge base with noise. * fix(connectors): never index a degraded document extraction `DocParser` and `PptxParser` never throw by design — on a legacy OLE `.doc`/`.ppt` or a deck with no extractable text they return a placeholder sentence or scraped ZIP internals so an interactive upload still shows the user something. Verified against real OOXML fixtures: an image-only `.pptx` yields 1.9KB of `[Content_Types].xml…` as "content", and a legacy `.ppt` yields "Unable to extract text from PowerPoint file." A connector sync would embed that into the vector index at scale, so it needs to tell a real extraction from a fabricated one. Adds a declared `degraded` flag to `FileParseMetadata`, set by exactly those two fallback paths, rather than having callers sniff `extractionMethod`. `DocParser`'s plaintext branch stays unflagged: a text file misnamed `.doc` is a genuine extraction. `extractConnectorText` now raises `ConnectorTextExtractionError` when a parsed format comes back degraded or blank, and SharePoint/OneDrive surface it as a skipped document via the existing `markSkipped` path — so the file appears in the knowledge base as a failed row telling the user to re-save it as DOCX/PPTX/XLSX, instead of being silently dropped or indexed as junk. The upload path is unaffected; it ignores the new flag. * fix(file-parsers): register the document variants the parsers already handle A document library holds whole format families, not just the headline extension of each. These all extract correctly with the libraries already installed — they were simply never registered, so every one of them was reported as an unsupported file type: docm dotx (WordprocessingML — mammoth reads word/document.xml regardless of the package content type) xlsm xlsb xltx ods (SheetJS reads every workbook container natively) pptm potx (PresentationML) odt odp (OpenDocument, via a new OpenDocumentParser) Verified against real fixtures built with jszip and SheetJS rather than assumed: officeparser identifies a Buffer by sniffing content with `file-type`, not by the name we pass, so the routing had to be measured. `ods` goes to the spreadsheet parser rather than OpenDocumentParser so its output keeps per-sheet structure. `rtf` is deliberately excluded: nothing bundled extracts it, and DocParser's plaintext branch would pass its control words through as if they were prose. Converts the registry from `require()` inside per-parser `try/catch` blocks that only logged to static imports. Every parser dependency is a regular, non-optional one, so a resolution failure should fail loudly — the old form produced a silently **empty** registry in which every format became `Unsupported file type`, with an empty "Supported types are:" list as the only clue. The heavy extraction libraries are still deferred inside the individual parsers, and connectors now import the registry lazily so the ~60 connectors that never touch a file do not pull SheetJS. Adds registry.test.ts, which exercises the real module: index.test.ts mocks `@/lib/file-parsers` itself, so it validated its own fake routing table and the real registry had no coverage at all. The new test gates every member of SupportedFileType on having a registered parser that supports buffer parsing. * fix(file-parsers): resolve the parser registry through a Map, not object keys The registry rewrite switched extension lookup from `Object.keys(parsers).includes(ext)` to a bracket read on an object literal, which also resolves inherited keys. `PARSERS['constructor']` therefore returned `Object` — truthy, with no parse methods — so a caller-supplied extension of `constructor` fell through to "does not support buffer parsing" instead of being rejected as an unsupported type, and `parseFile` would have raised a TypeError. It also disagreed with `isSupportedFileType`, which used `Object.hasOwn` and correctly returned false for the same input. A Map has no prototype chain to walk, so lookup and support check now agree by construction. `isSupportedFileType` also guards a non-string argument, which the try/catch it replaced used to absorb.
* fix(cli): resolve findings from a full command-surface audit Exercised all 147 commands against a live deployment. Fixes the defects that surfaced, plus the docs and generator drift they exposed. Transport - Stop following redirects. A bare domain that 301s to www silently converted POST to GET and dropped the body, so reads worked while every write failed with a misleading validation error and login returned 405. Both the client and the device flow now explain the redirect and name the endpoint to configure, rather than carrying credentials off-origin. - Report a non-JSON response as one instead of printing the HTML page. - Name the personal-API-key remedy on a workspace-key refusal, reading the machine-readable code the API actually sends. - Drop union-branch noise from validation errors that contradicted itself. - Show paging progress on stderr for multi-page fetches. Output - Clamp record values for table only. text is the format built for pipes, and it was truncating signed URLs and tool source mid-value. - Infer timestamp, duration, bytes and boolean formatting for API-owned keys so undeclared commands stop printing raw ISO and float ms. Skips user-defined table cells and leaves json/yaml on the raw payload. - Render a declared-but-absent field as an em dash; billing credits were vanishing silently. Paths, naming and validation - Percent-encode folder paths per segment and decode them for display, so a folder reads and types as the name shown in the app. - Reject a malformed endpoint where it is set and where it resolves, instead of crashing with a URL parse trace. - Request the detail level logs list's own columns need; its workflow column could never populate. - Rename three commands that described themselves wrongly and align two flags with their siblings. Old spellings still work: hidden, warned on stderr, and kept out of help and docs. - Verify whoami against the API, separating a bad key from an unreachable endpoint, and report the workspace by name. - Correct the --yes help text, which advertised skipping a prompt that does not exist. Docs - Teach the docs generator that a flag required by the runtime is required, and that hidden commands are not documented. * fix(cli): clear the paging progress line when a page fails Progress is written without a trailing newline so it can be overwritten in place, and both paging loops cleaned it up only on success. A page that threw part-way through left `fetched 1200…` on the line the error was then printed onto, so the two ran together. * fix(cli): name a working API root when an endpoint redirects The suggested endpoint was the redirect target's origin, which drops a path prefix. A self-hosted deployment reached at https://host/sim was told to set https://www.host — not an API root, so following the advice replaced one broken endpoint with another. Derive it by stripping the request's own path from the target instead, so a prefix survives, and say nothing about --set-endpoint when the target resolves to the endpoint already configured: a trailing-slash or path normalization redirect keeps the origin, and naming the value the caller already has explains nothing. The login poll shared both faults and now shares the helper.
* fix(workflows,connectors): close pre-merge audit findings Recover subblock values orphaned by the id renames in this release, and stop truncated knowledge-base listings from reporting themselves complete. - Add operation-scoped subblock id migrations so a saved workflow's stored value survives a rename. Cloudflare create/update DNS record, ServiceNow read record, and Okta deactivate/delete previously lost their stored value: the create path substituted a seeded default (an A record where the user chose CNAME, and unproxied where they chose proxied), and the update path silently no-opped while reporting success. A migration is used rather than a legacy-id fallback so no subblock id carries two value spaces at runtime. - Webflow, Zendesk: a listing that stops for a reason the connector cannot rule out now reports as capped instead of exhausted. A malformed envelope, an unfollowable continuation link, or an absent collection list previously read as a complete listing and let deletion reconciliation hard-delete every document past the truncation point. - Sentry: pin the listing window in the request rather than inheriting the server default, so the range cannot silently narrow into hard deletes. - Fork sync: a parent re-pick no longer writes a blank over a hidden optional dependent's stored target value, and a required field stays on screen once it is filled. Add hook-level coverage for the submitted payload. - Fork file copy: a file whose name is already taken in a reused target folder is de-duplicated instead of dropped. - Delete an orphaned Shopify OAuth route that built a credential from unsigned cookies. It had no writer, no caller, and no inbound link. - Tailwind: drop two content globs that scanned 5.4k files to emit one unused rule, keeping the ones that fix brand tile icon color. - Correct the API route-count baseline, add an Evernote docs redirect, align library copy with the language rules, and fix a stale turbo filter. * fix(connectors,forking): trim the audit fixes to their minimum A legitimacy review found several changes closed no live defect, and two introduced problems of their own. - Zendesk: narrow the cursor fix to a signal change. Treating a missing meta envelope as truncation had also made the walk follow links.next and keep paginating, and the ticket cursor has no page-depth valve, so a source advertising a next page with no meta could loop without terminating. The page-fetch set now matches the previous behavior; only the flag is new. - Zendesk: drop the search next_page branch. The existing count check already caps every case where a missing key could lose documents. - Webflow: drop the empty-collections flag. The sync engine already blocks the first sync on an empty listing and reconciles only when a second sync agrees, which handles a transient fault better and still removes documents when a source is genuinely emptied. The flag short-circuited that and suppressed reconciliation permanently. Restore the previous loud failure on a non-array envelope, and drop the unreachable collection-id filter. - Webflow: soften a docstring that claimed pagination.total is always present. It is documented optional, so its absence proves nothing either way and treating it as unprovable truncation is the fail-safe reading. - Sentry: drop the pinned statsPeriod. Sentry's issue search floors every query at 90 days in the executor regardless of the request, and the endpoint this release moved away from hit the same floor, so there was no window to close. Keep the tests and the docstring recording that. - Fork copy: drop the renamed counter, which no caller reads. - Repair check-block-registry, which stopped exempting migrated subblock ids when the migration map became an array — `in` was testing array indices. - Drop mdx from a Tailwind content glob that emits nothing, and loosen an exact compiled-SQL assertion to the invariant it was pinning. * fix(migrations): keep a ServiceNow write body off the read projection Review findings from the first round. - A legacy ServiceNow block can hold a Create/Update Record JSON body under `fields` while its stored operation is Read Records: the id served both value spaces before the rename, and a subblock value is not cleared when the operation changes. The scoped migration moved that body onto `readFields`, where it would reach the wire as sysparm_fields. Migration entries can now carry a `whenValue` predicate for the case where the stored operation alone cannot separate two value spaces, and the ServiceNow entry uses it to move only a plausible comma-separated projection. - Type the fork copy test harness instead of using `any`, without weakening it: every predicate shape it does not model still throws rather than matching. - Correct the dependent-omission comments. Omitting a parent-invalidated field preserves the target's stored value on Save and across an undo, where the parent nets out unchanged; on a Sync the written state is source-derived, so what it prevents there is an explicit blank reaching the fields the remap's clearing pass does not cover, nested tool params in particular. Okta's migration scope is left as-is: `okta_remove_user_from_app` and the sendEmail split shipped in the same release, so no saved block can hold legacy state for it, and widening the scope would promote an activation-era value onto the deactivation switch. Tests document the boundary. * chore(forking): move the fork-sync changes to their own PR The dependent-omission fix and the fork file-copy de-duplication are reviewed separately in #6787. They are the only changes here that overlap #6776, and they carry their own design tradeoff, so they should not ride along with the unrelated audit fixes in this PR. * fix(migrations): separate a ServiceNow write body from a projection by parsing The guard tested for a `{` or `[` prefix, so a stored scalar body — `true`, `"short_description"`, `42` — read as a field list and was promoted onto `readFields`, where it would go out as sysparm_fields. A Create/Update Record body is JSON and a projection is a bare comma-separated field list, which is never valid JSON, so parsing is the whole test rather than a guess at its opening character. Ambiguity still resolves to "not a projection", leaving the value where the Create/Update control owns it. * test(connectors,credentials): tie two assertions to what they actually prove - Webflow: a non-array collections envelope reaching `for...of` throws, which is the intended loud failure. Assert the spec-mandated TypeError plus a single request and no write-back, rather than matching V8's wording. - Credentials: the second guard test cannot observe "not deleted" — the proxy driver replays canned rows — so name it for what it does verify, that the reference check carries no workspace predicate and an empty RETURNING logs nothing. Making the driver decide the outcome would fake the database. - Drop `vi.importActual`; a plain `drizzle-orm/pg-proxy` import works now that `drizzle-orm` is un-mocked. * fix(migrations): identify a ServiceNow projection by its own shape Recognising a write body was the wrong way round. A saved body is not always well-formed: it can be a half-typed draft or carry an unquoted block reference, so neither "opens with a brace" nor "fails to parse as JSON" identifies one — and a body misread as a projection is moved to readFields with its original key dropped, losing the draft. Match the projection instead: a comma-separated list of ServiceNow field names, which are word characters plus the dot of a dotted walk. A brace, quote, colon, angle bracket or interior space fails that shape. Parsing then removes the bare scalars that satisfy it by accident.
…e key (#6789) A mothership chat attachment is minted with the same storage key shape as a workspace file — `resolveUploadStorage` calls `generateWorkspaceFileKey` for `mothership_attachment` — but its row is written with `context = 'mothership'`. The serve route branched on the key prefix alone, so every attachment entered the workspace-file use case, which matches on `context = 'workspace'` and resolved it to nothing: `{"error":"FileNotFoundError","message":"File not found"}` for every chat thumbnail and click-through. The `context=mothership` query param on the serve URL is decorative; the route never read it. Serve now resolves the storage context from the key's stored binding, which is server-authored at upload and the only thing that separates the two, and passes it into the cloud and local handlers instead of re-inferring. Genuine workspace files still go through the authorized use case. `verifyWorkspaceFileAccess` takes the context too, so an attachment authorizes from its database row rather than falling through to storage-object metadata. A soft-deleted attachment is now denied, matching workspace files. The route tests are what let this ship: they mocked `inferContextFromKey` to return 'mothership' for a `workspace/…` key, which it never does. With the mock made honest, nine of them fail against the old route.
`sim.ai` answers /api/** with a 301 to `www.sim.ai`, and the client refuses to follow redirects — a 301 rewrites a POST into a bodyless GET, so following one turns a write into a silent no-op and hands the API key to whatever host Location names. Defaulting to the apex therefore failed every command for anyone who never set an endpoint. Before the refusal shipped it was quieter and worse: reads succeeded while writes did nothing. Also trims the provider catalogue from eleven inferred columns to seven. `docsUrl`, `helpText`, `requiresClientGeneratedCredentialId` and the nested `fields` are what you read once you have chosen a provider, not what you scan to choose one, and they pushed the table well past a terminal. Both ids stay: `credentials connect` names an OAuth provider by `serviceId`, `credentials create` matches a service account on `providerId`.
…PI (#6790) * fix(sap_concur): align the integration with SAP Concur's documented API Validated all 70 tools, the block, and both proxy routes against SAP's published API docs. Auth: - add the password and companyUuid to the token cache key so a request with the wrong password can no longer be served a cached token minted from someone else's - wire the documented company-level flow (username = company UUID, credtype = authtoken) so companyUuid actually scopes a token - expand the datacenter allowlist to the documented set (adds glz, apj1, usg, the impl hosts, and the www- twins) and drop the undocumented cn host; validate the returned geolocation by shape instead of membership - coalesce concurrent token fetches so a fan-out mints one token - forward Retry-After so 429 retries pace off Concur's own hint - handle the errorMessageList, SCIM detail, and legacy Error.Message shapes instead of falling through to a generic HTTP message - pin redirects and cap the response body Block: - collapse six contextType subBlocks that disagreed on their default, so a new block no longer seeds MANAGER for every operation - clamp contextType to each operation's documented set - stop requiring a userId and contextType that the default operation's tool does not accept, and scope the receipt fields to the upload ops - reach six params that had no subBlock, and pass userId on travel request updates so a stale value cannot impersonate Tools: - correct response shapes that resolved to undefined: budget headers, budget categories, allocations, receipts, SCIM nextCursor, and the delete endpoints that return a bare boolean - use the Travel Request Amount schema (currency, not currencyCode) - narrow the four XML-only travel tools to a documented string payload and request application/xml - surface real errors instead of a JSON parse failure when the proxy returns a non-JSON body - cap receipt uploads at the documented sizes before downloading Adds 106 tests covering the token cache, geolocation validation, path traversal, and error extraction. * fix(sap_concur): drop the removed forwardId subblock via a migration Removing the `forwardId` subblock without a migration entry breaks deployed workflows that still carry a value under that key. It fed a `concur-forwardid` request header that is documented nowhere in Concur's Receipts v4 or Image v1 references, so it was never honored. There is no replacement subblock and the value is an opaque caller-chosen string rather than a secret, so it is dropped outright. * fix(sap_concur): stop swallowing upload response-read failures The upload route caught every error from the bounded response read and continued down the success path, so a size-limit breach or a stream failure surfaced as an upstream success with a null or header-only body. Concur returns Content-Length: 0 on a successful image-only upload, and readResponseTextWithLimit already returns an empty string for that without throwing, so dropping the catch keeps the legitimate empty-body case working while letting real read failures reach the route's handler. * fix(sap_concur): unblock company auth and correct the body wand prompt The password grant marked username required, so the company-level flow — which sends the company UUID as the token username and has no user login — could not be configured at all, even though the request schema and token fetch already accept companyUuid without a username. Username is now optional for that grant and the server-side check reports which of the two is missing. Relabels the password and companyUuid fields to say what they carry in the company flow. The shared body wand prompt also still described several payloads the way they looked before this branch: quick expenses in PascalCase rather than v4 camelCase, travel requests and expected expenses using currencyCode where the Request v4 Amount schema uses currency, the standard SCIM SearchRequest URN instead of Concur's, startIndex as a search parameter when it is unsupported, and a cash advance shape that does not match the documented request. A wand-generated body was therefore rejected for most of the create operations it covers. * fix(sap_concur): keep Concur's status when an error body fails to read Removing the blanket catch from the upload read fixed one failure mode and introduced its inverse: a cap breach or stream error while reading a non-success body threw before the route reached the branch that preserves Concur's status, so an upstream 4xx surfaced as a Sim 500 and could trigger a retry the caller should not make. Both routes now split the two cases. On a success status the body is the result, so a read failure still propagates. On an error status the body only supplies the message, so a read failure resolves empty and the upstream status survives, with the message falling back to the generic HTTP-status form. Adds 21 tests covering both helpers over success, error, empty-body and boundary statuses; inverting the status check turns 14 of them red.
…y refusal (#6792) Sends a `User-Agent` on both request paths. Without one a CLI request is indistinguishable from any other API traffic, so a bug that reproduces on a single CLI version cannot be found in the server's own logs; the runtime and platform ride along because they are the first things asked about a transport failure only some users hit. The version moves into its own module so the HTTP client can read it. It could not import `program.ts` — program builds the commands, which reach the client — and duplicating the manifest read would let the two disagree. Also recognises `PRINCIPAL_KIND_NOT_PERMITTED`. The same refusal is raised at two layers under two codes, and only the workspace-key one was matched, so the audit-log commands reported "Principal kind workspace_api_key cannot perform operation audit_logs.list" — accurate, written for a server log, and missing the one sentence that tells the reader a personal key resolves it.
* fix(cli): keep a path prefix in the login URLs
`sim login` built both of its URLs with `new URL('/path', endpoint)`. A
leading-slash path is absolute, so it resolves against the endpoint's origin
and drops any path the endpoint carries: a deployment served at
`https://host/sim` sent the browser to `https://host/cli/auth` and polled
`https://host/api/cli/auth/poll`, neither of which exists there.
Every other command builds its URL by concatenation and was unaffected, so
the endpoint looked correct and login alone failed.
Both now go through the client's `buildUrl`, which is exported for it rather
than duplicated — one URL builder for the whole CLI is the point, since two
of them is how the halves drifted apart. Its TSDoc now names the trap.
* test(cli): restore fetch with spyOn so the stub cannot outlive its test
vi.stubGlobal is not undone by restoreAllMocks, so the completed-auth
response would have leaked into whatever ran next. The rest of this file
already spies on globalThis.fetch, which the existing teardown restores.
* feat(files): support zip extraction * fix(files): harden zip extraction safety * fix(files): batch extraction notifications * fix(files): defer rollback storage cleanup * fix(files): restore reliable drag uploads * fix(files): use explicit archive extraction route * fix(ci): account for archive extraction route * refactor(files): bound every archive extraction and trim the extractor's option surface `maxMaterializedItems` was opt-in, so only the new unzip route bounded its output tree — the copilot `materialize_file` and `POST /api/tools/file/manage` extract paths had no cap on folder creation at all. An archive within MAX_ARCHIVE_ENTRIES can still imply far more folders than files, so the cap now defaults to MAX_WORKSPACE_FILE_BULK_AFFECTED_ITEMS and applies to all three callers. `materializedRootFolderCount` was a hand-maintained number that had to agree with what an opaque callback would create, and the callee could not check it; drift surfaced only as an over-limit archive slipping past the cap. It is now derived from whether `prepareRootFolder` ran, so the contract is just "the callback creates exactly one folder". Also single-sources the ArchiveError -> HTTP status map (it was copied into both the internal error policy and the tools route), drops IdempotencyService config that only `executeWithIdempotency` reads (the extraction lease uses atomicallyClaim/release, so no result is ever stored), hoists the duplicated predicates in purgeCreatedWorkspaceFile and archiveWorkspaceFileFolderIfEmpty so a lock and its write cannot diverge, and names UPLOAD_SESSION_LOCAL_PUT_MAX_BYTES rather than overloading the multipart part size as the local single-PUT ceiling. Adds coverage for the two guards nothing exercised: the re-validation of the segments `prepareRootFolder` actually returned, and the default cap applying with no caller opt-in. UI: the drop overlay used --surface-4 unconditionally, which renders grey over the light-mode canvas; matches the canonical overlay's --white/dark:--surface-4 and swaps arbitrary px type sizes for named tokens. * fix(files): bound the extraction write loop so it cannot outlive its lease Cursor Bugbot flagged two related holes, both rooted in the write loop being unbounded: 1. `maxDuration` is a Next.js route-segment config that serverless platforms enforce and self-hosted deployments do not. A slow extraction (up to 1000 sequential uploads) could therefore outrun the six-minute lease, and `IdempotencyService` reclaims an expired in-progress claim — so a second unzip of the same archive could start beside the first. 2. Nothing rolls back a process killed mid-pass-2, so a timeout stranded the destination folder and every file written so far. `decompressArchiveBufferToWorkspaceFiles` now takes an `AbortSignal` and checks it between entries in both passes, and the extraction use case supplies a 180s deadline. The abort unwinds through the existing all-or-nothing rollback, so the work stops on our terms with the tree cleaned up, well inside both the route's 300s budget and the 360s lease. That closes (1) outright — the holder can no longer outlive its lease on any platform — and converts (2) from a stranded partial tree into a clean rollback for the slow case that actually triggers it. A SIGKILL still cannot be caught; that needs a durable job and is out of scope here. The overrun surfaces as a caller-fixable 413 naming the archive rather than an opaque 500 from the raw DOMException. * fix(files): only remap the deadline abort itself, and stop overclaiming rollback Two follow-ups on the budget deadline, both reported by Cursor Bugbot: `deadline.aborted` stays true for the rest of the request once the timer fires, so it cannot decide whether *this* error was the abort. An `ArchiveError` or storage failure thrown mid-entry after the timer fired was being relabelled as a timeout and returned as a 413, hiding the real cause. The catch now matches the thrown value against `deadline.reason` — `throwIfAborted()` throws exactly that object, so the check is identity-exact and cannot capture an unrelated failure. The message also claimed a rollback that has not necessarily happened: the budget covers the archive download too, so it can fire before the first write, when there is nothing to roll back. It now says the unzip was cancelled and claims nothing about what was written. Including the download in the budget is deliberate — the lease it has to fit inside starts earlier still — so the TSDoc says that rather than "the extraction itself". --------- Co-authored-by: Waleed Latif <walif6@gmail.com>
…nt (#6794) * fix(sap_concur): key the token cache with an HMAC and document rate casing The cache key hashed a user-chosen password with a bare SHA-256. The key never leaves the process, but a password is low-entropy enough to brute-force out of a plain digest if one ever reached a heap dump or a debug log, which is what CodeQL flags. Keying the digest with a server-side secret makes it useless without that secret. A password-hashing KDF would be the wrong tool here: this runs on every token fetch, and the goal is collision-free partitioning rather than verification of a stored credential. The body wand prompt also claimed every payload family is camelCase. Exchange rate uploads are the exception — they take a snake_case currency_sets array of from_crn_code, to_crn_code, start_date and rate — and that operation is in BODY_OPS, so the blanket claim produced bodies Concur rejects. * fix(sap_concur): wire the travel request sendback comment through the block move_travel_request accepts a documented query comment that Concur applies to the sendback action, but the params branch never passed it and the block's only comment field is gated to create_report_comment, so the value was unreachable from the UI. Uses a dedicated sendbackComment subblock rather than widening the existing comment field: that one is required for create_report_comment while this is optional, so sharing an id would both clash on required-ness and let a value bleed between the two operations. * fix(sap_concur): gate the sendback comment and merge duplicate TSDoc The sendbackComment field was conditioned only on the operation, so it rendered for submit, approve, cancel and every other workflow action even though Concur applies the comment to sendback alone. It is now gated on the action as well, and the params branch only forwards it for sendback so a value retained from an earlier sendback cannot ride along once the field is hidden. Also folds the two consecutive TSDoc blocks left above tokenCacheKey into one. Only the nearest block binds to the declaration, so the separator and collision reasoning in the earlier block was detached.
…idth (#6795) Two loose ends from the command-surface audit. `--help` stated where the profile files live and named only `~/.sim`, so it was wrong for anyone who had set `SIM_CONFIG_DIR` — which every CI job pointing the CLI at a scratch directory has. The variable is documented in the README and the guides; help was the one place that omitted it. `knowledge search` printed the raw similarity double, `0.2818957269585687`: a nineteen-character column whose last dozen digits cannot separate one result from another. A `score` format fixes it to four decimals, the width `cost` already uses, so the column stays put down the page. `json` and `yaml` still carry the full double, which is what a script compares.
* feat(secrets): add optional descriptions to workspace secrets Workspace secrets already have a backing credential row with a description column, but nothing surfaced it. Teammates had no way to record what a secret is for. - Add a Description field to the secret detail page, matching the integrations credential page, gated on workspace-secret admin - Fold the value and description editors into one Save/Discard pair and one unsaved-changes guard; two guards cannot coexist, since each seeds its own same-URL history entry - Match descriptions in the secrets settings search - Expose description on GET/PUT /api/v2/secrets and in the CLI Descriptions are workspace-only: env_personal credential rows are per-workspace mirrors of one user-global secret, so one saved there would exist in a single workspace, and a personal secret has no teammates to inform. The API rejects a description on personal scope rather than silently dropping it, and omitting it on PUT leaves any existing description untouched so a value rotation cannot erase it. * fix(secrets): address review findings on secret descriptions - Patch the credential detail cache optimistically on update. `onMutate` cancelled the detail query but only patched the lists, so a detail-backed editor stayed dirty after a successful save until the refetch landed — long enough for Discard to restore the pre-save value over the committed one, and for Back to open the unsaved-changes guard. - Memoize `useSecretValue`'s returned callbacks and object, per the hook convention, so the composed form's save/discard stop churning per render. - Reject a description on a personal secret in the domain layer rather than only at the v2 boundary. The internal credential update path accepted one for any type, writing data every reader hides. - Normalize an empty description to null so the API and UI agree. - Correct the secrets documentation, which described a Display Name field the detail view does not have and omitted the scope rule. - Drop the CLI's copy of the 500-character bound; it can't import the contract, so a copy only drifts from the message the API already returns. - Collapse a redundant save guard and align the description write gate with the render gate. Leaves the integrations credential page byte-identical to staging. * fix(secrets): keep the API docs example and CLI column order stable Backward-compatibility fixes for anyone who never sets a description. - Move the blank-to-null normalization out of the contract and into the route. A Zod `.transform()` on any property drops the whole request schema's OpenAPI examples, which had silently removed the Set Secret request example from the published docs. - Append the CLI `description` column instead of inserting it before `updated`. `--output text` is positional, so inserting would shift every field an existing script cuts. - Reject a description on a personal secret with a message that says so, rather than dropping the field and falling through to the generic "no updatable fields" error.
* fix(cli): bound, trace, and explain the requests the CLI makes Four transport gaps, all of which failed silently. A request had no timeout, so a connection that was accepted and then never answered hung the terminal indefinitely. `SIM_TIMEOUT_SECONDS` now bounds one, defaulting to 3600s — deliberately above every timeout the server itself applies, since a synchronous workflow run is allowed 3000s on a paid plan and a tighter default would abort real work and report it as a transport failure. `0` removes the bound, for a self-hosted deployment that runs executions without one of its own. The caller's abort signal is composed with the timeout rather than replaced, so neither masks the other. Node ignores HTTP(S)_PROXY unless NODE_USE_ENV_PROXY opts in, and only from v22.21 and v24.5, so on a network that reaches the API only through a proxy every command failed to connect while the variable that would have fixed it was already set. The CLI cannot enable that from inside the process — Node reads it at startup — so it says what to do rather than bundling an HTTP stack for a setting the platform now owns. An API key was sent to any http:// endpoint with no signal. Now a warning, not a refusal: http is the documented way to reach a local dev server, and a deployment terminating TLS at a gateway is real. Loopback stays silent. `SIM_DEBUG=1` traces method, URL, status and duration. Bodies and headers are deliberately absent — the request carries the API key, and `secrets set` carries the secret itself. All four write to stderr, so a piped stdout stays parseable. * fix(cli): make the request bound safe on every runtime it supports Two ways the new timeout could fail before the request was made. `AbortSignal.any` arrived in Node 20.3 and this package supports Node 20, so composing a caller's abort signal with the timeout threw a bare TypeError on the earliest 20.x releases. It is now used when present and composed through an AbortController when not. `AbortSignal.timeout` rejects a fractional millisecond outright, and past 2^31-1 ms it does not fail at all — it clamps to 1ms, so the longest timeout anyone asked for became the shortest. The value is now rounded and refused above what Node can actually wait, pointing at 0 for an unbounded wait. Also unstubs env vars between tests: `stubEnv` is not undone by `unstubAllGlobals`, so a SIM_TIMEOUT_SECONDS set for one test configured every test after it. * fix(cli): correct the proxy version table, and classify a timeout mid-body `runtimeCanProxy` treated any release between 22 and 24 as capable, so on Node 23 — which reached end of life before the backport — a configured proxy was ignored and the CLI stayed silent about it, which is the exact failure the warning exists to report. The table is now the two lines that shipped the support, and anything after them. `AbortSignal.timeout` keeps firing after `fetch` resolves, so a bound that elapsed while the body was still being read — a large `files get` — escaped the client's own handling and printed a raw TimeoutError stack. The top-level handler now names it, which covers the streaming path as well as the JSON one. A user's own Ctrl-C raises AbortError and is deliberately left alone. * fix(cli): report a timed-out download as a timeout `files get --output-file` streams the body to disk, and `streamToFile` converted anything the stream threw into a write failure. So a request bound elapsing mid-download read as `Could not write <path>: ...`, sending the reader to check permissions and free space for a timeout they can raise, and hiding the one instruction that resolves it. The predicate and that instruction now live beside the timeout that raises them, so the client, the top-level handler and the download path all say the same thing. The wrapping stays where it is: the staged-download cleanup runs off that failure, and rethrowing past it would leak the temporary directory. * fix(cli): keep a sub-millisecond timeout bounded Zero is how this function says "no bound", so rounding a positive SIM_TIMEOUT_SECONDS down to zero inverted the request: anything under 0.0005s asked for the shortest possible timeout and got none at all, leaving a stalled request to hang. Introduced by the rounding that fixed the fractional-millisecond rejection. Floored at 1ms for every positive value; only a literal 0 still disables.
* feat(jotform): trigger a workflow on every new form submission
Jotform's only webhook event is a new submission, so the block gets one
trigger. Deploying it registers the callback on the form through the API
and undeploying removes it again.
Two things about this provider needed handling:
Jotform posts submissions as multipart/form-data, which the shared webhook
body parser did not read — the delivery died as a 400 before any handler
saw it. The parser now flattens a multipart body the same way it already
flattens a urlencoded one, reducing an uploaded part to its filename so a
stray file cannot inflate the execution input.
The form's webhooks are identified by their position in the form's webhook
map, so an id captured at registration goes stale the moment any other
webhook on that form is removed. Nothing persists it; cleanup re-resolves
the id by matching the callback URL. Registration checks the same way,
because Jotform answers a rejected request with the unchanged list rather
than an error.
Answers are exposed as the parsed `rawRequest` rather than re-keyed by
question label — the labels are not unique, and the payload shape is only
documented as the raw q{qid}_{slug} map.
The trigger's region field is named `apiRegion` so it does not collide
with the block's own advanced-mode `region`.
* fix(jotform): make webhook registration idempotent and URL matching tolerant
Validated the trigger against Jotform's API reference and a captured
delivery (zulip's multipart fixture), which confirmed every mapped field —
formID, submissionID, formTitle, username, ip, type, pretty, rawRequest —
and turned up three things worth correcting.
Jotform keeps a form's webhooks as a plain list and does not treat the URL
as a key, so posting one it already holds leaves the form delivering every
submission twice. Registration now consults the list first and only posts
when the URL is absent. The documented POST sample returns the new entry as
"0", renumbering the rest, which is further reason nothing persists an id.
URL matching no longer lets a trailing slash decide the outcome. Jotform
stores the URL verbatim in every sample seen, but an exact match failing
would hard-fail deploy, and Pipedream's client normalizes the same way.
The rawRequest description claimed the field holds the submitted answers.
A real payload also carries slug, buildDate, submitSource and
jsExecutionTracker, and a file answer appears under the bare slugified
label as upload URLs rather than under a q{qid}_ key — which is also why
filtering to q-prefixed keys would silently drop file answers.
* fix(jotform): keep the callback when an active deployment still needs it
Redeploying prepares the replacement webhook row alongside the live one and
a workflow keeps its path across deployments, so both rows resolve to a
single callback on a single form. Registration adopts the callback already
present instead of posting a duplicate, which left the retired row's
cleanup deleting the one the new row had just adopted — the trigger went
silent after a redeploy that changed the trigger config.
Teardown now skips when another webhook row belonging to an active
deployment resolves to the same form and callback URL, matching how the
Telegram handler skips deleteWebhook while an active deployment still uses
the same bot. A genuine undeploy has no such row and still cleans up.
…27.0.0.1] (#6799) * fix(email): greet SMTP relays with a qualified hostname instead of [127.0.0.1] Nodemailer derives the EHLO greeting from os.hostname() and substitutes the address literal [127.0.0.1] whenever that name contains no dot. Kubernetes pod hostnames never contain one, so every k8s deployment introduced itself to the relay as loopback and strict relays refused the session before any mail moved. Send the domain the app is served from instead, as RFC 5321 4.1.4 asks, with SMTP_EHLO_NAME to override it for relays that expect a different identity. * fix(email): parse EHLO address literals and drop a port from the app domain Review round 1. The bracketed branch matched a character class rather than an address, so [::::] and [13] reached the relay as a greeting it would refuse. Parse the address with node:net instead, which also admits the RFC 5321 IPv6: form. getEmailDomain reports a URL host, so a deployment served on a non-default port failed the qualified-name check and fell back to nodemailer's default — [127.0.0.1] again on Kubernetes, the exact failure this change exists to fix. Strip the port before validating. * fix(email): accept any casing of the IPv6 literal tag, and stop owning SMTP_EHLO_NAME in setup Review round 2. RFC 5321 tags the IPv6 address-literal form, and RFC 5234 makes ABNF string literals case-insensitive, so [ipv6:2001:db8::1] is as valid as [IPv6:...]. The exact-prefix check routed it to isIPv4 and discarded it. Drop SMTP_EHLO_NAME from the email capability's optional fields. SMTP_SECURE, the same kind of optional transport knob on the same provider, is not modelled there either, and claiming the field obliged the setup wizard to prompt for it — a field whose entire purpose is to stay unset now that the default is right.
…xt (#6803) Follows the serve fix by stating the contract it relies on, so the next reader does not re-derive module ownership from the key prefix. The prefix is authoritative for where the bytes live — bucket and tenant — and nothing more. Which module owns an object is `workspace_files.context`, which is server-authored like the key but, unlike the key, mutable: a chat attachment becomes a workspace file when `materialize_file` flips that column, and encoding a mutable fact in an immutable key would mean copying the bytes on every such transition just to restate them. `resolveTrustedFileContext` claimed the prefix was flatly authoritative. That claim is what made routing on it look safe. It is now scoped to what it actually defends — a caller-supplied context can still never relabel a private key — and `resolveStoredFileContext` is documented as the sanctioned way to ask who owns an object rather than as a workaround. `verifyWorkspaceFileAccess` resolved its binding with the lookup filtered to `context = 'workspace'`, so an attachment missed the row and fell through to object metadata, which cannot see a soft delete. It now matches either workspace-scoped context, which is also what every caller already wanted: the LLM-attachment and presigned-URL paths pass 'workspace' for attachment keys today. A soft-deleted attachment is now denied on all of them. The parse route carried the same gate and labelled parsed attachments with the raw storage segment instead of the uploaded filename. Module-scoped filters are deliberately untouched: the Files module, its folder manager, forking and the workspace-file use cases all match `context = 'workspace'` because they mean the Files module, not the bucket.
…get value (#6787) * fix(forking): stop a parent re-pick blanking a dependent's stored target value A dependent selector (a sheet under a spreadsheet, a label under a mailbox) is invalidated when its parent is re-picked, because the stored child no longer exists under the new parent. That invalidation was recorded by writing an empty string into the in-session override map — the same value the user's own "clear this field" produces. The two are not the same thing, and the map is submitted verbatim and written into the target workflow's configuration, so an invalidated field cleared the target's real stored value. The sharpest case is an undo. Re-pick a parent away from its original target, then back. The parent nets out unchanged, so nothing is remapped and the remap's own clearing pass never runs — but the child is still blank, and that blank lands on a value the user never touched, with nothing in the UI showing it happened. Record the invalidation with a distinct marker instead. It reads as blank in the selector, the in-block chain context, and the Sync gate, so a required invalidated field still blocks Sync and still renders; but it is omitted from the submitted payload rather than sent as empty, so no override is written and the target keeps what it had. A blank the user picked themselves is still submitted and still clears the target. Also: skip the cascade entirely when a re-pick selects the value the field already had, since the selector fires its change handler either way. Fork file copy: a file whose name is already taken in a reused target folder is de-duplicated with the same allocator the ordinary upload path uses, rather than colliding with the folder-name unique index and being dropped from the fork with its blob deleted. Adds hook-level coverage for the submitted payload, which had none. * fix(forking): preserve dependent-chain semantics * fix(forking): preserve edits during fork sync * fix(workflows): clear stale dependent inputs in Mothership edits --------- Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>
…6801) * docs(helm): document null as the way to remove an inherited env key Setting `app.env.KEY: ""` cannot clear a key that `app.envDefaults` sets: the Secret template drops empty values, and the deployment template treats an empty override as "not overridden" and still inlines the default. Helm's own `KEY: null` deletion is the supported mechanism and already works. The empty-string behavior is load-bearing, not a bug — every key under `app.env` ships as a "" placeholder, and ten collide with a real `envDefaults` value (NEXT_PUBLIC_APP_URL, BETTER_AUTH_URL, ...), so "" has to read as "unspecified" or a default install would blank them out. - README: document `null`, with the --reuse-values and Argo CD valuesObject caveats; correct the claim that `app.env` always wins over `app.envDefaults` - values.yaml + self-hosting docs: same guidance where operators look - sim-helm skill: record why an unset list is the wrong shape here - tests: lock in that null removes a key and "" does not * docs(helm): correct the verify command's chart path and scope the required-secret claim - The verify snippet used a `sim/sim` repo alias that this chart never publishes; every other instruction installs from the local `./helm/sim` path, so the command could not run as written - Nulling a boot-critical key only fails at template time with the chart-managed Secret. `existingSecret` mode skips that validation entirely (the chart cannot read a pre-created Secret), and under ESO the key must instead be mapped in externalSecrets.remoteRefs.app * docs(helm): say null must be applied in every layer that sets a key `null` deletes a key from the map it is applied to, not from the pod. A key set in both `app.env` and `app.envDefaults` survives a null on the app.env entry alone — the deployment then inlines the envDefaults value again. Under ESO a retained `externalSecrets.remoteRefs.app` mapping keeps syncing the key regardless of app.env. - README and self-hosting docs: drop the "works in all three secret modes" shorthand and spell out that every layer setting the key must be nulled, including the ESO remote mapping - tests: cover both halves — nulling only app.env restores the envDefault, nulling both actually removes the key - chart 1.5.4; staging took 1.5.3 in the meantime
|
Too many files changed for review (280 files, 100 file limit). Bypass the limit by tagging |
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
PR SummaryLow Risk Overview CLI reference and guides now document the default endpoint as Configuration docs add Jotform integration docs add a webhook trigger section (form submission payload fields). SAP Concur integration docs are expanded to match SAP’s API: OAuth client-credentials wording, optional vs required bodies, XML responses as strings, pagination cursors, SCIM/search payloads, workflow and recall token requirements, and many output field corrections. Platform credentials docs note viewing/editing key/value and workspace secret descriptions. A migrate-application-operation skill checklist line changes the type-check turbo filter from Reviewed by Cursor Bugbot for commit edc25aa. Configure here. |
…ite paths (#6808) * test(table): characterize the single-row route before migrating it The single-row surface (GET/PATCH/DELETE) had no route-level tests despite being the hottest table write path. Pin the behavior it emits today so the migration onto the shared internal route builder is verifiable rather than hopeful. Covers status codes, body shapes, ISO-8601 timestamp serialization, the access level each method demands, collaborator invocation, and the dual-caller wire keying — session callers pass column ids through untouched while internal-JWT callers translate names to ids in both directions. Verified to fail: mutating the wire translator, the deleted count, and the workspace-ownership guard each turn the corresponding tests red. * feat(table): model wire keying and actor attribution on row write use cases The row write use cases assumed every caller speaks column names. That holds for /api/v2, /api/v1 and the Copilot tools, but not for the first-party grid or the internal /api/table routes, which address cells by stable storage id. Feeding id-keyed data through the name remap drops every key it does not recognise — a storage id names no column name — so the write would store nothing and still report success. Make the wire an explicit, required property of the input rather than an assumption. `dataKeying: 'names' | 'ids'` sits alongside `strictWrite` and is required for the same reason: a new write surface must state which contract it publishes. Strictness now means the same thing on either wire — an unknown column id is refused exactly as an unknown column name already was. Single-row writes also gain optional actor attribution, so the acting tab can skip refetching its own write. It is optional and absent by default, so every existing caller keeps broadcasting to all subscribers as before. Only the single-row create, update and delete paths accept it; a batch write is not reconciled locally by the actor and must still refetch. The attribution pin moves with the behaviour: what selects the actor-scoped signal is no longer which file calls it but which surface supplies an actor, so that is now what the test pins. Verified to fail: ignoring the keying discriminator, and dropping actor attribution, each turn the corresponding tests red. * perf(table): remove two round trips from every single-row update A single-cell PATCH spends far more time in sequential round trips to a remote Postgres than in the UPDATE it issues. Prepared statements are disabled for PgBouncer transaction mode, so every await is a full Parse/Bind/Execute. Two of them were avoidable. getRowById issued the row lookup and its executions sidecar in series, but the sidecar is keyed on the row id the caller already supplied, so it never depended on the lookup. Issuing both together makes it one round trip. A miss now pays one redundant sidecar read, which is the rare path and costs no extra wall time. The uniqueness probe ran whenever the table had any unique column, passing the fully merged row, so editing an unrelated cell re-probed every unique column — its own transaction plus one query per column. It is now scoped to the columns the patch actually writes. A merge cannot newly violate uniqueness on a column it leaves alone: that value is the one already stored, and it satisfied the constraint when it was written. Sized before changing: few tables declare a unique column, but write traffic concentrates in the ones that do, so this is the larger of the two savings in practice. Verified to fail: reverting the probe scoping turns the covering test red. * perf(table): start the workspace load with the table load when a workspace is asserted resolveActiveTableContext ran two sequential round trips: load the table, then load the workspace it turned out to live in. When the caller asserts a workspace the second input is already in hand, so both can start together. What makes that safe is unchanged: requireTable still compares the table's canonical workspaceId against the assertion and reports a mismatch as not_found. The table outcome is inspected first and unconditionally, so any path that returns has proven the assertion equal to the canonical id, and a failing workspace load can never replace the concealing not_found. A final identity check on the loaded context restates the invariant at the point of return, so even with the first check removed the function cannot hand back a foreign workspace. Promise.allSettled keeps the discarded branch from surfacing as an unhandled rejection. With no asserted workspace the path stays sequential — the table load is what reveals which workspace to load, so there is nothing to start early. One existing assertion changed: it required the workspace load not to have been issued yet on a mismatched assertion, which is internal sequencing rather than caller-observable behaviour and is definitionally untrue once the loads overlap. The observable half is kept, and two timing tests now cover the sequencing directly. Verified to fail: removing either mismatch check, reading the workspace outcome first, and swapping allSettled for bare awaits each turn the corresponding tests red. * perf(table): read a table and its latest job in one round trip getTableById issued the table SELECT and then awaited latestJobForTable, so every table request paid two sequential round trips. With prepared statements disabled for PgBouncer transaction mode every await is a full round trip, and this loader is on essentially every table route. The job read cannot be skipped: a table's reported rowCount is the stored count minus the job's pendingDeleteRemaining, so dropping it would overstate the count during a pending delete and could wrongly reject inserts as over capacity. An opt-out flag would have made that a caller's trap. Instead the job is read in the same statement, as a correlated jsonb subquery in the select list — the select-list form of a LEFT JOIN LATERAL, which is what drizzle can type here. Output is unchanged for every input. latestJobForTable is deleted rather than left dangling: getTableById was its only caller, and keeping it would have carried a third copy of the exports-excluded / newest-started_at / limit-one rule. mapJobRow is now exported so the batch path and the lateral share one implementation of the doomedCount and pendingDeleteRemaining logic. The batch DISTINCT ON path used by the list endpoint is untouched. Verified to fail: dropping the export filter, reversing or re-keying the sort, dropping the limit, dropping the correlation, loosening either doomedCount condition, and removing the rowCount subtraction each turn tests red. Dropping the lateral from the projection initially survived, because the shared db mock returns queued rows regardless of predicate; a projection assertion now covers it. * chore(table): drop the unused single-row GET contract Authored as groundwork for migrating the internal row routes onto the shared builder, which is not in this change. An exported contract nothing consumes is dead code, so it lands with the migration that needs it instead. * fix(table): resolve strict id-keyed columns through getColumnId assertKnownColumnIds read column.id directly, but a column id is optional — pre-backfill columns have none and are stored under their name, which is why getColumnId exists and is what every other consumer of the schema uses. Such columns still exist, so a strict id-keyed write naming one would have been refused as unknown. Latent today: no surface yet combines dataKeying 'ids' with strictWrite. Fixed before one does. Verified to fail: reading column.id turns the covering test red. * refactor(table): apply review findings from the quality pass Four parallel reviews (reuse, simplification, efficiency, altitude) converged on the same set. Applied: Removed actorClientId entirely. It had no supplier anywhere in the repo, so every call reached signalTableRowsChangedByActor(id, undefined), which is byte-identical to the broadcast it replaced — three optional fields, three verbatim doc blocks and a pin test asserting the empty set, all inert. It belongs with the route migration that supplies an actor. The attribution pin is restored to its original form. The uniqueness probe was only half-narrowed: the patched column list was computed and then discarded, and the probe re-derived every unique column from the full schema. It now receives only the columns the patch touched, so a table with several unique columns runs one query instead of all of them. assertKnownColumnIds hand-rolled an id index and duplicated the sibling assert's message verbatim. It now reuses buildColumnNameById — which already keys by getColumnId, so the legacy pre-backfill column case is handled by the shared helper rather than by a special case here — and both asserts share one message builder. The job field list had become two copies, one drizzle-checked and one an unchecked sql<T> cast that could silently return undefined for a renamed field. The lateral now derives its jsonb pairs from JOB_PROJECTION, which satisfies Record<keyof LatestJobRow, Column>. That compile-time guarantee replaces the runtime drift test it makes redundant. Also: hoisted the id index out of the batch loop to match the names path, dropped a never-supplied parameter, narrowed an over-broad parameter type, removed a dead timer cleanup and its now-unused import, replaced dynamic re-imports with the static one already present, hoisted a repeated stub, and documented why filters need no keying counterpart and how laxness differs between the two wires. Verified to fail: removing a field from JOB_PROJECTION breaks the build in two places. * test(table): share one table-definition fixture factory buildTable was copy-pasted into 13 route test files under app/api/table, each a near-identical TableDefinition literal. A required field added to that type would have failed 13 files individually. packages/testing already owned createTableColumn and createTableRow but no definition factory, and — as it turns out — did not export any of them from the barrel, so they were unreachable from @sim/testing. Adds createTableDefinition beside them and exports all three. The fixture type is a structural stand-in rather than TableDefinition itself, because packages/* must not import from apps/* — the same approach the existing factories in that file already take. No assertion changed. Call sites that varied a field pass it as an override; the four files with several call sites hoist a shared options const and spread it so each call still gets a fresh object. * perf(table): project only the job field the row count needs Both job reads selected the whole payload jsonb, but mapJobRow reads exactly one number out of it, and only for a running delete. The payload also carries the delete job's filter and an unbounded excludeRowIds array, and the latest non-export job is read on essentially every table request — a table that once ran a large delete would ship that id list on every read, forever. LatestJobRow.payload becomes doomedCount, extracted in SQL. Both readers share JOB_PROJECTION so one edit reaches the batch DISTINCT ON and the correlated subquery alike; the compile-time constraint widens to Column | SQL rather than being dropped. Behaviour is identical. `->` keeps the value jsonb, which postgres-js decodes through its built-in JSON.parse handler, so it arrives as a number with no boundary coercion. A null payload, a payload without the key, a non-object payload and an explicit JSON null all collapse to the same `?? 0` the previous optional chain produced. Sized honestly before claiming a win: payloads are small in practice today, so this is defensive rather than impactful — it removes an unbounded growth path, not a measured cost. Verified against a real Postgres, not just the mocked driver: the generated correlated subquery returns doomedCount 12 for a delete job, null for an import job, and a null row for a table with no job. Also from the review pass: re-homes the strictWrite explanation onto rowWriteOptions, where six {@link} references now point; records why replaceProjectedWireRows carries no keying discriminator; notes the one case the uniqueness-narrowing invariant does not cover; pins the lax id-wire passthrough with a test; and renames a parameter that misled once only its keys were read. * chore(table): drop two comments the code already says One restated the identifier below it (checkUniqueConstraintsDb), and one was a section-divider banner in the factories barrel, which the repo's comment convention rules out. Everything else that survives is a why the code cannot express: round-trip rationale, sql.raw input safety, the narrowing invariant and the one case it does not cover. * refactor(table): move the internal row routes onto the application boundary (#6809) * refactor(table): move the single-row route onto the application boundary The hottest table write path authorized in its own handler and queried the database from the adapter — the two things a surface adapter must never do. It now declares itself with defineInternalJsonRoute against the readRow, updateRow and deleteRow use cases: 127 lines instead of 274, with no db import, no drizzle import and no checkAccess. Doing that surfaced why the violation existed. Write-provenance resolution needs the canonical schema to map a caller's column key to the storage column it certifies, and the adapter could only do that because it was already loading the table illegally. The envelope is now split along the real seam: the adapter reads the header and payload field, which is transport, and the use case resolves the selections against the canonical table, which is domain. That split has to preserve a distinction the defaulting logic would erase. An internal caller that sends no envelope stays deliberately untracked; defaulting it to an exact-empty stamp would certify "this write introduced no secrets" on a runtime write that may well have introduced some. Only an interactive caller certifies exact-empty, over the storage columns its write actually persists. Two further changes fell out of it: present() now receives the same { principal, input } pair its sibling hooks responseHeaders and finalizeResponse already got. This route serves a session and a workflow execution on one path and owes them different column keyings, so rendering per caller kind is presentation rather than domain. That was a gap in the builder, not a special case for this route. tableRowWireSchema describes what the single-row routes actually return. The contract claimed a full TableRow, carrying the executions sidecar and Date objects — true of the list and query routes, and never true here. The hand- rolled handler was never checked against its own contract, so the drift was invisible until the builder started validating it. Wire changes, both deliberate and both narrower than before: a cross-tenant table now conceals as 404 where the old blanket handler answered 403, while an in-workspace role denial still answers 403. Nothing in hooks/queries/tables.ts branches on either. Verified to fail: forcing one keying, dropping the actor, pre-resolving the envelope, certifying an untracked internal write, and skipping the bundle completeness check each turn the covering tests red. * refactor(table): move the upsert route onto the application boundary Same shape as the single-row route: declares itself against upsertTableRow, hands the provenance envelope over unresolved, and derives its column keying from the principal rather than assuming one. The keying and presentation helpers the two routes shared are now in row-wire beside the translators they wrap, so a third route does not restate them. Two response details changed on purpose. The row now carries `position`, which the use case always had and this route alone omitted — every other single-row response already returned it, and the contract now describes one shape instead of three. The upsert result also carries read-back provenance, which the route previously assembled for itself. The surface had no route-level tests; it has six now, covering both caller keyings, both operations, and the envelope handover. * refactor(table): move the enrichment-detail route onto the application boundary The last internal adapter that queried the database for itself. It now runs through readTableRowEnrichmentDetail, a new use case that shares tableOperations.readRow — reading a cell's cascade breakdown is a projection of the same row under the same role, not a second semantic operation. Its tests move to the same seam and gain one the old suite could not express: a cross-tenant table now conceals rather than confirming it exists. * fix(table): mirror the storage rule when keying write provenance storageKeyByWireKey mapped an unrecognised id-keyed key to null, but rowDataToStorage persists that key when the caller is not writing strictly. A cell would have been written with no provenance recorded, under a stamp still marked complete — the same failure the bundle completeness check exists to prevent, arriving through the keying map instead of the selection set. The two wires genuinely differ and the code now says so: the name path drops an unrecognised key, so it maps to null; the id path stores what it is given, so every key it sends is a storage key. Unreachable today, since no delegated surface uses id keying and a session bundle is refused earlier. Fixed because the function's stated invariant — that it mirrors how the row data itself is normalized — was not true. Also corrects the delegated principal fixture in these tests, which used a kind that is not in the Principal union, so the subject-id branch was never actually exercised. It is now, and the scope check is asserted to receive the acting principal's own subject id. Verified to fail: restoring the schema-based lookup turns the covering test red. * fix(table): restore executor access to the migrated row routes The migration swapped checkSessionOrInternalAuth for the delegation policy, and that broke every Table block call to these endpoints in two ways at once. The old policy accepted a legacy internal token. The new one requires a delegation token, which the executor only mints when the tool asks for it — and none of the four table row tools did, so get/update/delete/upsert row would each have failed with a 401. The knowledge tools already declare it, because their routes migrated first. Even with a valid token the operations denied the caller: readRow, updateRow, deleteRow and upsertRow ran under a policy whose delegatedServices is ['copilot'], so the executor got a 403. They now use the tool-facing policy that already existed for the group operations. Neither was visible to the route tests, which mock the auth policy wholesale — so the gap is closed at the layer that actually decides: one test pinning that each tool requests delegation and that its operation admits the executor, mutation-verified against both failure modes. Also fixes findings from the review pass: the read surfaces no longer load an executions sidecar none of them put on the wire (two readers rather than a flag, so a caller cannot silently read an empty one); the provenance name index is built once per batch instead of once per row; the uniqueness comment now names the concurrent-insert race as well as the retro-added constraint; and the presenter context gets NoInfer plus a note that the v2 builder passes something different. * fix(table): keep the lock on a 423 and pin the remaining wire changes The rows error policy was built on the concealment base rather than the lock-aware one, so a TableLockedError fell through to the generic handler and the response lost its `lock` field — the only thing that tells a client which lock to clear. A row write is exactly as lockable as a group mutation, so it now shares that base. Also pins the two wire changes the review found undocumented: a mismatched workspace assertion answers 404 rather than 400, which is a superset of the cross-tenant concealment already intended, and an unclassified failure answers the builder's shared "Internal server error" rather than the old per-route text. Both are consistent with the ~80 routes already on this builder; they are asserted so they read as decisions rather than drift. Verified to fail: reverting the policy base turns the lock test red. * refactor(table): apply the quality pass Four parallel reviews (reuse, simplification, efficiency, altitude). The highest-value finding cut against the branch's own purpose: the rows error policy sat in the barrel-exported route-policies module, so its import of the 1,200-line row use-case graph was paid by every one of the ~28 table routes that can never throw a row error — the barrel's import cost went from ~1.1s to ~1.7s. row-route-policies.ts already existed for exactly this and is deliberately not re-exported; the policy now lives there with its v2 sibling. The upsert path still loaded an executions sidecar no surface puts on the wire, and did it inside the write transaction, holding it open for a discarded result. The read path got that fix earlier; the write path next to it did not. rowKeyingForPrincipal fell through to name keying for anything that was not a session. The operation policy admits API-key principals, so the first one to reach these routes would have had every id-keyed cell dropped and the write reported as successful. It is now an exhaustive switch over the two kinds the auth policy yields — which immediately failed three tests using a principal kind that exists nowhere in the repo, so those fixtures are real now too. Also: reuses toWireTimestamp and createUnknownTableRowSecretProvenance instead of re-inlining them; shares one helper for the provenance choice the update and upsert use cases both make; keeps one canonical actorClientId doc with two cross-references; merges two maps keyed by the same string into one; stops round-tripping the principal through the legacy AuthType enum; hoists the presenter function type out of both conditional branches; drops a subsumed test; and freezes the shared locks fixture so a mutating test cannot poison its siblings.
Uh oh!
There was an error while loading. Please reload this page.