Skip to content

fix(knowledge): list knowledge bases on the same authority that creates them - #6770

Merged
waleedlatif1 merged 5 commits into
stagingfrom
fix/kb-not-showing-staging
Aug 17, 2026
Merged

fix(knowledge): list knowledge bases on the same authority that creates them#6770
waleedlatif1 merged 5 commits into
stagingfrom
fix/kb-not-showing-staging

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Knowledge base lists re-derived workspace access from a permissions row, while every other gate in the module (create, read, update, delete, v1 middleware) resolves it through the shared resolver that also honors organization-admin inheritance. An org admin with no explicit workspace permission row could create a knowledge base and never see it listed — the row query filtered it out after authorization had already passed. Tables and files carry no such join, which is why only knowledge was affected.
  • Both list surfaces now read the workspace's rows after authorizing, through one listWorkspaceAndLegacyKnowledgeBases. Legacy workspace-less bases keep riding along, since they belong to no workspace and are reachable nowhere else.
  • Opening a document flashed "Document not ready" before the chunks appeared: the row builder rendered the loading state as a status claim. It now renders nothing until the document is known.
  • A partial upload failure skipped every cache invalidation, so the list stayed missing rows the server had already created.
  • Document and chunk context menus captured the row they opened on, so Enable/Disable went stale under the list's own polling. They hold an id and resolve against live data.
  • The action bar's "Select all"/"Clear" links used --brand-primary, which is defined nowhere — they fell back to currentColor and were invisible against the text beside them.
  • Cleanup across the module: mutations no longer invalidate the whole detail subtree for single-document writes, dead hook surface removed, page folded into the document list's param group so a search resets pagination in one debounced write, icons import from @sim/emcn/icons, and the two list queries share one projection instead of being near-copies.

Type of Change

  • Bug fix

Testing

Tested manually. bun run type-check, bun run lint, bun run check:audits (29 audits), and the knowledge module test suite (67 files, 651 tests) all pass. New tests cover the org-admin list path, the two-source merge, and the partial-upload invalidation; each was verified to fail against the old code.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…reates them

GET /api/knowledge authorizes the session against the canonical workspace and
then re-derives access inside the row query from a `permissions` join. Those two
no longer agree: workspace `admin` can come from an organization role alone, with
no workspace permission row behind it. Such a caller passes authorization, creates
a knowledge base, and then sees an empty list forever — the row is filtered out by
the join. Tables and files carry no equivalent join, which is why only knowledge
is affected.

Read the workspace's own rows through `getWorkspaceKnowledgeBases` once the
operation is authorized. The caller-scoped query stays only on the path with no
workspace to authorize against.
…t query

The module resolves workspace authority one way everywhere — an explicit
permission row OR an organization admin role — except in the listing query,
which joined `permissions` and required a row. Both list surfaces authorized the
caller and then contradicted that authorization: an org admin could create a
knowledge base through /api/knowledge or /api/v1/knowledge and never see it
listed. Tables and files carry no such join.

Replace the caller-scoped query with `getLegacyPersonalKnowledgeBases`, which
answers only for workspace-less bases whose creator IS their only authority, and
have both surfaces read the workspace's own rows through
`getWorkspaceKnowledgeBases` after authorizing. The legacy rows keep riding along
so they stay reachable. The permissions join now appears nowhere in the module,
and the duplicated connector projection collapses onto the shared helper that
enforces the row cap.
…tate bugs

Cleanup pass over the knowledge module — effects, state, memo, callback, React
Query, url-state, emcn, and comments — keeping the fixes that change behavior for
the better and leaving the ones that would change how the UI feels.

Bugs found and fixed:
- Opening a document flashed "Document not ready" for a frame. The chunk-row
  builder rendered the loading state as a status claim: with no document loaded
  yet it fell through to the branch that reports a missing processing status.
- A partial upload failure skipped every cache invalidation, because the throw
  jumped past them, so the list stayed missing rows the server had already
  created. Admission failures create nothing and still skip the refetch.
- The document and chunk context menus captured the row they opened on, so the
  Enable/Disable label went stale under the list's own polling. They hold an id
  and resolve against live data now.
- The action bar's "Select all"/"Clear" links were painted with `--brand-primary`,
  which is defined nowhere: the links fell back to `currentColor` and were
  indistinguishable from the text beside them.

Consistency and weight:
- Mutations no longer invalidate `detail` non-exactly for writes that touch one
  document: that key is the parent of every documents page, chunk page, tag
  definition, and connector row cached for the base.
- Dead hook surface removed (five exports with no consumer, a query instantiated
  only to reach a cache helper, a `goToPage` that only range-checked), unused
  parameters dropped, `getErrorMessage` replacing hand-rolled instanceof checks.
- `page` joins the document list's param group, so a search resets pagination in
  the same debounced write instead of writing the URL on every keystroke.
- Icons import from `@sim/emcn/icons`, the action bar composes
  `chipFilledFillTokens` instead of restating it three times, chunk cells use the
  canonical content-label chrome, and the icon-only buttons have accessible names.
Follow-up from the quality pass. The two list queries had grown into near-copies
of each other — same 14-column projection, same document join, same cap check,
same row mapping — and the workspace-plus-legacy composition was pasted into both
the internal use case and the v1 route, one of which is a surface adapter that
should not be composing domain reads at all.

Both queries now read through one private projection, so a column added to one
list cannot go missing from the other half of the same rendered list, and
`listWorkspaceAndLegacyKnowledgeBases` owns the composition both surfaces call.
That merge also projects connector types once over the merged set instead of once
per source, and skips the copy-and-sort entirely when there are no legacy rows —
the common case.

Also from the review: the chunk-row memo depends on the two primitives it reads
rather than the whole polled document object, the selected chunk resolves in one
scan instead of two, an aborted chunk-search pagination throws instead of caching
a truncated result as complete, upload cache reconciliation no longer delays the
rejected promise, the key-hierarchy rule is stated once on the key factory rather
than six times at its call sites, and `TagDefinition` has one declaration.
@vercel

vercel Bot commented Aug 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Aug 17, 2026 5:14am

Request Review

@cursor

cursor Bot commented Aug 17, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes authorization and listing paths for knowledge bases (including v1) and broad React Query invalidation behavior across document/chunk mutations; incorrect keys could cause stale UI but not data corruption.

Overview
Fixes knowledge base visibility for callers authorized via organization admin (no workspace permissions row): list and v1 routes now use listWorkspaceAndLegacyKnowledgeBases after the same workspace authorization as create, merging workspace rows with legacy workspace-less bases through a shared row projection instead of re-deriving access from a permissions join.

Client/data layer: React Query invalidation is narrowed so single-document/chunk writes refresh document and documentLists siblings, not the whole detail subtree; partial batch uploads invalidate list/detail caches; connector sync invalidates connector keys only. Knowledge-base mutations no longer take workspaceId on the hook (invalidation uses global list keys).

Workspace UI: Document/chunk context menus track ids and resolve live rows under polling; document open avoids a “Document not ready” flash until the document query resolves; KB document page is grouped with search/filter URL state so pagination resets in one debounced write; action bar fixes invisible Select all/Clear links (--brand-secondary) and shared action button styling.

Reviewed by Cursor Bugbot for commit 29ed288. Configure here.

@greptile-apps

greptile-apps Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR aligns knowledge-base listing authorization and data retrieval, while also tightening document cache invalidation and several knowledge-module UI behaviors.

  • Lists authorized workspace and legacy knowledge bases through a shared projection.
  • Invalidates document-list and row caches after document, tag, and chunk mutations.
  • Reconciles caches after partially successful uploads.
  • Resolves polling-sensitive context-menu actions against current row data.
  • Consolidates document search, filtering, and pagination state.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported document-list cache issue is addressed by a prefix invalidation that covers all parameterized document-list queries.

Important Files Changed

Filename Overview
apps/sim/hooks/queries/kb/knowledge.ts The prior stale-list issue is fixed by invalidating both the individual document key and the document-list prefix after every relevant document-scoped mutation.
apps/sim/hooks/queries/utils/knowledge-keys.ts Adds a correctly shaped prefix covering every parameterized document-list cache key.
apps/sim/lib/knowledge/service.ts Consolidates workspace and legacy knowledge-base listing around a shared projection and removes the permissions-row dependency from authorized workspace listing.
apps/sim/lib/knowledge/application/knowledge-bases.ts Updates application-level knowledge-base listing to use the shared workspace-and-legacy retrieval behavior.
apps/sim/app/workspace/[workspaceId]/knowledge/hooks/use-knowledge-upload.ts Reconciles document and knowledge-base list caches when a batch upload partially succeeds.
apps/sim/app/workspace/[workspaceId]/knowledge/[id]/base.tsx Uses live document rows for context-menu actions and groups pagination with document filters.
apps/sim/app/workspace/[workspaceId]/knowledge/[id]/[documentId]/document.tsx Uses current chunk data for actions and suppresses the misleading not-ready status before document data loads.

Reviews (2): Last reviewed commit: "fix(knowledge): refresh the document lis..." | Re-trigger Greptile

Comment thread apps/sim/hooks/queries/kb/knowledge.ts
Comment thread apps/sim/hooks/queries/kb/knowledge.ts
Review caught a regression in the invalidation narrowing: `documents` (the list
pages) and `document` (one row) are SIBLINGS under `detail`, not parent and
child, so scoping a write to the row key left every list rendering the filename,
status, tags, `tokenCount`, and `chunkCount` it had just changed.

The key factory now exposes a `documentLists` prefix and all six document-scoped
mutations invalidate it alongside the row — the chunk mutations included, since
every chunk write moves the parent document's `tokenCount`. `detail` stays
`exact: true` where only the base's own totals move.

Also repoints the shared list-convention test at `getWorkspaceKnowledgeBases`;
it exercised the caller-scoped query this branch removed.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 29ed288. Configure here.

@waleedlatif1
waleedlatif1 merged commit 11fe848 into staging Aug 17, 2026
30 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/kb-not-showing-staging branch August 17, 2026 05:21
waleedlatif1 added a commit that referenced this pull request Aug 17, 2026
#6770 routed GET /api/knowledge through the workspace read, which carried a
10,000-row cap. Staging crossed it, and the first list request after the deploy
returned 500 with "Knowledge base list exceeds the 10000 row limit" — against
data that had served fine for days.

The cap could never have worked. Its throw was guarded by `limit === undefined`,
so it fired only for callers that had NOT asked for a page: exactly the callers
with no cursor to retry with and no way to ask for less. A paged caller never
reached it. It also read one row PAST the cap before throwing, so it refused to
serve rows it had already materialized — most of the memory was already spent.
Soft-delete cleanup reclaims archived rows only past a retention window, and not
at all where none is configured, so a workspace that archives faster than that
window crosses any fixed count on its own.

Remove it. An unpaged read is unbounded, matching the sibling internal lists
(`listTables`, workspace files), and paged callers keep their page. That fixes
the same latent 500 in the archived list, the catalog read, and the VFS name
lookup, which are all unpaged too, rather than only the surface that failed.

Two more of the same shape found while auditing for others:

- `attachConnectorTypes` threw a bare Error above its own cap, on those same
  unpaged callers. Archiving a knowledge base archives its connectors, so the
  growth curve that broke staging could not reach it — but it is the identical
  construct, and the sibling `latestJobsForTables` has no equivalent.
- The VFS path lookup read every knowledge base whose name merely CONTAINED the
  term and then exact-matched in JS, so a single-row lookup scaled with the
  workspace. It now queries the exact name and reads two rows, mirroring
  `findActiveTablesByExactName`.
waleedlatif1 added a commit that referenced this pull request Aug 17, 2026
#6770 routed GET /api/knowledge through the workspace read, which carried a
10,000-row cap. Staging crossed it, and the first list request after the deploy
returned 500 with "Knowledge base list exceeds the 10000 row limit" — against
data that had served fine for days.

The cap could never have worked. Its throw was guarded by `limit === undefined`,
so it fired only for callers that had NOT asked for a page: exactly the callers
with no cursor to retry with and no way to ask for less. A paged caller never
reached it. It also read one row PAST the cap before throwing, so it refused to
serve rows it had already materialized — most of the memory was already spent.
Soft-delete cleanup reclaims archived rows only past a retention window, and not
at all where none is configured, so a workspace that archives faster than that
window crosses any fixed count on its own.

Remove it. An unpaged read is unbounded, matching the sibling internal lists
(`listTables`, workspace files), and paged callers keep their page. That fixes
the same latent 500 in the archived list, the catalog read, and the VFS name
lookup, which are all unpaged too, rather than only the surface that failed.

Two more of the same shape found while auditing for others:

- `attachConnectorTypes` threw a bare Error above its own cap, on those same
  unpaged callers. Archiving a knowledge base archives its connectors, so the
  growth curve that broke staging could not reach it — but it is the identical
  construct, and the sibling `latestJobsForTables` has no equivalent.
- The VFS path lookup read every knowledge base whose name merely CONTAINED the
  term and then exact-matched in JS, so a single-row lookup scaled with the
  workspace. It now queries the exact name and reads two rows, mirroring
  `findActiveTablesByExactName`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant