Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .claude/rules/sim-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ packages/ # @sim/* — audit, auth, db, logger, realtime-protocol
- `apps/* → packages/*` only. Packages never import from `apps/*`.
- `apps/realtime` avoids Next.js, React, the block/tool registry, provider SDKs, and the executor; never add `@/lib/webhooks/providers/*`, `@/executor/*`, `@/blocks/*`, or `@/tools/*` imports to any package it consumes. CI enforces this via `scripts/check-monorepo-boundaries.ts` and `scripts/check-realtime-prune-graph.ts`.

## The `'use client'` server boundary

Every export of a `'use client'` module becomes a *client reference* on the server — server-evaluated code (RSC pages/layouts, `prefetch.ts`, route handlers, block definitions, triggers) can only *render* it as a component or pass it as a prop, never *call* it (doing so throws at runtime, e.g. `tableKeys.list is not a function`; `next build` does not catch it). Keep server-importable query primitives (key factories, fetchers, mappers, constants) in non-`'use client'` modules — see `.claude/rules/sim-queries.md`. Enforced by `scripts/check-client-boundary-imports.ts`.

## Feature Organization

Features live under `app/workspace/[workspaceId]/`:
Expand Down
11 changes: 11 additions & 0 deletions .claude/rules/sim-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,17 @@ Never use inline query keys — always use the factory.

**Every identifier the `queryFn` forwards into the fetch MUST appear in the `queryKey`.** (Query-machinery identifiers — `signal`, `pageParam` — are exempt; they aren't fetch-scoping args.) If the fetch is scoped by `workspaceId`, `cursor`, `limit`, an org id, etc., those values must be part of the key — otherwise distinct fetch args share one cache entry (a cross-tenant / per-param cache collision). The lone exception is a globally-unique id used as the key while a second fetch arg is only an authz scope that cannot collide; annotate those with `// rq-lint-allow: <reason>`. Enforced by the `key-fetch-arg-drift` check in `scripts/check-react-query-patterns.ts`.

## Server-importable query primitives must NOT live in a `'use client'` module

Next.js rewrites **every** export of a `'use client'` module into a *client reference* in the server bundle. Server-evaluated code — RSC `page.tsx`/`layout.tsx`, `prefetch.ts`, route handlers, **block definitions**, triggers/workers — can only *render* such an export as a component or pass it as a prop; **calling** one throws at runtime (`Attempted to call X from the server but X is on the client` — for an object export it surfaces as `X.list is not a function`). `next build` does **not** catch this — only SSR/runtime does.

So any **query-key factory, standalone `requestJson` fetcher, mapper, or constant** that a server module imports must live in a **non-`'use client'`** module:

- key factories → `hooks/queries/utils/<entity>-keys.ts` (see `folder-keys.ts`, `table-keys.ts`, `credential-keys.ts`)
- standalone fetchers/mappers → `hooks/queries/utils/fetch-*.ts` / `*-list-query.ts` (see `fetch-workflow-envelope.ts`, `fetch-credential-set.ts`)

The `'use client'` hook module then imports these back for its hooks. **Never** define a server-imported factory/fetcher directly in a `'use client'` hooks file — it crashes SSR (this caused the tables-page crash). Enforced for prefetch/route/trigger/block files by `scripts/check-client-boundary-imports.ts` (`bun run check:client-boundary`, run in CI). Escape hatch for a genuinely browser-only path: `// client-boundary-allow: <reason>` on the line above the import.

## File Structure

```typescript
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/test-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,9 @@ jobs:
- name: React Query pattern audit
run: bun run check:react-query

- name: Client boundary import audit
run: bun run check:client-boundary

- name: Verify realtime prune graph
run: bun run check:realtime-prune

Expand Down
464 changes: 464 additions & 0 deletions apps/docs/content/docs/en/integrations/gitlab.mdx

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions apps/docs/content/docs/en/workflows/blocks/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
"title": "Core Blocks",
"pages": [
"agent",
"pi",
"api",
"function",
"condition",
Expand All @@ -17,6 +16,7 @@
"human-in-the-loop",
"variables",
"wait",
"credential"
"credential",
"pi"
]
}
6 changes: 3 additions & 3 deletions apps/docs/content/docs/en/workflows/blocks/pi.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Pick the mode with the **Mode** dropdown. The fields below it change to match.
Cloud runs entirely inside a disposable sandbox, so it never touches your machine. It clones the repo, lets the agent work with full read/shell/edit/git, pushes a branch, and opens a PR you review and merge.

- Requires sandbox execution to be enabled (the Cloud option only appears when it is).
- Requires **your own provider API key (BYOK)** — the model key is handed to the sandbox, so Sim never injects a hosted key there.
- Requires **your own provider API key (BYOK)** — the model key is handed to the sandbox.
- Needs a **GitHub token** with permission to clone, push, and open a PR (see [Setup](#setup-cloud)).
- The deliverable is a **pull request** — nothing is committed to your default branch directly.

Expand Down Expand Up @@ -118,7 +118,7 @@ The one case neither layer can rescue is a *first* prompt that already exceeds t

## Setup

### Cloud
### Cloud [#setup-cloud]

Cloud runs in a sandbox image with the Pi CLI and git baked in.

Expand All @@ -128,7 +128,7 @@ Cloud runs in a sandbox image with the Pi CLI and git baked in.
- *Fine-grained:* select the repo, then **Contents: Read and write** + **Pull requests: Read and write**.
- *Classic:* the **`repo`** scope. For org repos, authorize the token for SSO.

### Local
### Local [#setup-local]

1. **Enable SSH** on the target machine (on macOS: System Settings → General → Sharing → Remote Login).
2. **Expose it on a public host.** Sim blocks `localhost`/LAN, so use a TCP tunnel — for example `ngrok tcp 22`, which gives a `host:port` to put in **Host** and **Port**.
Expand Down
1 change: 1 addition & 0 deletions apps/realtime/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
"@sim/logger": "workspace:*",
"@sim/platform-authz": "workspace:*",
"@sim/realtime-protocol": "workspace:*",
"@sim/runtime-secrets": "workspace:*",
"@sim/security": "workspace:*",
"@sim/utils": "workspace:*",
"@sim/workflow-persistence": "workspace:*",
Expand Down
9 changes: 9 additions & 0 deletions apps/realtime/src/bootstrap.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
/**
* Container entrypoint. Hydrates `process.env` from the runtime secret before
* loading the Socket.IO server, whose modules (`@/env`, DB preflight) read env
* at import time. See `@sim/runtime-secrets`.
*/
import { loadRuntimeSecrets } from '@sim/runtime-secrets'

await loadRuntimeSecrets()
await import('@/index')
18 changes: 7 additions & 11 deletions apps/sim/app/api/files/export/[id]/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import { NextResponse } from 'next/server'
import { fileExportContract } from '@/lib/api/contracts/storage-transfer'
import { parseRequest } from '@/lib/api/server'
import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
import { extractEmbeddedImageIds } from '@/lib/copilot/tools/server/files/embedded-image-refs'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import type { StorageContext } from '@/lib/uploads/config'
import { USE_BLOB_STORAGE } from '@/lib/uploads/config'
Expand All @@ -19,9 +20,6 @@ const logger = createLogger('FilesExportAPI')

const MARKDOWN_MIME_TYPES = new Set(['text/markdown', 'text/x-markdown'])
const MARKDOWN_EXTENSIONS = new Set(['md', 'markdown'])
const VIEW_URL_RE =
/\/api\/files\/view\/([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})/gi
const MAX_EMBEDDED_IMAGES = 50

function isMarkdown(originalName: string, contentType: string): boolean {
if (MARKDOWN_MIME_TYPES.has(contentType)) return true
Expand Down Expand Up @@ -82,10 +80,7 @@ export const GET = withRouteHandler(
})
let mdContent = mdBuffer.toString('utf-8')

const imageIds = [...new Set([...mdContent.matchAll(VIEW_URL_RE)].map((m) => m[1]))].slice(
0,
MAX_EMBEDDED_IMAGES
)
const imageIds = extractEmbeddedImageIds(mdContent)

logger.info('Exporting markdown', { id, imageCount: imageIds.length })

Expand Down Expand Up @@ -139,10 +134,11 @@ export const GET = withRouteHandler(
for (const [imageId, asset] of assetMap) {
const escapedId = imageId.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
const replacement = `./assets/${asset.filename}`
mdContent = mdContent.replace(
new RegExp(`/api/files/view/${escapedId}`, 'g'),
() => replacement
)
// Rewrite both embed spellings the extractor resolves to this id — the view URL and the in-app
// `/workspace/<ws>/files/<id>` path — so a bundled asset never leaves a broken link in the export.
mdContent = mdContent
.replace(new RegExp(`/api/files/view/${escapedId}`, 'g'), () => replacement)
.replace(new RegExp(`/workspace/[A-Za-z0-9-]+/files/${escapedId}`, 'g'), () => replacement)
}

const zip = new JSZip()
Expand Down
116 changes: 116 additions & 0 deletions apps/sim/app/api/files/public/[token]/inline/route.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
/**
* @vitest-environment node
*/
import { NextRequest } from 'next/server'
import { beforeEach, describe, expect, it, vi } from 'vitest'

const { mockResolveShare, mockRateLimit, mockValidateAuth, mockDownloadFile, mockResolveImage } =
vi.hoisted(() => ({
mockResolveShare: vi.fn(),
mockRateLimit: vi.fn(),
mockValidateAuth: vi.fn(),
mockDownloadFile: vi.fn(),
mockResolveImage: vi.fn(),
}))

vi.mock('@/lib/public-shares/share-manager', () => ({
resolveActiveShareByToken: mockResolveShare,
}))
vi.mock('@/lib/public-shares/rate-limit', () => ({ enforcePublicFileRateLimit: mockRateLimit }))
vi.mock('@/lib/core/security/deployment-auth', () => ({ validateDeploymentAuth: mockValidateAuth }))
vi.mock('@/lib/uploads/core/storage-service', () => ({ downloadFile: mockDownloadFile }))
vi.mock('@/lib/uploads/server/inline-image', () => ({
resolveWorkspaceInlineImage: mockResolveImage,
}))

import { GET } from '@/app/api/files/public/[token]/inline/route'

const TOKEN = 'tok_share_123456'
const DOC_KEY = 'workspace/ws-1/doc.md'
const IMG_KEY = 'workspace/ws-1/photo.png'
const FILE_ID = 'wf_YwDXi8eWOkTxn0sbgChlB'
const PNG = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00])

const params = { params: Promise.resolve({ token: TOKEN }) }
const req = (q: string) => new NextRequest(`http://localhost/api/files/public/${TOKEN}/inline?${q}`)

const share = {
share: { id: 'sh_1', token: TOKEN, authType: 'public' },
file: { id: 'wf_doc', key: DOC_KEY, workspaceId: 'ws-1', originalName: 'doc.md' },
workspaceName: 'Acme',
ownerName: 'Jane',
}

/** doc bytes embed the image via the view form; image bytes are a real PNG */
function downloadByKey(docContent = `![a](/api/files/view/${FILE_ID})`) {
return ({ key }: { key: string }) =>
Promise.resolve(key === DOC_KEY ? Buffer.from(docContent, 'utf-8') : PNG)
}

describe('GET /api/files/public/[token]/inline', () => {
beforeEach(() => {
vi.clearAllMocks()
mockRateLimit.mockResolvedValue(null)
mockResolveShare.mockResolvedValue(share)
mockValidateAuth.mockResolvedValue({ authorized: true })
mockResolveImage.mockResolvedValue({
key: IMG_KEY,
contentType: 'image/png',
filename: 'photo.png',
})
mockDownloadFile.mockImplementation(downloadByKey())
})

it('serves a same-workspace image referenced by the doc, typed from its bytes', async () => {
const res = await GET(req(`fileId=${FILE_ID}`), params)
expect(res.status).toBe(200)
expect(res.headers.get('content-type')).toBe('image/png')
})

it('serves a key-referenced image', async () => {
mockDownloadFile.mockImplementation(
downloadByKey(`![a](/api/files/serve/${encodeURIComponent(IMG_KEY)}?context=workspace)`)
)
const res = await GET(req(`key=${encodeURIComponent(IMG_KEY)}`), params)
expect(res.status).toBe(200)
})

it('404s when the reference is not embedded in the shared document', async () => {
mockDownloadFile.mockImplementation(downloadByKey('no images here'))
const res = await GET(req(`fileId=${FILE_ID}`), params)
expect(res.status).toBe(404)
expect(mockResolveImage).not.toHaveBeenCalled()
})

it('404s when the referenced file is not in the document workspace', async () => {
mockResolveImage.mockResolvedValue(null)
const res = await GET(req(`fileId=${FILE_ID}`), params)
expect(res.status).toBe(404)
})

it('404s when the bytes are not a renderable image', async () => {
mockDownloadFile.mockImplementation(({ key }: { key: string }) =>
Promise.resolve(
key === DOC_KEY
? Buffer.from(`![a](/api/files/view/${FILE_ID})`, 'utf-8')
: Buffer.from('<svg/>', 'utf-8')
)
)
const res = await GET(req(`fileId=${FILE_ID}`), params)
expect(res.status).toBe(404)
})

it('401s and never reads storage when the share is unauthorized', async () => {
mockValidateAuth.mockResolvedValue({ authorized: false, error: 'auth_required_password' })
const res = await GET(req(`fileId=${FILE_ID}`), params)
expect(res.status).toBe(401)
expect(mockDownloadFile).not.toHaveBeenCalled()
})

it('404s for an unknown or inactive token', async () => {
mockResolveShare.mockResolvedValue(null)
const res = await GET(req(`fileId=${FILE_ID}`), params)
expect(res.status).toBe(404)
expect(mockDownloadFile).not.toHaveBeenCalled()
})
})
99 changes: 99 additions & 0 deletions apps/sim/app/api/files/public/[token]/inline/route.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
import { createLogger } from '@sim/logger'
import type { NextRequest } from 'next/server'
import { NextResponse } from 'next/server'
import { getPublicInlineFileContract } from '@/lib/api/contracts/public-shares'
import { parseRequest } from '@/lib/api/server'
import {
extractEmbeddedImageIds,
extractEmbeddedImageKeys,
} from '@/lib/copilot/tools/server/files/embedded-image-refs'
import { validateDeploymentAuth } from '@/lib/core/security/deployment-auth'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { enforcePublicFileRateLimit } from '@/lib/public-shares/rate-limit'
import { resolveActiveShareByToken } from '@/lib/public-shares/share-manager'
import { downloadFile } from '@/lib/uploads/core/storage-service'
import { resolveWorkspaceInlineImage } from '@/lib/uploads/server/inline-image'
import { serveInlineImage } from '@/app/api/files/serve-inline-image'
import { createErrorResponse, FileNotFoundError } from '@/app/api/files/utils'

export const dynamic = 'force-dynamic'

const logger = createLogger('PublicInlineFileAPI')

/**
* GET /api/files/public/[token]/inline?key=<cloudKey>|fileId=<id>
*
* Cascades a markdown document's public share to the images it embeds, so a logged-out viewer sees them
* instead of broken icons. The share grants the document bytes; this route extends that grant to the
* document's referenced images only, behind three gates that together hold the security boundary:
*
* 1. Referenced-by-doc — the requested key/id must appear in the shared document's current bytes. The
* token is a capability for the document and its embeds, never an arbitrary workspace file.
* 2. Same-workspace — the referenced file must be a `workspace` file in the document's own workspace
* ({@link resolveWorkspaceInlineImage}). This blocks any cross-workspace reference (which an author
* can write but must never resolve) from loading.
* 3. Content-truth — the served content type is sniffed from the bytes, not the client-declared type,
* and only genuine raster images are served. A file spoofing `image/png` while holding HTML/SVG is
* refused rather than rendered inline.
*/
export const GET = withRouteHandler(
async (request: NextRequest, context: { params: Promise<{ token: string }> }) => {
const requestId = generateRequestId()

try {
const limited = await enforcePublicFileRateLimit(request, 'content')
if (limited) return limited

const parsed = await parseRequest(getPublicInlineFileContract, request, context)
if (!parsed.success) return parsed.response
const { token } = parsed.data.params
const ref = parsed.data.query

const resolved = await resolveActiveShareByToken(token)
if (!resolved) {
throw new FileNotFoundError('Not found')
}

const auth = await validateDeploymentAuth(
requestId,
resolved.share,
request,
undefined,
'file'
)
if (!auth.authorized) {
return NextResponse.json({ error: auth.error ?? 'auth_required_password' }, { status: 401 })
}

const { file: doc } = resolved
if (!doc.workspaceId) {
throw new FileNotFoundError('Not found')
}

// Referenced-by-doc gate: the share grants exactly the images the document embeds.
const docText = (await downloadFile({ key: doc.key, context: 'workspace' })).toString('utf-8')
const referenced = ref.fileId
? extractEmbeddedImageIds(docText).includes(ref.fileId)
: extractEmbeddedImageKeys(docText).includes(ref.key as string)
if (!referenced) {
throw new FileNotFoundError('Not found')
}

// Same-workspace gate: resolve scoped to the document's own workspace.
const image = await resolveWorkspaceInlineImage(doc.workspaceId, ref)
if (!image) {
throw new FileNotFoundError('Not found')
}

// Content-truth gate (`sniff`): render only genuine raster image bytes.
return await serveInlineImage(image, { sniff: true })
} catch (error) {
if (error instanceof FileNotFoundError) {
return createErrorResponse(error)
}
logger.error('Error serving public inline image:', error)
return createErrorResponse(error instanceof Error ? error : new Error('Failed to serve file'))
}
}
)
Loading
Loading