diff --git a/MINTLIFY_PORT.md b/MINTLIFY_PORT.md new file mode 100644 index 000000000..af0058a7b --- /dev/null +++ b/MINTLIFY_PORT.md @@ -0,0 +1,38 @@ +# Mintlify documentation port + +This fork includes a community-maintained Mintlify port of the current TanStack documentation in [`mintlify/`](./mintlify). It is not an official TanStack deployment. + +## Scope + +- 16 public TanStack products represented with Mintlify's product switcher +- 2,720 Markdown/MDX pages copied from pinned upstream `main` commits +- all assets found under each upstream documentation directory +- navigation recreated from each product's `docs/config.json` +- generated compatibility pages for examples and API references that TanStack.com builds dynamically +- MIT license copies and commit-level provenance in `mintlify/attribution.mdx` + +Historical version branches are not duplicated; the canonical TanStack site remains the source for archived versions. + +## Validate locally + +```sh +cd mintlify +npx mint@latest validate +npx mint@latest broken-links +npx mint@latest dev +``` + +## Regenerate + +The generator expects current TanStack product repositories in a sibling `upstreams/` directory by default. Override that location with `TANSTACK_SOURCE_ROOT`. + +```sh +TANSTACK_SOURCE_ROOT=/path/to/upstreams node scripts/port-to-mintlify.mjs +``` + +The source mapping and pinned commit SHAs live at the top of `scripts/port-to-mintlify.mjs`. Regeneration is deterministic for a matching set of source checkouts. + +## Deploy + +Connect this repository to Mintlify, enable **Set up as monorepo**, and set the documentation path to `/mintlify`. + diff --git a/README.md b/README.md index f168f491e..94ef81f27 100644 --- a/README.md +++ b/README.md @@ -12,6 +12,9 @@ The home of the TanStack ecosystem. Built with [TanStack Router](https://tanstac ## Development +> [!NOTE] +> This fork also contains a community [Mintlify documentation port](./MINTLIFY_PORT.md) under `mintlify/`. + ### Quick Start From your terminal: diff --git a/mintlify/ai/adapters/acp-compatible.md b/mintlify/ai/adapters/acp-compatible.md new file mode 100644 index 000000000..c2c7539a4 --- /dev/null +++ b/mintlify/ai/adapters/acp-compatible.md @@ -0,0 +1,286 @@ +--- +title: ACP-Compatible Harness +id: acp-compatible-harness +description: "Plug any Agent Client Protocol (ACP) coding agent into a TanStack AI sandbox with one generic harness adapter — no dedicated package required." +keywords: + - tanstack ai + - acp + - agent client protocol + - coding agent + - harness + - sandbox + - custom adapter +--- + +Coding-agent CLIs that speak the [Agent Client Protocol](https://agentclientprotocol.com) (ACP) — `grok`, `gemini --acp`, and others — expose a long-lived JSON-RPC session you can drive from a sandbox. Instead of a dedicated package per agent, `acpCompatible` builds a `chat()` adapter for **any** ACP-compliant CLI: configure how to launch it once, select a model per call, and pass it into a sandbox. + +It is the harness equivalent of the [OpenAI-Compatible adapter](/ai/adapters/openai-compatible). Use it when your agent speaks ACP but has no `@tanstack/ai-*` package. If a dedicated harness adapter exists ([Grok Build](/ai/adapters/grok-build), and others), prefer it — those carry curated per-model metadata and vendor-specific behavior. + +## Installation + +`acpCompatible` ships in `@tanstack/ai-acp`. You drive it inside a sandbox, so install the sandbox package and a provider too: + +```bash +npm install @tanstack/ai-acp @tanstack/ai @tanstack/ai-sandbox @tanstack/ai-sandbox-docker +``` + +## Basic Usage + +Configure the harness once with `acpCompatible({ name, command })`, then select a model per call. `command` builds the shell command that launches the agent's ACP server over **stdio** inside the sandbox: + +```ts +import { chat } from '@tanstack/ai' +import { acpCompatible } from '@tanstack/ai-acp' +import { + createSecrets, + defineSandbox, + defineWorkspace, + githubRepo, + withSandbox, +} from '@tanstack/ai-sandbox' +import { dockerSandbox } from '@tanstack/ai-sandbox-docker' +import { messages } from './chat-context' + +// Configure the "pi" agent harness once: +const pi = acpCompatible({ + name: 'pi', + command: ({ model, harnessCwd }) => `pi --acp -m ${model} --cwd ${harnessCwd}`, + authMethodId: 'pi-api-key', // when the harness advertises an ACP auth method + refusalMessage: 'Pi refused the request.', +}) + +const sandbox = defineSandbox({ + id: 'pi-agent', + provider: dockerSandbox({ image: 'node:22' }), + workspace: defineWorkspace({ + source: githubRepo({ repo: 'owner/app' }), + setup: ['npm install -g pi-cli'], // install the agent CLI into the image + secrets: createSecrets({ PI_API_KEY: process.env.PI_API_KEY ?? '' }), + }), +}) + +const stream = chat({ + adapter: pi('pi-fast'), + messages, + middleware: [withSandbox(sandbox)], +}) +``` + +You get the full ACP flow for free: sandbox resolution, `chat()`-tool → MCP bridging, session resume, permission handling, abort, and AG-UI event translation. + +## One-Shot Usage + +For a single model, skip the harness-factory and build the adapter inline with `acpCompatibleText`: + +```ts +import { chat } from '@tanstack/ai' +import { acpCompatibleText } from '@tanstack/ai-acp' +import { withSandbox } from '@tanstack/ai-sandbox' +import { sandbox } from './sandbox' +import { messages } from './chat-context' + +const stream = chat({ + adapter: acpCompatibleText('pi-fast', { + name: 'pi', + command: ({ model }) => `pi --acp -m ${model}`, + }), + messages, + middleware: [withSandbox(sandbox)], +}) +``` + +## Typed models & options + +Like `openaiCompatible`, you can declare the harness's **models** and its +per-call **options** so the whole thing is type-checked. `models` constrains the +factory's argument; `modelOptions` is a type-only brand (`{} as { … }`, unused at +runtime) describing what `chat({ modelOptions })` accepts. Declared options are +merged with the base ACP options and handed to `command` / `openTransport` as +`ctx.modelOptions`, so you can turn them into CLI flags: + +```ts +import { acpCompatible } from '@tanstack/ai-acp' + +const pi = acpCompatible({ + name: 'pi', + models: ['pi-fast', 'pi-pro'], + modelOptions: {} as { reasoningEffort?: 'low' | 'high' }, + command: ({ model, harnessCwd, modelOptions }) => + `pi --acp -m ${model} --cwd ${harnessCwd}` + + (modelOptions?.reasoningEffort ? ` --effort ${modelOptions.reasoningEffort}` : ''), +}) + +pi('pi-pro') // ok +// pi('pi-ultra') // type error — not in `models` +``` + +```ts +import { chat } from '@tanstack/ai' +import { withSandbox } from '@tanstack/ai-sandbox' +import { pi } from './pi-harness' +import { sandbox } from './sandbox' +import { messages } from './chat-context' + +const stream = chat({ + adapter: pi('pi-pro'), + modelOptions: { reasoningEffort: 'high' }, // typed against the declared options + messages, + middleware: [withSandbox(sandbox)], +}) +``` + +The base options are always available on `modelOptions` regardless of what you +declare: `sessionId` (resume), `cwd`, `authMethodId`, and `permissionMode`. + +## Configuration + +| Field | Purpose | +| --- | --- | +| `name` (required) | Harness label, log prefix, and the `.session-id` CUSTOM event name. | +| `models` | The model ids this harness accepts — declaring them makes `harness('id')` type-safe (unknown ids are rejected). Omit to accept any string. | +| `modelOptions` | Type-only brand for the per-call options accepted via `chat({ modelOptions })`. Declare with `{} as { … }`; merged with the base options and exposed on `ctx.modelOptions` in `command` / `openTransport`. | +| `command` | Build the **stdio** launch command from `{ model, cwd, harnessCwd, sandbox, env, modelOptions, signal }`. Required unless `openTransport` is given. | +| `skillsDir` | The harness's skills directory (relative to the workspace root, e.g. `'.pi/skills'`) — its native convention, like Claude Code's `.claude/skills`. `withSandbox` workspace `gitSkill`s are linked here. Omit and gitSkills are left unlinked (warned). | +| `openTransport` | Open any `AcpSessionTransport` yourself (e.g. boot a `serve` process and connect over WebSocket). Overrides `command`. | +| `cwd` | Working directory inside the sandbox (default `/workspace`). | +| `env` | Extra environment variables for the harness process. | +| `authMethodId` | ACP auth method to select before the session starts. | +| `permissionMode` | `'default'` \| `'acceptEdits'` \| `'bypassPermissions'` (default). | +| `permissions` | `'headless'` (auto-resolve, default) or `'interactive'` (emit approval-requested events for `ask` prompts). | +| `onPermissionRequest` | Custom permission handler; overrides `permissions`/`permissionMode`. | +| `refusalMessage` | `RUN_ERROR` message when the harness refuses a request. | +| `planEventName` | Emit ACP `plan` updates as a CUSTOM event under this name. | +| `emitDiff` | Emit the post-run `git diff` of `cwd` as a `file.changed` CUSTOM event (off by default). | +| `onExtNotification` | Handle vendor `_x/…` JSON-RPC notifications. | +| `buildPrompt` | Override how chat history maps to the harness prompt. | + +## WebSocket and Custom Transports + +Some harnesses run an ACP server you reach over WebSocket rather than stdio (the `grok agent serve` pattern). Open the transport yourself with `openTransport` — it receives the same context and returns an `AcpSessionTransport`. Put all teardown in the returned transport's `dispose`: + +```ts +import { acpCompatible, startAcpServerInSandbox } from '@tanstack/ai-acp' + +const myAgent = acpCompatible({ + name: 'my-agent', + openTransport: async ({ sandbox, model, harnessCwd, signal }) => { + const server = await startAcpServerInSandbox(sandbox, { + port: 9100, + cwd: harnessCwd, + command: `my-agent serve --bind 0.0.0.0:9100 -m ${model}`, + readyMarker: 'listening', + buildWsUrl: ({ channel, port }) => + `${channel.url.replace(/^http/i, 'ws')}:${port}`, + ...(signal ? { signal } : {}), + }) + const ws = await server.connect(signal) + return { + kind: 'stream', + stream: ws.stream, + dispose: async () => { + ws.close() + await server.dispose() + }, + } + }, +}) +``` + +## Permissions + +Inside a sandbox the sandbox itself is the security boundary, so the default `'headless'` strategy with `permissionMode: 'bypassPermissions'` lets the agent edit files and run commands without prompting. To surface tool approvals to a client instead, switch to `'interactive'`: + +```ts +import { acpCompatible } from '@tanstack/ai-acp' + +const pi = acpCompatible({ + name: 'pi', + command: ({ model }) => `pi --acp -m ${model}`, + permissions: 'interactive', // emit approval-requested events for `ask` prompts + permissionMode: 'acceptEdits', // still auto-approve file edits +}) +``` + +`chat()`-provided tools bridged into the agent are always auto-approved, regardless of mode. + +## Session Resume + +On every run the adapter emits the harness session id as a CUSTOM event named `.session-id` (e.g. `pi.session-id`). Thread that id back through `modelOptions.sessionId` on the next call and the harness resumes the session — only the trailing user message is sent, since the agent already holds the prior context: + +```ts +import { chat, chatParamsFromRequest, toServerSentEventsResponse } from '@tanstack/ai' +import { withSandbox } from '@tanstack/ai-sandbox' +import { pi } from './pi-harness' // the configured `acpCompatible(...)` factory +import { sandbox } from './sandbox' + +export async function POST(request: Request) { + const params = await chatParamsFromRequest(request) + const sessionId = + typeof params.forwardedProps.sessionId === 'string' + ? params.forwardedProps.sessionId + : undefined + + const stream = chat({ + adapter: pi('pi-fast'), + messages: params.messages, + middleware: [withSandbox(sandbox)], + modelOptions: { sessionId }, + }) + + return toServerSentEventsResponse(stream) +} +``` + +## Workspace skills + +When you provision a [workspace](/ai/sandbox/workspace) via `withSandbox`, +`acpCompatible` projects its skills into the harness — each kind lands where that +harness expects it: + +| Workspace input | How `acpCompatible` projects it | +| --- | --- | +| `mcpSkill(name, config)` | Passed to the agent over **ACP natively** via `newSession`'s `mcpServers` (secrets/bearer headers resolved). No config file — that's the ACP advantage over file-based harnesses. | +| `gitSkill({ repo })` | Cloned during bootstrap, then linked into your declared [`skillsDir`](#configuration) (e.g. `.pi/skills`). Omit `skillsDir` and it's left unlinked (with a warning). | +| `fileSkill({ path, content })` | Written into the workspace root during bootstrap (provider-agnostic). | +| `instructions` | Written to `AGENTS.md` (and symlinks) during bootstrap. | +| `agentSkill(name)`, `plugins` | No generic ACP primitive — warned and skipped. Provide a `gitSkill` or MCP server instead. | + +`secrets` declared on the workspace are injected into the agent's environment at +create/resume (never persisted to snapshots), so the harness CLI picks them up +like any env var. + +## Protocol coverage + +`acpCompatible` implements the **client / orchestration** side of ACP — enough to +drive an agent through a full prompt turn, not the entire protocol surface. It is +a compliant *minimal* client: everything it doesn't implement is either +capability-gated (so advertising non-support is the spec-defined behavior) or a +rendering choice, never a violation. + +**Covered:** + +- `initialize` handshake — sends `clientInfo` + the protocol version, negotiates the version, advertises capabilities. +- `authenticate` (when the agent advertises auth methods), `session/new`, `session/load` (resume), `session/prompt`, `session/cancel`. +- `session/request_permission` with all four option kinds, mapped by [permission mode](#permissions). +- All streamed `session/update`s that carry turn output: `agent_message_chunk`, `agent_thought_chunk` (→ reasoning), `tool_call` / `tool_call_update`, `plan`. +- All five stop reasons (`end_turn`, `max_tokens`, `max_turn_requests`, `refusal`, `cancelled`). + +**Surfaced as `CUSTOM` stream events** (the AG-UI chat-event protocol has no +first-class event for non-text assistant output, so these ride on `CUSTOM`): + +- `.session-id` — the harness session id, for [resume](#session-resume). +- `.message-content` — non-text agent content (`image` / `audio` / `resource` / `resource_link` blocks). Its `value` is `{ content: }`. Non-text **tool** content (diffs, terminal, images) is preserved inside the `TOOL_CALL_RESULT` payload. +- the plan event, when you set `planEventName`. + +**Not implemented (by design):** + +- `fs/read_text_file`, `fs/write_text_file`, `terminal/*` — advertised as unsupported. The agent runs inside the sandbox with direct filesystem and shell access, so it never delegates these back to the client. +- Sending **multimodal prompts** — prompts are sent as text. (Agent multimodal *output* is surfaced via `message-content` above.) +- Incremental `usage_update` (final turn usage is reported instead), `available_commands_update`, `current_mode_update`, and experimental features (elicitation, NES, providers, session modes/config). + +## Next Steps + +- [Sandbox Overview](/ai/sandbox/overview) — how harnesses run inside a sandbox +- [Grok Build Adapter](/ai/adapters/grok-build) — a first-class ACP harness adapter +- [Sandbox Tools](/ai/sandbox/tools) — bridge your app's tools into the agent +- [OpenAI-Compatible Adapter](/ai/adapters/openai-compatible) — the same idea for model providers diff --git a/mintlify/ai/adapters/anthropic.md b/mintlify/ai/adapters/anthropic.md new file mode 100644 index 000000000..1d003ee37 --- /dev/null +++ b/mintlify/ai/adapters/anthropic.md @@ -0,0 +1,511 @@ +--- +title: Anthropic +id: anthropic-adapter +order: 2 +description: "Use Anthropic Claude models with TanStack AI — Claude Fable 5, Claude Sonnet 5, Claude Opus, and more via the @tanstack/ai-anthropic adapter." +keywords: + - tanstack ai + - anthropic + - claude + - claude fable 5 + - claude sonnet 5 + - claude opus + - adapter + - llm +--- + +The Anthropic adapter provides access to Claude models, including Claude Fable 5, Claude Sonnet 5, Claude Opus 4.8, and more. + +## Installation + +```bash +npm install @tanstack/ai-anthropic +``` + +## Basic Usage + +```typescript +import { chat } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; + +const stream = chat({ + adapter: anthropicText("claude-sonnet-4-6"), + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Basic Usage - Custom API Key + +```typescript +import { chat } from "@tanstack/ai"; +import { createAnthropicChat } from "@tanstack/ai-anthropic"; + +const adapter = createAnthropicChat("claude-sonnet-4-6", process.env.ANTHROPIC_API_KEY!, { + // ... your config options +}); + +const stream = chat({ + adapter, + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Configuration + +```typescript +import { createAnthropicChat, type AnthropicTextConfig } from "@tanstack/ai-anthropic"; + +const config: Omit = { + baseURL: "https://api.anthropic.com", // Optional, for custom endpoints +}; + +const adapter = createAnthropicChat("claude-sonnet-4-6", process.env.ANTHROPIC_API_KEY!, config); +``` + + +## Example: Chat Completion + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: anthropicText("claude-sonnet-4-6"), + messages, + }); + + return toServerSentEventsResponse(stream); +} +``` + +## Example: With Tools + +```typescript +import { chat, toServerSentEventsResponse, toolDefinition } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; +import { z } from "zod"; + +const searchDatabaseDef = toolDefinition({ + name: "search_database", + description: "Search the database", + inputSchema: z.object({ + query: z.string(), + }), +}); + +const searchDatabase = searchDatabaseDef.server(async ({ query }) => { + // Search database + return { results: [] }; +}); + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: anthropicText("claude-sonnet-4-6"), + messages, + tools: [searchDatabase], + }); + + return toServerSentEventsResponse(stream); +} +``` + +## Model Options + +Anthropic supports various provider-specific options. Sampling parameters live here too — `temperature`, `top_p`, and `max_tokens` — rather than as root-level props on `chat()`: + +```typescript +import { chat } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; + +const stream = chat({ + adapter: anthropicText("claude-sonnet-4-6"), + messages: [{ role: "user", content: "Hello!" }], + modelOptions: { + max_tokens: 4096, + temperature: 0.7, + top_p: 0.9, + top_k: 40, + stop_sequences: ["END"], + }, +}); +``` + +> If you previously passed `temperature` / `topP` / `maxTokens` at the root of `chat()`, see [Moving Sampling Options into modelOptions](/ai/migration/sampling-options-to-model-options). + +#### `max_tokens` default + +Anthropic's Messages API _requires_ `max_tokens` on every request, so the adapter always sends a value. When you don't set `modelOptions.max_tokens`, it defaults to the selected model's full output ceiling (`max_output_tokens` from the model metadata — e.g. 64K for Sonnet, 128K for Opus), falling back to a safe constant for unrecognized models. `max_tokens` is a ceiling, not a reservation — billing is on tokens actually generated — so this default costs nothing extra and avoids the silent mid-response truncation (`stop_reason: "max_tokens"`) that a low default would cause. Set `max_tokens` explicitly only when you want to _cap_ output below the model ceiling. If a response is truncated while using the default cap, the adapter logs a warning (visible with [debug logging](/ai/advanced/debug-logging) enabled). + +One exception: structured output (`chat({ outputSchema })`) on models that use the non-streaming finalization path clamps this default to ~21K tokens. The Anthropic SDK rejects a non-streaming request whose `max_tokens` could exceed its 10-minute timeout, so the full ceiling can't be used there. Streaming chat is unaffected. To raise the structured-output ceiling toward a model's true max, stream the response. + +### Thinking (Extended Thinking) + +Enable extended thinking with a token budget. This allows Claude to show its reasoning process, which is streamed as `thinking` chunks: + +```typescript ignore +modelOptions: { + thinking: { + type: "enabled", + budget_tokens: 2048, // Maximum tokens for thinking + }, +} +``` + +**Note:** `budget_tokens` must be less than `modelOptions.max_tokens` — set `max_tokens` high enough to leave room for the visible response alongside the thinking budget, or the request is rejected. + +### Adaptive Thinking (Claude 4.6+, Sonnet 5, Fable 5) + +Newer Claude models use adaptive thinking — the model decides when and how +much to think, and depth is tuned with `output_config.effort` instead of a +token budget: + +```typescript +import { chat } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; + +const stream = chat({ + adapter: anthropicText("claude-sonnet-5"), + messages: [{ role: "user", content: "Plan a database migration." }], + modelOptions: { + thinking: { type: "adaptive", display: "summarized" }, + output_config: { effort: "xhigh" }, + max_tokens: 64_000, + }, +}); +``` + +Per-model rules (enforced by the adapter's types): + +- **`claude-sonnet-5`, `claude-opus-4-8`, `claude-opus-4-7`** — adaptive + thinking with an explicit `{ type: "disabled" }` opt-out. The manual + `{ type: "enabled", budget_tokens }` shape is rejected with a 400, and + the sampling parameters (`temperature`, `top_p`, `top_k`) are not + accepted (on Sonnet 5 the API rejects non-default values; on Opus + 4.7/4.8 the parameters are removed entirely). +- **`claude-fable-5`** — thinking is always on. The only accepted explicit + config is `{ type: "adaptive" }` (both `disabled` and `budget_tokens` + return a 400), and sampling parameters are rejected. +- **`claude-opus-4-6` / `claude-sonnet-4-6`** — accept + `{ type: "adaptive" }` alongside the deprecated + `{ type: "enabled", budget_tokens }` shape, and still accept sampling + parameters. +- **`display`** defaults to `"omitted"` on Opus 4.7+ and the 5-generation + models — set `"summarized"` to stream the reasoning text. +- **`effort`** accepts `"low" | "medium" | "high" | "xhigh" | "max"`; + `"xhigh"` is available on Claude Opus 4.7+, Claude Sonnet 5, and + Claude Fable 5. + +### Prompt Caching + +Cache prompts for better performance and reduced costs: + +```typescript +import { chat } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; + +const stream = chat({ + adapter: anthropicText("claude-sonnet-4-6"), + messages: [ + { + role: "user", + content: [ + { + type: "text", + content: "What is the capital of France?", + metadata: { + cache_control: { + type: "ephemeral", + }, + }, + }, + ], + }, + ], +}); +``` + +## Summarization + +Anthropic supports text summarization: + +```typescript +import { summarize } from "@tanstack/ai"; +import { anthropicSummarize } from "@tanstack/ai-anthropic"; + +const result = await summarize({ + adapter: anthropicSummarize("claude-sonnet-4-6"), + text: "Your long text to summarize...", + maxLength: 100, + style: "concise", // "concise" | "bullet-points" | "paragraph" +}); + +console.log(result.summary); +``` + +## Environment Variables + +Set your API key in environment variables: + +```bash +ANTHROPIC_API_KEY=sk-ant-... +``` + +## API Reference + +Every factory pair follows the same shape: the short factory (`anthropicText`, `anthropicSummarize`) reads `ANTHROPIC_API_KEY` from the environment, while `createAnthropicChat` / `createAnthropicSummarize` take an explicit API key. Both take `model` as the first argument. + +### `anthropicText(model, config?)` / `createAnthropicChat(model, apiKey, config?)` + +Creates an Anthropic chat adapter. + +**Parameters:** + +- `model` - Claude model id (e.g. `"claude-sonnet-5"`, `"claude-fable-5"`, `"claude-opus-4-8"`) +- `config?.baseURL` - Custom base URL (optional) + +### `anthropicSummarize(model, config?)` / `createAnthropicSummarize(model, apiKey, config?)` + +Creates an Anthropic summarization adapter. + +## Limitations + +- **Image Generation**: Anthropic does not support image generation. Use OpenAI or Gemini for image generation. + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about tools +- [Other Adapters](/ai/adapters/openai) - Explore other providers + +## Provider Tools + +Anthropic exposes several native tools beyond user-defined function calls. +Import them from `@tanstack/ai-anthropic/tools` and pass them into +`chat({ tools: [...] })`. + +> For the full concept, a comparison matrix, and type-gating details, see +> [Provider Tools](/ai/tools/provider-tools). + +### `webSearchTool` + +Enables Claude to run Anthropic's native web search with inline citations. +Scope the search with `allowed_domains` or `blocked_domains` (mutually +exclusive); set `max_uses` to cap per-turn cost. + +```typescript +import { chat } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; +import { webSearchTool } from "@tanstack/ai-anthropic/tools"; + +const stream = chat({ + adapter: anthropicText("claude-opus-4-7"), + messages: [{ role: "user", content: "What's new in AI this week?" }], + tools: [ + webSearchTool({ + name: "web_search", + type: "web_search_20250305", + max_uses: 2, + }), + ], +}); +``` + +**Supported models:** every registered Claude model. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `webFetchTool` + +Lets Claude fetch the contents of a URL directly, useful when you want the +model to read a specific page rather than run a search. Takes no required +arguments — pass an optional config object to override defaults. + +```typescript +import { chat } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; +import { webFetchTool } from "@tanstack/ai-anthropic/tools"; + +const stream = chat({ + adapter: anthropicText("claude-sonnet-4-6"), + messages: [{ role: "user", content: "Summarise https://example.com" }], + tools: [webFetchTool()], +}); +``` + +**Supported models:** Claude Sonnet 4.x and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `codeExecutionTool` + +Gives Claude a sandboxed code-execution environment so it can run Python +snippets, analyse data, and return results inline. Choose the version string +that matches your desired API revision. + +```typescript +import { chat } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; +import { codeExecutionTool } from "@tanstack/ai-anthropic/tools"; + +const stream = chat({ + adapter: anthropicText("claude-sonnet-4-6"), + messages: [{ role: "user", content: "Plot a histogram of [1,2,2,3,3,3]" }], + tools: [ + codeExecutionTool({ name: "code_execution", type: "code_execution_20250825" }), + ], +}); +``` + +**Supported models:** Claude Sonnet 4.x and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +#### Attaching hosted skills + +Pass a `skills` array as the second argument to load provider-managed skill +bundles into the sandbox. The adapter auto-lifts them into the API's +`container.skills` param and adds the required beta headers for you. + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; +import { codeExecutionTool } from "@tanstack/ai-anthropic/tools"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: anthropicText("claude-sonnet-4-5"), + messages, + tools: [ + codeExecutionTool( + { type: "code_execution_20250825", name: "code_execution" }, + { + skills: [{ type: "anthropic", skill_id: "pptx", version: "latest" }], + }, + ), + ], + }); + + return toServerSentEventsResponse(stream); +} +``` + +For the full reference — skill shape, constraints, scope, and the OpenAI +equivalent — see [Provider Skills](/ai/tools/provider-skills). + +### `computerUseTool` + +Allows Claude to observe a virtual desktop (screenshots) and interact with it +via keyboard and mouse events. Provide the screen resolution so Claude can +calculate accurate coordinates. + +```typescript +import { chat } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; +import { computerUseTool } from "@tanstack/ai-anthropic/tools"; + +const stream = chat({ + adapter: anthropicText("claude-sonnet-4-6"), + messages: [{ role: "user", content: "Open the browser and go to example.com" }], + tools: [ + computerUseTool({ + type: "computer_20250124", + name: "computer", + display_width_px: 1024, + display_height_px: 768, + }), + ], +}); +``` + +**Supported models:** Claude Sonnet 3.5 and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `bashTool` + +Provides Claude with a persistent bash shell session, letting it run arbitrary +commands, install packages, or manipulate files on the host. Choose the type +string that matches your API revision. + +```typescript +import { chat } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; +import { bashTool } from "@tanstack/ai-anthropic/tools"; + +const stream = chat({ + adapter: anthropicText("claude-sonnet-4-6"), + messages: [{ role: "user", content: "List all TypeScript files in src/" }], + tools: [bashTool({ name: "bash", type: "bash_20250124" })], +}); +``` + +**Supported models:** Claude Sonnet 3.5 and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `textEditorTool` + +Gives Claude a structured text-editor interface for viewing and modifying files +using `str_replace`, `create`, `view`, and `undo_edit` commands. Choose the +type string for the API revision you target. + +```typescript +import { chat } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; +import { textEditorTool } from "@tanstack/ai-anthropic/tools"; + +const stream = chat({ + adapter: anthropicText("claude-sonnet-4-6"), + messages: [{ role: "user", content: "Fix the bug in src/index.ts" }], + tools: [ + textEditorTool({ type: "text_editor_20250124", name: "str_replace_editor" }), + ], +}); +``` + +**Supported models:** Claude Sonnet 3.5 and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `memoryTool` + +Enables Claude to store and retrieve information across conversation turns +using Anthropic's managed memory service. Call with no arguments to use +default configuration. + +```typescript +import { chat } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; +import { memoryTool } from "@tanstack/ai-anthropic/tools"; + +const stream = chat({ + adapter: anthropicText("claude-sonnet-4-6"), + messages: [{ role: "user", content: "Remember that I prefer metric units" }], + tools: [memoryTool()], +}); +``` + +**Supported models:** Claude Sonnet 4.x and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `customTool` + +Creates a tool with an inline JSON Schema input definition instead of going +through `toolDefinition()`. Useful when you need fine-grained control over the +schema shape or want to add `cache_control`. Unlike branded provider tools, +`customTool` returns a plain `Tool` and is accepted by any chat model. + +```typescript +import { chat } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; +import { customTool } from "@tanstack/ai-anthropic/tools"; +import { z } from "zod"; + +const stream = chat({ + adapter: anthropicText("claude-sonnet-4-6"), + messages: [{ role: "user", content: "Look up user 42" }], + tools: [ + customTool( + "lookup_user", + "Look up a user by ID and return their profile", + z.object({ userId: z.number() }), + ), + ], +}); +``` + +**Supported models:** all current Claude models. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). diff --git a/mintlify/ai/adapters/bedrock.md b/mintlify/ai/adapters/bedrock.md new file mode 100644 index 000000000..5e8efc39b --- /dev/null +++ b/mintlify/ai/adapters/bedrock.md @@ -0,0 +1,256 @@ +--- +title: Amazon Bedrock +id: bedrock-adapter +order: 7 +description: "Use Amazon Bedrock with TanStack AI — the Converse API is the default, reaching Claude, Nova, Llama, Mistral, DeepSeek, and more. Opt into OpenAI-compatible Chat Completions or Responses for open-weight and gpt-oss models. Supports streaming, tools, reasoning, and API-key or SigV4 auth." +keywords: + - tanstack ai + - amazon bedrock + - aws + - bedrock + - converse api + - openai compatible + - chat completions + - responses api + - sigv4 + - claude + - nova + - llama + - adapter +--- + +The Bedrock adapter connects TanStack AI to [Amazon Bedrock](https://aws.amazon.com/bedrock/) with three API paths: + +- **Converse** (default) — Bedrock's model-agnostic API built on `@aws-sdk/client-bedrock-runtime`. Reaches the broad chat catalog including Anthropic Claude, Amazon Nova, Meta Llama, Mistral, DeepSeek, Cohere, AI21, and OpenAI gpt-oss models. +- **Chat Completions** (`api: 'chat'`) — Bedrock's OpenAI-compatible Chat Completions endpoint. Reaches open-weight models only (gpt-oss, DeepSeek V3.x, Gemma, Qwen, Mistral open models, GLM, etc.). Does NOT reach Claude, Nova, or Llama. +- **Responses** (`api: 'responses'`) — Bedrock's OpenAI-compatible Responses API, mantle-only. Currently the OpenAI gpt-oss family. + +All paths support streaming and client-side tool calling. Reasoning output is +surfaced when the model emits it (e.g. DeepSeek R1, gpt-oss); on the Converse +path, request-side enablement of extended thinking (e.g. a Claude thinking +budget) is not yet wired, so only models that reason by default surface it. + +## Installation + +```bash +pnpm add @tanstack/ai-bedrock +``` + +No additional packages are required. SigV4 authentication is handled by `@aws-sdk/client-bedrock-runtime`, which is a direct dependency. + +## Quick Start (Converse — default) + +The default `bedrockText` call uses the Converse API and reaches the broad model catalog: + +```typescript ignore +// ignore: iterating a chat() stream and reading chunk.type/chunk.delta needs the +// AG-UI base event fields, which come from @ag-ui/core. It's a transitive dep of +// @tanstack/ai, so kiira (resolving @tanstack/ai from source under the dist->src +// heuristic) can't follow it and those base fields drop off StreamChunk. The code +// is correct (the same pattern is used throughout ai-client); see +// getting-started/quick-start-server for the type-checked consumption shape. +import { bedrockText } from '@tanstack/ai-bedrock' +import { chat } from '@tanstack/ai' + +const adapter = bedrockText('us.anthropic.claude-haiku-4-5-20251001-v1:0', { + region: 'us-east-1', +}) + +for await (const chunk of chat({ + adapter, + messages: [{ role: 'user', content: 'What is the capital of France?' }], +})) { + if (chunk.type === 'TEXT_MESSAGE_CONTENT') process.stdout.write(chunk.delta ?? '') +} +``` + +Equivalent to passing `{ api: 'converse' }` explicitly. Returns a `bedrock-converse` adapter. + +## Authentication + +Bedrock supports two authentication modes. + +### API Key + +Bedrock issues API keys from the AWS Console. See the [Bedrock API keys guide](https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html) for instructions. + +Set one of the following environment variables and the adapter picks it up automatically: + +```bash +BEDROCK_API_KEY=your-bedrock-api-key +# or the legacy name: +AWS_BEARER_TOKEN_BEDROCK=your-bedrock-api-key +``` + +### SigV4 (AWS credential chain) + +For workloads using IAM roles, instance profiles, or `~/.aws/credentials`, set `auth: 'sigv4'` (or leave it as `'auto'` with no API key in the environment). SigV4 works out of the box via `@aws-sdk/client-bedrock-runtime` — no additional packages required. + +```bash +AWS_ACCESS_KEY_ID=... +AWS_SECRET_ACCESS_KEY=... +AWS_SESSION_TOKEN=... # optional, for temporary credentials +``` + +### Auth resolution order (`auth: 'auto'`, the default) + +1. Explicit `apiKey` passed to the factory +2. `BEDROCK_API_KEY` environment variable +3. `AWS_BEARER_TOKEN_BEDROCK` environment variable +4. SigV4 via the standard AWS credential chain + +## Configuration + +`BedrockClientConfig` accepts the following options: + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `api` | `'converse' \| 'chat' \| 'responses'` | `'converse'` | Bedrock API to use | +| `region` | `string` | `'us-east-1'` | AWS region string (e.g. `'us-west-2'`) | +| `auth` | `'apikey' \| 'sigv4' \| 'auto'` | `'auto'` | Authentication mode | +| `apiKey` | `string` | — | Explicit API key (overrides env vars) | +| `baseURL` | `string` | — | Override the computed base URL entirely | +| `endpoint` | `'runtime' \| 'mantle'` | `'runtime'` | Bedrock endpoint to target (Chat Completions path only) | + +The `endpoint` option only applies when `api: 'chat'`. The `runtime` endpoint (`bedrock-runtime`) hosts the broad open-weight catalog; `mantle` is an alternative. The Responses API always targets mantle. + +## Converse API (default) + +`bedrockText(model)` or `bedrockText(model, { api: 'converse' })` returns a `bedrock-converse` adapter backed by `@aws-sdk/client-bedrock-runtime`. This is Bedrock's model-agnostic conversational API and is the recommended path for most use cases. + +**Model scope:** Anthropic Claude, Amazon Nova, Meta Llama, Mistral, DeepSeek, Cohere, AI21, OpenAI gpt-oss, and other models accessible in your account. See [Model availability](#model-availability) below. + +```typescript +import { bedrockText } from '@tanstack/ai-bedrock' +import { chat } from '@tanstack/ai' + +// Claude via Converse +const claudeAdapter = bedrockText('us.anthropic.claude-haiku-4-5-20251001-v1:0', { + region: 'us-east-1', +}) + +// Amazon Nova via Converse +const novaAdapter = bedrockText('us.amazon.nova-pro-v1:0', { + region: 'us-east-1', +}) + +// Meta Llama via Converse +const llamaAdapter = bedrockText('us.meta.llama4-maverick-17b-instruct-v1:0', { + region: 'us-east-1', +}) +``` + +### Explicit API key (Converse) + +```typescript +import { createBedrockText } from '@tanstack/ai-bedrock' + +const adapter = createBedrockText( + 'us.anthropic.claude-haiku-4-5-20251001-v1:0', + 'your-bedrock-api-key', + { region: 'us-west-2' }, +) +``` + +## Chat Completions API (`api: 'chat'`) + +Set `api: 'chat'` to use Bedrock's OpenAI-compatible Chat Completions endpoint. Returns a `bedrock` adapter. + +**Model scope:** Open-weight models only — gpt-oss, DeepSeek V3.x, Gemma, Qwen, Mistral open models, GLM, and similar. Claude, Nova, and Llama are NOT available on this endpoint. See the [AWS API compatibility matrix](https://docs.aws.amazon.com/bedrock/latest/userguide/models-api-compatibility.html) for the current list. + +```typescript ignore +// ignore: see the Converse quick-start above — iterating a chat() stream and +// reading chunk.type/chunk.delta needs @ag-ui/core base fields kiira can't +// resolve transitively through @tanstack/ai source. +import { bedrockText } from '@tanstack/ai-bedrock' +import { chat } from '@tanstack/ai' + +const adapter = bedrockText('openai.gpt-oss-20b-1:0', { + region: 'us-east-1', + api: 'chat', +}) + +for await (const chunk of chat({ + adapter, + messages: [{ role: 'user', content: 'What is the capital of France?' }], +})) { + if (chunk.type === 'TEXT_MESSAGE_CONTENT') process.stdout.write(chunk.delta ?? '') +} +``` + +## Responses API (`api: 'responses'`) + +Set `api: 'responses'` to use Bedrock's OpenAI-compatible Responses API. Returns a `bedrock-responses` adapter. This API is mantle-only. + +**Model scope:** Currently the OpenAI gpt-oss family. The Responses API is stateful — pass `previous_response_id` and `store` through `modelOptions` to continue a conversation server-side. + +```typescript ignore +// ignore: see the Converse quick-start above — iterating a chat() stream and +// reading chunk.type/chunk.delta needs @ag-ui/core base fields kiira can't +// resolve transitively through @tanstack/ai source. +import { bedrockText } from '@tanstack/ai-bedrock' +import { chat } from '@tanstack/ai' + +const adapter = bedrockText('openai.gpt-oss-120b-1:0', { + region: 'us-east-1', + api: 'responses', +}) + +for await (const chunk of chat({ + adapter, + messages: [{ role: 'user', content: 'Summarize the Bedrock pricing page.' }], +})) { + if (chunk.type === 'TEXT_MESSAGE_CONTENT') process.stdout.write(chunk.delta ?? '') +} +``` + +## Model Availability + +The adapter ships with a hand-seeded snapshot catalog (`src/model-catalog.generated.ts`) of confirmed model IDs. This catalog can be refreshed by the maintainer script `scripts/fetch-bedrock-models.ts`, which calls `ListFoundationModels` with AWS credentials. + +**Actual model availability depends on your AWS account's model access configuration and the region you are targeting.** Enable model access in the [Amazon Bedrock console](https://console.aws.amazon.com/bedrock/home#/modelaccess) before use. + +For the full list of models and which API endpoints they support, see the [AWS API compatibility matrix](https://docs.aws.amazon.com/bedrock/latest/userguide/models-api-compatibility.html). + +## Supported Capabilities + +- Streaming chat completions +- Client-side tool calling +- Reasoning output (extended thinking surfaced when the model emits it; request-side enablement on Converse is not yet wired) +- Multimodal input (text, images, documents — model-dependent) +- JSON schema / structured output + +## API Reference + +### `bedrockText(model, config?)` + +Creates a Bedrock adapter using environment-variable auth. + +- `model` — Model ID (e.g. `'us.anthropic.claude-haiku-4-5-20251001-v1:0'`) +- `config.api` — `'converse'` (default), `'chat'`, or `'responses'` +- `config.region` — AWS region string (default `'us-east-1'`) +- `config.auth` — `'auto'` (default), `'apikey'`, or `'sigv4'` +- `config.apiKey` — Explicit API key (overrides env vars) +- `config.baseURL` — Override base URL +- `config.endpoint` — `'runtime'` (default) or `'mantle'` (Chat Completions path only) + +Returns a chat adapter for use with `chat()` or `generate()`. + +| `api` value | Adapter name | Underlying SDK | +|---|---|---| +| `'converse'` (default) | `bedrock-converse` | `@aws-sdk/client-bedrock-runtime` | +| `'chat'` | `bedrock` | `openai` (OpenAI-compatible) | +| `'responses'` | `bedrock-responses` | `openai` (OpenAI-compatible) | + +### `createBedrockText(model, apiKey, config?)` + +Creates a Bedrock adapter with an explicit API key, bypassing the environment-variable lookup. + +## Next Steps + +- [Amazon Bedrock API keys](https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html) — Create and manage API keys +- [Amazon Bedrock model access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) — Enable models in your account +- [AWS API compatibility matrix](https://docs.aws.amazon.com/bedrock/latest/userguide/models-api-compatibility.html) — Which models work with which APIs +- [Converse API reference](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) — Native Converse API docs +- [Streaming Guide](/ai/chat/streaming) — Learn about streaming responses +- [Tools Guide](/ai/tools/tools) — Learn about tool calling diff --git a/mintlify/ai/adapters/claude-code.md b/mintlify/ai/adapters/claude-code.md new file mode 100644 index 000000000..302b54ba7 --- /dev/null +++ b/mintlify/ai/adapters/claude-code.md @@ -0,0 +1,181 @@ +--- +title: Claude Code +id: claude-code-adapter +order: 11 +description: "Use Claude Code as a chat backend in TanStack AI — agent harness with local tool execution, stateful coding sessions, and tool bridging via @tanstack/ai-claude-code." +keywords: + - tanstack ai + - claude code + - claude agent sdk + - anthropic + - harness + - agent + - coding agent + - adapter +--- + +The Claude Code adapter runs [Claude Code](https://docs.anthropic.com/en/docs/claude-code) (via the `@anthropic-ai/claude-agent-sdk`) as a chat backend. Unlike HTTP provider adapters, this is a **harness adapter**: Claude Code runs its own agent loop and executes its own tools — bash, file reads and edits, glob/grep search, web search — locally on your server. Each `chat()` call runs one full harness turn; the harness's tool activity streams back as already-resolved tool-call events your UI can render. + +> **Server-only.** The harness spawns the Claude Code runtime as a subprocess, so this adapter only works in a Node.js server environment — never in the browser. Treat it like giving Claude a shell on the machine it runs on, and configure permissions accordingly. + +## Installation + +```bash +npm install @tanstack/ai-claude-code +``` + +A runnable demo lives at [`examples/sandbox-web`](https://github.com/TanStack/ai/tree/main/examples/sandbox-web) — switch the harness (Claude Code, Codex, OpenCode, Grok Build) and sandbox provider per run, with session resume, the harness tool timeline, permission modes, and tool bridging, wired into a TanStack Start app. + +## Authentication + +The harness resolves credentials the same way Claude Code does: + +- `ANTHROPIC_API_KEY` in the server's environment (or the `apiKey` config option), or +- an existing Claude subscription login on the machine (`claude login`). + +## Basic Usage + +```typescript +import { chat } from "@tanstack/ai"; +import { claudeCodeText } from "@tanstack/ai-claude-code"; + +const stream = chat({ + adapter: claudeCodeText("claude-opus-4-8", { + cwd: "/path/to/project", + permissionMode: "acceptEdits", + }), + messages: [{ role: "user", content: "Fix the failing test in utils.test.ts" }], +}); +``` + +## Configuration + +| Option | Description | +| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | +| `cwd` | Working directory for the harness session. Defaults to `process.cwd()`. | +| `permissionMode` | Claude Code permission mode (`'default'`, `'acceptEdits'`, `'bypassPermissions'`, `'plan'`, `'dontAsk'`, `'auto'`). See the permissions note below. | +| `allowedTools` | Built-in tools the harness may use without prompting (e.g. `['Read', 'Grep', 'Bash(npm test:*)']`). | +| `disallowedTools` | Built-in tools removed from the harness entirely. | +| `maxTurns` | Maximum harness-internal turns per run. | +| `systemPromptMode` | `'append'` (default) keeps Claude Code's preset system prompt and appends your `systemPrompts`; `'replace'` sends yours as the entire prompt. | +| `mcpServers` | Extra MCP servers passed through to the harness untouched. | +| `apiKey` | Anthropic API key for the harness subprocess. | +| `env` | Extra environment variables for the harness subprocess. | +| `pathToClaudeCodeExecutable` | Use a specific Claude Code executable instead of the SDK's bundled one. | +| `streamPartials` | Emit true token-level text deltas (default `true`). | +| `canUseTool` | Custom permission handler; replaces the adapter's default handler. | +| `settingSources` | Claude Code settings tiers to load. Default `['project']`: the `cwd`'s CLAUDE.md and project settings apply, but user-level config on the host (`~/.claude` plugins, hooks, skills) is ignored. Pass `['user', 'project', 'local']` for CLI-equivalent behavior, or `[]` for full isolation. | + +**Permissions on headless servers.** Without an explicit `permissionMode` or `canUseTool`, the adapter installs a safe default handler: bridged TanStack tools always run, and any built-in tool call that would normally prompt a human is denied with guidance instead of hanging the request. To let the harness edit files or run commands, set `permissionMode: 'acceptEdits'` / `'bypassPermissions'`, or enumerate `allowedTools`. + +## Stateful Sessions + +Claude Code sessions are stateful — the harness keeps the full working context (files read, commands run, conclusions reached) between turns. The adapter surfaces the session id of every run as a custom stream event named `claude-code.session-id`; thread it back via `modelOptions.sessionId` to resume the session. When resuming, only the latest user message is sent — the harness already holds the prior context. + +Server endpoint: + +```typescript +import { + chat, + chatParamsFromRequest, + toServerSentEventsResponse, +} from "@tanstack/ai"; +import { claudeCodeText } from "@tanstack/ai-claude-code"; + +export async function POST(request: Request) { + const params = await chatParamsFromRequest(request); + + // Extra fields the client puts in the connection `body` arrive here. + const sessionId = + typeof params.forwardedProps.sessionId === "string" + ? params.forwardedProps.sessionId + : undefined; + + const stream = chat({ + adapter: claudeCodeText("claude-opus-4-8", { + cwd: "/path/to/project", + permissionMode: "acceptEdits", + }), + messages: params.messages, + modelOptions: { sessionId }, + }); + + return toServerSentEventsResponse(stream); +} +``` + +Client (React) — capture the session id from the custom event and send it back on subsequent requests: + +```typescript +import { useState } from "react"; +import { useChat } from "@tanstack/ai-react"; +import { fetchServerSentEvents } from "@tanstack/ai-client"; + +function CodingAssistant() { + const [sessionId, setSessionId] = useState(undefined); + + const { messages, sendMessage } = useChat({ + connection: fetchServerSentEvents("/api/chat", () => ({ + body: { sessionId }, + })), + onCustomEvent: (name, value) => { + if ( + name === "claude-code.session-id" && + typeof value === "object" && + value !== null && + "sessionId" in value && + typeof value.sessionId === "string" + ) { + setSessionId(value.sessionId); + } + }, + }); + + // ... render messages; harness tool activity (Bash, Edit, Read, ...) + // arrives as regular tool-call parts with their results attached. +} +``` + +Sessions are stored on the machine that ran them (`~/.claude/projects/`), so resuming only works on the same server instance. Pass `modelOptions: { forkSession: true }` alongside `sessionId` to branch a session instead of continuing it. + +## Tools + +Two kinds of tools flow through this adapter: + +1. **Built-in harness tools** (`Bash`, `Read`, `Write`, `Edit`, `Glob`, `Grep`, `WebSearch`, ...) are executed by Claude Code itself. Their activity streams back as tool-call events with results already attached, so `useChat` UIs render them with no extra wiring — but your code never executes them. + +2. **Your TanStack tools** are bridged *into* the harness as an in-process MCP server. Define them as usual with `toolDefinition().server()`; the model sees them as `mcp__tanstack__` and the adapter strips the prefix on the way back out, so events match the names you registered. + +```typescript +import { z } from "zod"; +import { chat, toolDefinition } from "@tanstack/ai"; +import { claudeCodeText } from "@tanstack/ai-claude-code"; + +const lookupTicket = toolDefinition({ + name: "lookup_ticket", + description: "Look up an issue ticket by id", + inputSchema: z.object({ ticketId: z.string() }), +}).server(async ({ ticketId }) => { + return { ticketId, status: "open", title: "Crash on startup" }; +}); + +const stream = chat({ + adapter: claudeCodeText("claude-opus-4-8"), + messages: [{ role: "user", content: "What's the status of ticket T-123?" }], + tools: [lookupTicket], +}); +``` + +**Client-side and approval-gated tools are not supported.** The harness executes tools inside a live subprocess, which cannot pause across HTTP requests to wait for a browser round-trip or a human approval. Passing a tool without a server `execute()` implementation — or one marked `needsApproval` — fails fast with a descriptive error. Run those tools outside the harness with a regular provider adapter. + +## Structured Output + +`structuredOutput()` uses the harness's native JSON-schema output format in a one-shot run (single turn, no tools). It works for finalization after a chat, but a plain provider adapter (e.g. `@tanstack/ai-anthropic`) is the better choice when structured extraction is the primary job — it's faster and doesn't spawn a subprocess. + +## Limitations + +- **Server-only (Node).** The harness spawns a subprocess; Windows support is untested. +- **The harness owns the agent loop.** TanStack's agent-loop strategies and per-iteration middleware don't apply inside a harness turn; `maxTurns` is the equivalent control. +- **No sampling controls.** `temperature`-style options don't exist here. +- **Sessions are machine-local.** Resume requires hitting the same server instance. +- **Cold starts.** Each call spawns a harness turn; expect higher first-token latency than HTTP adapters. diff --git a/mintlify/ai/adapters/codex.md b/mintlify/ai/adapters/codex.md new file mode 100644 index 000000000..c84ff31e2 --- /dev/null +++ b/mintlify/ai/adapters/codex.md @@ -0,0 +1,182 @@ +--- +title: Codex +id: codex-adapter +order: 12 +description: "Use OpenAI Codex as a chat backend in TanStack AI — agent harness with local tool execution, stateful coding sessions, and tool bridging via @tanstack/ai-codex." +keywords: + - tanstack ai + - codex + - codex sdk + - openai + - harness + - agent + - coding agent + - adapter +--- + +The Codex adapter runs [OpenAI Codex](https://developers.openai.com/codex) (via the `@openai/codex-sdk`) as a chat backend. Unlike HTTP provider adapters, this is a **harness adapter**: Codex runs its own agent loop and executes its own tools — shell commands, file changes, web search — locally on your server, inside its sandbox. Each `chat()` call runs one full harness turn; the harness's tool activity streams back as already-resolved tool-call events your UI can render. + +> **Server-only.** The harness spawns the Codex runtime (bundled with the SDK) as a subprocess, so this adapter only works in a Node.js server environment — never in the browser. The sandbox mode is the safety boundary; configure it deliberately. + +## Installation + +```bash +npm install @tanstack/ai-codex +``` + +A runnable demo lives at [`examples/sandbox-web`](https://github.com/TanStack/ai/tree/main/examples/sandbox-web) — switch the harness (Claude Code, Codex, OpenCode, Grok Build) and sandbox provider per run, with session resume, the harness tool timeline, sandbox modes, and tool bridging, wired into a TanStack Start app. + +## Authentication + +The harness resolves credentials the same way the Codex CLI does: + +- the `apiKey` config option (exported to the subprocess as `CODEX_API_KEY`; usage-based billing), or +- an existing ChatGPT login on the machine (`codex login`). + +## Basic Usage + +```typescript +import { chat } from "@tanstack/ai"; +import { codexText } from "@tanstack/ai-codex"; + +const stream = chat({ + adapter: codexText("gpt-5.1-codex", { + cwd: "/path/to/project", + sandboxMode: "workspace-write", + }), + messages: [{ role: "user", content: "Fix the failing test in utils.test.ts" }], +}); +``` + +## Configuration + +| Option | Description | +| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | +| `cwd` | Working directory for the harness session. Defaults to `process.cwd()`. | +| `sandboxMode` | Codex sandbox: `'read-only'` (harness default), `'workspace-write'`, or `'danger-full-access'`. This is the safety boundary on a server. | +| `approvalPolicy` | Codex approval policy. Defaults to `'never'` — headless runs have no approval UI, so anything else can stall a turn. | +| `modelReasoningEffort` | `'minimal'` \| `'low'` \| `'medium'` \| `'high'` \| `'xhigh'`. | +| `skipGitRepoCheck` | Skip the harness's git-repo safety check. Defaults to `true` (server adapters routinely point at scratch directories). | +| `networkAccessEnabled` | Allow network access inside the `workspace-write` sandbox. | +| `webSearchMode` | `'disabled'` \| `'cached'` \| `'live'`. | +| `additionalDirectories`| Extra writable directories beyond `cwd`. | +| `apiKey` | OpenAI API key for the harness subprocess. | +| `baseUrl` | Override the Codex backend base URL. | +| `codexPathOverride` | Use a specific codex executable instead of the SDK's bundled binary. | +| `env` | Environment variables for the subprocess. When set, `process.env` is **not** inherited (Codex SDK semantics). | +| `config` | Extra `--config key=value` overrides passed to the Codex CLI (e.g. additional `mcp_servers` entries). | + +Per-call overrides — `sessionId`, `sandboxMode`, `approvalPolicy`, `modelReasoningEffort`, `workingDirectory`, `skipGitRepoCheck` — go through `modelOptions`. + +## Stateful Sessions + +Codex threads are stateful — the harness keeps the full working context (files read, commands run, conclusions reached) between turns. The adapter surfaces the thread id of every fresh run as a custom stream event named `codex.session-id`; thread it back via `modelOptions.sessionId` to resume. When resuming, only the latest user message is sent — the harness already holds the prior context. + +Server endpoint: + +```typescript +import { + chat, + chatParamsFromRequest, + toServerSentEventsResponse, +} from "@tanstack/ai"; +import { codexText } from "@tanstack/ai-codex"; + +export async function POST(request: Request) { + const params = await chatParamsFromRequest(request); + + // Extra fields the client puts in the connection `body` arrive here. + const sessionId = + typeof params.forwardedProps.sessionId === "string" + ? params.forwardedProps.sessionId + : undefined; + + const stream = chat({ + adapter: codexText("gpt-5.1-codex", { + cwd: "/path/to/project", + sandboxMode: "workspace-write", + }), + messages: params.messages, + modelOptions: { sessionId }, + }); + + return toServerSentEventsResponse(stream); +} +``` + +Client (React) — capture the session id from the custom event and send it back on subsequent requests: + +```typescript +import { useState } from "react"; +import { useChat } from "@tanstack/ai-react"; +import { fetchServerSentEvents } from "@tanstack/ai-client"; + +function CodingAssistant() { + const [sessionId, setSessionId] = useState(undefined); + + const { messages, sendMessage } = useChat({ + connection: fetchServerSentEvents("/api/chat", () => ({ + body: { sessionId }, + })), + onCustomEvent: (name, value) => { + if ( + name === "codex.session-id" && + typeof value === "object" && + value !== null && + "sessionId" in value && + typeof value.sessionId === "string" + ) { + setSessionId(value.sessionId); + } + }, + }); + + // ... render messages; harness tool activity (command_execution, + // file_change, ...) arrives as regular tool-call parts with results. +} +``` + +Sessions are stored on the machine that ran them (`~/.codex/sessions/`), so resuming only works on the same server instance. + +## Tools + +Two kinds of tools flow through this adapter: + +1. **Built-in harness tools** are executed by Codex itself and stream back as tool-call events with results already attached: `command_execution` (shell), `file_change` (patches), `web_search`, and `todo_list` (the agent's running plan). Your code never executes them. + +2. **Your TanStack tools** are bridged *into* the harness: the adapter starts a short-lived Streamable-HTTP MCP server on `127.0.0.1` for the duration of the turn and points Codex at it. Define tools as usual with `toolDefinition().server()`; tool-call events come back under the names you registered. + +```typescript +import { z } from "zod"; +import { chat, toolDefinition } from "@tanstack/ai"; +import { codexText } from "@tanstack/ai-codex"; + +const lookupTicket = toolDefinition({ + name: "lookup_ticket", + description: "Look up an issue ticket by id", + inputSchema: z.object({ ticketId: z.string() }), +}).server(async ({ ticketId }) => { + return { ticketId, status: "open", title: "Crash on startup" }; +}); + +const stream = chat({ + adapter: codexText("gpt-5.1-codex"), + messages: [{ role: "user", content: "What's the status of ticket T-123?" }], + tools: [lookupTicket], +}); +``` + +**Client-side and approval-gated tools are not supported.** The harness executes tools inside a live subprocess, which cannot pause across HTTP requests to wait for a browser round-trip or a human approval. Passing a tool without a server `execute()` implementation — or one marked `needsApproval` — fails fast with a descriptive error. Run those tools outside the harness with a regular provider adapter. + +## Structured Output + +`structuredOutput()` uses Codex's native `outputSchema` support in a fresh, read-only, one-shot thread whose final message is a JSON string conforming to your schema. It works for finalization after a chat, but a plain provider adapter (e.g. `@tanstack/ai-openai`) is the better choice when structured extraction is the primary job — it's faster and doesn't spawn a subprocess. + +## Limitations + +- **No token-level text streaming.** The Codex SDK reports assistant text and reasoning only as completed items, so text arrives message-at-a-time. Tool activity (commands starting/finishing) still streams live, which keeps the UI feeling alive during long turns. +- **Server-only (Node).** The harness spawns a subprocess. +- **The harness owns the agent loop.** TanStack's agent-loop strategies and per-iteration middleware don't apply inside a harness turn. +- **No sampling controls.** `temperature`-style options don't exist here. +- **Sessions are machine-local.** Resume requires hitting the same server instance. +- **Cold starts.** Each call spawns a harness turn; expect higher first-token latency than HTTP adapters. diff --git a/mintlify/ai/adapters/elevenlabs.md b/mintlify/ai/adapters/elevenlabs.md new file mode 100644 index 000000000..739d12038 --- /dev/null +++ b/mintlify/ai/adapters/elevenlabs.md @@ -0,0 +1,375 @@ +--- +title: ElevenLabs +id: elevenlabs-adapter +order: 9 +description: "Build realtime voice-to-voice conversational AI with ElevenLabs agents in TanStack AI via the @tanstack/ai-elevenlabs adapter." +keywords: + - tanstack ai + - elevenlabs + - realtime voice ai + - conversational ai + - voice chat + - voice agents + - adapter +--- + +The ElevenLabs adapter is **voice-focused**. It exposes four capabilities: + +- **Realtime voice agents** (`elevenlabsRealtime` / `elevenlabsRealtimeToken`) — full-duplex voice-to-voice conversations powered by ElevenLabs Conversational AI agents. +- **Text-to-speech** (`elevenlabsSpeech`) — one-shot speech generation via `generateSpeech()`. +- **Music & sound effects** (`elevenlabsAudio`) — one-shot audio generation via `generateAudio()`. +- **Transcription** (`elevenlabsTranscription`) — speech-to-text via `generateTranscription()`. + +It does not support text `chat()` or `summarize()` — use OpenAI, Anthropic, or Gemini for those. + +The realtime adapter uses an **agent-based architecture** where you configure your conversational AI agent in the [ElevenLabs dashboard](https://elevenlabs.io/) (voice, personality, knowledge base, tools) and then connect to it at runtime. The adapter wraps the `@elevenlabs/client` SDK for seamless integration with `useRealtimeChat` and `RealtimeClient`. + +## Installation + +```bash +npm install @tanstack/ai-elevenlabs +``` + +Peer dependencies: + +```bash +npm install @tanstack/ai @tanstack/ai-client +``` + +## Server Setup + +The server generates a **signed WebSocket URL** so your API key never reaches the client. The signed URL is valid for 30 minutes. + +```typescript group=elevenlabs-1 +import { realtimeToken } from '@tanstack/ai' +import { elevenlabsRealtimeToken } from '@tanstack/ai-elevenlabs' + +// In your API route (Express, Hono, TanStack Start, etc.) +export async function POST() { + const token = await realtimeToken({ + adapter: elevenlabsRealtimeToken({ + agentId: process.env.ELEVENLABS_AGENT_ID!, + }), + }) + + return Response.json(token) +} +``` + +### With Overrides + +You can override agent settings at token generation time without changing your dashboard configuration: + +```typescript group=elevenlabs-1 +const token = await realtimeToken({ + adapter: elevenlabsRealtimeToken({ + agentId: process.env.ELEVENLABS_AGENT_ID!, + overrides: { + voiceId: 'custom-voice-id', + systemPrompt: 'You are a helpful voice assistant.', + firstMessage: 'Hello! How can I help you today?', + language: 'en', + }, + }), +}) +``` + +## Client Setup + +### React (useRealtimeChat) + +```tsx +import { useRealtimeChat } from '@tanstack/ai-react' +import { elevenlabsRealtime } from '@tanstack/ai-elevenlabs' + +function VoiceChat() { + const { + status, + mode, + messages, + connect, + disconnect, + pendingUserTranscript, + pendingAssistantTranscript, + inputLevel, + outputLevel, + } = useRealtimeChat({ + getToken: () => + fetch('/api/realtime-token', { method: 'POST' }).then((r) => r.json()), + adapter: elevenlabsRealtime(), + }) + + return ( +
+

Status: {status}

+

Mode: {mode}

+ + {pendingUserTranscript &&

You: {pendingUserTranscript}...

} + {pendingAssistantTranscript && ( +

AI: {pendingAssistantTranscript}...

+ )} + {messages.map((msg) => ( +
+ {msg.role}: + {msg.parts.map((part, i) => ( + + {part.type === 'text' ? part.content : null} + {part.type === 'audio' ? part.transcript : null} + + ))} +
+ ))} +
+ ) +} +``` + +### Non-React (RealtimeClient) + +```typescript +import { RealtimeClient } from '@tanstack/ai-client' +import { elevenlabsRealtime } from '@tanstack/ai-elevenlabs' + +const client = new RealtimeClient({ + getToken: () => + fetch('/api/realtime-token', { method: 'POST' }).then((r) => r.json()), + adapter: elevenlabsRealtime(), + onMessage: (message) => { + console.log(`${message.role}:`, message.parts) + }, + onStatusChange: (status) => { + console.log('Status:', status) + }, + onModeChange: (mode) => { + console.log('Mode:', mode) + }, +}) + +await client.connect() +``` + +## Client Tools + +ElevenLabs supports client-side tools that execute in the browser. Define tools using the standard `toolDefinition()` API: + +```typescript +import { toolDefinition } from '@tanstack/ai' +import { useRealtimeChat } from '@tanstack/ai-react' +import { elevenlabsRealtime } from '@tanstack/ai-elevenlabs' +import { z } from 'zod' + +const getWeatherDef = toolDefinition({ + name: 'getWeather', + description: 'Get weather for a location', + inputSchema: z.object({ + location: z.string(), + }), + outputSchema: z.object({ + temperature: z.number(), + conditions: z.string(), + }), +}) + +const getWeather = getWeatherDef.client(async ({ location }) => { + const res = await fetch(`/api/weather?location=${location}`) + return res.json() +}) + +// Pass tools to the hook +const chat = useRealtimeChat({ + getToken: () => + fetch('/api/realtime-token', { method: 'POST' }).then((r) => r.json()), + adapter: elevenlabsRealtime(), + tools: [getWeather], +}) +``` + +Tool results are automatically serialized to strings and returned to the ElevenLabs agent. The adapter converts TanStack tool definitions into the `@elevenlabs/client` clientTools format internally. + +## Configuration + +### `elevenlabsRealtimeToken` Options + +Used on the **server** to generate a signed WebSocket URL. + +| Option | Type | Required | Description | +|--------|------|----------|-------------| +| `agentId` | `string` | No\* | Agent ID configured in the ElevenLabs dashboard. \*Falls back to `ELEVENLABS_AGENT_ID`; required only if that env var is unset | +| `overrides.voiceId` | `string` | No | Custom voice ID to override the agent's default voice | +| `overrides.systemPrompt` | `string` | No | Custom system prompt to override the agent's default | +| `overrides.firstMessage` | `string` | No | First message the agent speaks when the session starts | +| `overrides.language` | `string` | No | Language code (e.g., `'en'`, `'es'`, `'fr'`) | + +### `elevenlabsRealtime` Options + +Used on the **client** to establish the connection. + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `connectionMode` | `'websocket' \| 'webrtc'` | auto-detect | Transport protocol for the connection | +| `debug` | `boolean \| DebugConfig` | `false` | Enable debug logging — pass `true` for all categories, or a `DebugConfig` to select categories/sink | + +## Differences from OpenAI Realtime + +ElevenLabs and OpenAI take different approaches to realtime voice: + +| | ElevenLabs | OpenAI | +|---|---|---| +| **Configuration** | Agent-based. Configure voice, personality, and knowledge in the ElevenLabs dashboard or via `overrides` at token time. | Session-based. Configure `instructions`, `voice`, `temperature`, etc. per session via `useRealtimeChat` options. | +| **Token type** | Signed WebSocket URL (valid 30 minutes) | Ephemeral API token (valid ~10 minutes) | +| **Transport** | WebSocket (default) or WebRTC | WebRTC | +| **Audio handling** | `@elevenlabs/client` SDK manages audio capture and playback automatically | TanStack AI manages WebRTC peer connection and audio tracks | +| **VAD** | Handled by ElevenLabs server-side | Supports `server`, `semantic`, and `manual` modes | +| **Runtime updates** | Session config is set at creation time and cannot be changed mid-session | Supports `updateSession()` for mid-session config changes | +| **Image input** | Not supported | Supported via `sendImage()` | +| **Time domain data** | Not available from the SDK | Available for waveform visualizations | + +## Audio Visualization + +The ElevenLabs adapter provides audio visualization data through the same interface as other realtime adapters: + +```typescript +import { useRealtimeChat } from '@tanstack/ai-react' +import { elevenlabsRealtime } from '@tanstack/ai-elevenlabs' + +const { + inputLevel, // 0-1 normalized microphone volume + outputLevel, // 0-1 normalized speaker volume + getInputFrequencyData, // Uint8Array frequency spectrum + getOutputFrequencyData, +} = useRealtimeChat({ + getToken: () => + fetch('/api/realtime-token', { method: 'POST' }).then((r) => r.json()), + adapter: elevenlabsRealtime(), +}) +``` + +**Note:** ElevenLabs provides volume levels and frequency data but does not expose time-domain data. The `getInputTimeDomainData()` and `getOutputTimeDomainData()` methods return static placeholder arrays. The default audio sample rate is 16kHz. + +## Environment Variables + +Set these in your server environment: + +```bash +ELEVENLABS_API_KEY=your-elevenlabs-api-key +ELEVENLABS_AGENT_ID=your-agent-id +``` + +| Variable | Required | Description | +|----------|----------|-------------| +| `ELEVENLABS_API_KEY` | Yes | Your ElevenLabs API key, used server-side for generating signed URLs | +| `ELEVENLABS_AGENT_ID` | No | Default agent ID. Can also be passed directly to `elevenlabsRealtimeToken()` | + +Get your API key from the [ElevenLabs dashboard](https://elevenlabs.io/). Create and configure agents in the **Conversational AI** section of the dashboard. + +## Text-to-Speech + +For one-shot speech generation (not realtime), use `elevenlabsSpeech` with `generateSpeech()`: + +```typescript +import { generateSpeech } from "@tanstack/ai"; +import { elevenlabsSpeech } from "@tanstack/ai-elevenlabs"; + +const result = await generateSpeech({ + adapter: elevenlabsSpeech("eleven_v3"), + text: "Hello from ElevenLabs!", + voice: "Rachel", + format: "mp3", +}); + +console.log(result.audio); // Base64-encoded audio +``` + +## Music & Sound Effects + +`elevenlabsAudio` covers both music generation and sound effects depending on the model: + +```typescript +import { generateAudio } from "@tanstack/ai"; +import { elevenlabsAudio } from "@tanstack/ai-elevenlabs"; + +// Music generation +const music = await generateAudio({ + adapter: elevenlabsAudio("music_v1"), + prompt: "An upbeat synthwave track for a product launch", +}); + +// Sound effects +const sfx = await generateAudio({ + adapter: elevenlabsAudio("eleven_text_to_sound_v2"), + prompt: "A glass shattering on concrete", +}); +``` + +## Transcription + +Transcribe audio with `elevenlabsTranscription`: + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { elevenlabsTranscription } from "@tanstack/ai-elevenlabs"; +import { audioFile } from "./audio"; + +const result = await generateTranscription({ + adapter: elevenlabsTranscription("scribe_v1"), + audio: audioFile, +}); + +console.log(result.text); +``` + +## API Reference + +### `elevenlabsRealtimeToken(options)` + +Creates an ElevenLabs realtime token adapter for server-side use with `realtimeToken()`. + +**Parameters:** + +- `options.agentId` - Agent ID from the ElevenLabs dashboard +- `options.overrides?.voiceId` - Custom voice ID +- `options.overrides?.systemPrompt` - Custom system prompt +- `options.overrides?.firstMessage` - First message the agent speaks +- `options.overrides?.language` - Language code + +**Returns:** A `RealtimeTokenAdapter` for use with `realtimeToken()`. + +### `elevenlabsRealtime(options?)` + +Creates an ElevenLabs realtime client adapter for use with `useRealtimeChat` or `RealtimeClient`. + +**Parameters:** + +- `options.connectionMode?` - `'websocket'` or `'webrtc'` (default: auto-detect) +- `options.debug?` - Enable debug logging + +**Returns:** A `RealtimeAdapter` for use with `useRealtimeChat()` or `RealtimeClient`. + +### `elevenlabsSpeech(model, config?)` / `createElevenLabsSpeech(model, apiKey, config?)` + +Creates an ElevenLabs text-to-speech adapter for use with `generateSpeech()`. + +### `elevenlabsAudio(model, config?)` / `createElevenLabsAudio(model, apiKey, config?)` + +Creates an ElevenLabs audio adapter that covers both music generation and sound effects (selected via the model id) for use with `generateAudio()`. + +### `elevenlabsTranscription(model, config?)` / `createElevenLabsTranscription(model, apiKey, config?)` + +Creates an ElevenLabs transcription adapter for use with `generateTranscription()`. + +## Limitations + +- **No text chat support** -- Use OpenAI, Anthropic, Gemini, or another text adapter for `chat()`. +- **No summarization** -- Use a text adapter for `summarize()`. +- **No image input** (realtime) -- ElevenLabs realtime does not support sending images during a conversation. +- **No runtime session updates** (realtime) -- Session configuration is fixed at connection time. +- **No time-domain audio data** (realtime) -- Frequency data and volume levels are available, but waveform data is not. +- **Agent required** (realtime) -- You must create and configure an agent in the ElevenLabs dashboard before using the realtime adapter. + +## Next Steps + +- [Realtime Voice Chat Guide](/ai/media/realtime-chat) - Complete guide to building realtime voice applications +- [OpenAI Adapter](/ai/adapters/openai) - Alternative realtime voice provider with WebRTC +- [Tools Guide](/ai/tools/tools) - Learn about the isomorphic tool system diff --git a/mintlify/ai/adapters/fal.md b/mintlify/ai/adapters/fal.md new file mode 100644 index 000000000..6fab4b262 --- /dev/null +++ b/mintlify/ai/adapters/fal.md @@ -0,0 +1,494 @@ +--- +title: fal.ai +id: fal-adapter +description: "Generate images and videos with 600+ models on fal.ai using TanStack AI — Nano Banana Pro, FLUX, and more via the @tanstack/ai-fal adapter." +keywords: + - tanstack ai + - fal.ai + - fal + - image generation + - video generation + - flux + - nano banana + - adapter +--- + +The fal.ai adapter provides access to 600+ models on the fal.ai platform for image, video, audio, speech, and transcription. Unlike text-focused adapters, the fal adapter is **media-focused** — it supports `generateImage()`, `generateVideo()`, `generateAudio()`, `generateSpeech()`, and `generateTranscription()` but does not support `chat()` or tools. + +For a full working example, see the [fal.ai example app](https://github.com/TanStack/ai/tree/main/examples/ts-react-media). + +## Installation + +```bash +npm install @tanstack/ai-fal +``` + +## Type Safety with String Literals + +The fal adapter provides full type safety when you pass the model ID as a **string literal**. This gives you autocomplete for `size` and `modelOptions` specific to that model. Always use string literals — not variables — when creating adapters: + +```typescript +import { falImage } from "@tanstack/ai-fal"; + +// Good — full type safety and autocomplete +const adapter = falImage("fal-ai/z-image/turbo"); +``` + +```typescript +import { falImage } from "@tanstack/ai-fal"; + +// Bad — no type inference for model-specific options +const modelId = "fal-ai/z-image/turbo"; +const adapter = falImage(modelId); +``` + +You can also pass any string for new models that fal.ai hasn't provided types for yet — you just won't get type safety on those endpoints. + +## Basic Usage + +```typescript +import { generateImage } from "@tanstack/ai"; +import { falImage } from "@tanstack/ai-fal"; + +const result = await generateImage({ + adapter: falImage("fal-ai/flux/dev"), + prompt: "A futuristic cityscape at sunset", + numberOfImages: 1, +}); + +console.log(result.images); +``` + +## Basic Usage - Custom API Key + +```typescript +import { generateImage } from "@tanstack/ai"; +import { falImage } from "@tanstack/ai-fal"; + +const adapter = falImage("fal-ai/flux/dev", { + apiKey: process.env.FAL_KEY!, +}); + +const result = await generateImage({ + adapter, + prompt: "A futuristic cityscape at sunset", + numberOfImages: 1, +}); +``` + +## Configuration + +```typescript +import { falImage, type FalClientConfig } from "@tanstack/ai-fal"; + +// Direct API key +const adapter = falImage("fal-ai/flux/dev", { + apiKey: "your-api-key", +}); + +// Using a proxy URL (for client-side usage) +const proxiedAdapter = falImage("fal-ai/flux/dev", { + apiKey: "your-api-key", + proxyUrl: "https://your-server.com/api/fal/proxy", +}); +``` + +## Example: Image Generation + +From the [fal.ai example app](https://github.com/TanStack/ai/tree/main/examples/ts-react-media): + +```typescript +import { generateImage } from "@tanstack/ai"; +import { falImage } from "@tanstack/ai-fal"; + +// Use string literals for the model to get full type safety +const result = await generateImage({ + adapter: falImage("fal-ai/nano-banana-pro"), + prompt: "A futuristic cityscape at sunset", + numberOfImages: 1, + size: "16:9_4K", + modelOptions: { + output_format: "jpeg", + }, +}); +``` + +## Example: Image with Model Options + +Each fal.ai model has its own type-safe options. The adapter uses fal.ai's types to provide autocomplete and type checking for model-specific parameters. There are 1000s of combinations. If you provide the proper model endpoint id as as string literal, you will then only be able to provide `size` and `modelOptions` that the model supports. + +_IMPORTANT_: It is possible to pass strings and new endpoint ids that Fal has not provided types for. You just won't get type safety on those endpoints. You'll find this happens for very new models. + +```typescript +import { generateImage } from "@tanstack/ai"; +import { falImage } from "@tanstack/ai-fal"; + +// Model-specific options are type-safe +const result = await generateImage({ + adapter: falImage("fal-ai/z-image/turbo"), + prompt: "A serene mountain landscape", + numberOfImages: 1, + size: "landscape_16_9", + modelOptions: { + acceleration: "high", + enable_prompt_expansion: true, + }, +}); +``` + +## Image Size Options + +The fal adapter supports a flexible `size` paramater that maps either to `image_size` or to `aspect_ratio` and `resolution` parameters: + +| | `size` | Maps To | +|--------|---------|---------| +| named | `"landscape_16_9"` | `image_size: "landscape_16_9"` | +| width x height (OpenAI) | `"1536x1024"` | | `image_size: "1536x1024"` | +| aspect ratio & resolution | `"16:9_4K"` | `aspect_ratio: "16:9"`, `resolution: "4K"` | +| aspect ratio only | `"16:9"` | `aspect_ratio: "16:9"` | + +```typescript ignore +// Aspect ratio only +size: "16:9" + +// Aspect ratio with resolution +size: "16:9_4K" + +// Named size (model-specific) +size: "landscape_16_9" +``` + +## Video Generation (Experimental) + +> **Note:** Video generation is an experimental feature and may change in future releases. In particular, this version of the adapter does not map the duration paramater + +Video generation uses a queue-based workflow: submit a job, poll for status, then retrieve the video URL when complete. + +```typescript +import { generateVideo, getVideoJobStatus } from "@tanstack/ai"; +import { falVideo } from "@tanstack/ai-fal"; +``` + +## Example: Text-to-Video + +From the [fal.ai example app](https://github.com/TanStack/ai/tree/main/examples/ts-react-media): + +```typescript +import { generateVideo, getVideoJobStatus } from "@tanstack/ai"; +import { falVideo } from "@tanstack/ai-fal"; + +// 1. Submit the video generation job +const adapter = falVideo("fal-ai/kling-video/v2.6/pro/text-to-video"); + +const job = await generateVideo({ + adapter, + prompt: "A timelapse of a flower blooming", + size: "16:9", + modelOptions: { + duration: "5", + }, +}); + +// 2. Poll for status +const status = await getVideoJobStatus({ + adapter, + jobId: job.jobId, +}); + +console.log(status.status); // "pending" | "processing" | "completed" +``` + +## Example: Image-to-Video + +```typescript +import { generateVideo } from "@tanstack/ai"; +import { falVideo } from "@tanstack/ai-fal"; + +const job = await generateVideo({ + adapter: falVideo("fal-ai/kling-video/v2.6/pro/image-to-video"), + prompt: "Animate this scene with gentle wind", + modelOptions: { + start_image_url: "https://example.com/image.jpg", + generate_audio: true, + duration: "5", + }, +}); +``` + +## Text-to-Speech + +Text-to-speech uses `falSpeech()` with the `generateSpeech()` activity. The adapter fetches the generated audio from fal's CDN and returns it as base64-encoded data to match the `TTSResult` contract. + +```typescript +import { generateSpeech } from "@tanstack/ai"; +import { falSpeech } from "@tanstack/ai-fal"; + +const result = await generateSpeech({ + adapter: falSpeech("fal-ai/kokoro/american-english"), + text: "Hello from fal!", + voice: "af_heart", + speed: 1.0, +}); + +// result.audio is a base64-encoded string +console.log(result.format); // e.g. "wav" +console.log(result.contentType); // e.g. "audio/wav" +``` + +### Google Gemini 3.1 Flash TTS + +Google's newest TTS model (`fal-ai/gemini-3.1-flash-tts`) supports 80+ languages and introduces **granular audio tags** for expressive control — you can embed speaker tags and style cues directly in the text. + +```typescript +import { generateSpeech } from "@tanstack/ai"; +import { falSpeech } from "@tanstack/ai-fal"; + +const result = await generateSpeech({ + adapter: falSpeech("fal-ai/gemini-3.1-flash-tts"), + text: "[warm, enthusiastic] Welcome to TanStack AI! [pause] Let's build something great.", + voice: "Kore", +}); +``` + +> **Note:** This model is newer than `@fal-ai/client@1.9.1`'s type map, so `modelOptions` won't autocomplete. The call still works — the fal adapter accepts any model ID as a string. Type-safe autocomplete will land when fal's SDK types catch up. + +### ElevenLabs v3 + +```typescript +import { generateSpeech } from "@tanstack/ai"; +import { falSpeech } from "@tanstack/ai-fal"; + +const result = await generateSpeech({ + adapter: falSpeech("fal-ai/elevenlabs/tts/eleven-v3"), + text: "Welcome to TanStack AI.", + modelOptions: { + voice: "Rachel", + stability: 0.5, + }, +}); +``` + +## Transcription + +Speech-to-text uses `falTranscription()` with the `generateTranscription()` activity. The `audio` input accepts a URL string, `Blob`, `File`, or `ArrayBuffer` — `ArrayBuffer` is automatically wrapped in a `Blob` for upload. + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { falTranscription } from "@tanstack/ai-fal"; + +const result = await generateTranscription({ + adapter: falTranscription("fal-ai/whisper"), + audio: "https://example.com/recording.mp3", + language: "en", +}); + +console.log(result.text); +console.log(result.language); + +// When the model returns word/segment timestamps, they're mapped to result.segments +for (const segment of result.segments ?? []) { + console.log(`[${segment.start}s → ${segment.end}s] ${segment.text}`); +} +``` + +## Audio Generation (Music & Sound Effects) + +Music and sound-effect generation uses `falAudio()` with the `generateAudio()` activity. Unlike TTS, the result is returned as a URL in `result.audio.url` (you can fetch it yourself if you need raw bytes). + +```typescript +import { generateAudio } from "@tanstack/ai"; +import { falAudio } from "@tanstack/ai-fal"; + +// Music generation with MiniMax Music 2.6 (latest) +const music = await generateAudio({ + adapter: falAudio("fal-ai/minimax-music/v2.6"), + prompt: "City Pop, 80s retro, groovy synth bass, warm female vocal, 104 BPM, nostalgic urban night", +}); + +console.log(music.audio.url); +``` + +```typescript +import { generateAudio } from "@tanstack/ai"; +import { falAudio } from "@tanstack/ai-fal"; + +// DiffRhythm with explicit lyrics +const lyrical = await generateAudio({ + adapter: falAudio("fal-ai/diffrhythm"), + prompt: "An upbeat electronic track with synths", + modelOptions: { + lyrics: "[verse]\nHello world\n[chorus]\nLa la la", + }, +}); +``` + +```typescript +import { generateAudio } from "@tanstack/ai"; +import { falAudio } from "@tanstack/ai-fal"; + +// Sound effects +const sfx = await generateAudio({ + adapter: falAudio("fal-ai/elevenlabs/sound-effects/v2"), + prompt: "Thunderclap with rain", + duration: 5, +}); +``` + +## Popular Models + +### Image Models + +| Model | Description | +|-------|-------------| +| `fal-ai/nano-banana-pro` | Fast, high-quality image generation (4K) | +| `fal-ai/flux-2/klein/9b` | Enhanced realism, crisp text generation | +| `fal-ai/z-image/turbo` | Super fast 6B parameter model | +| `xai/grok-imagine-image` | xAI highly aesthetic images with prompt enhancement | + +### Video Models + +| Model | Mode | Description | +|-------|------|-------------| +| `fal-ai/kling-video/v2.6/pro/text-to-video` | Text-to-Video | High-quality text-to-video | +| `fal-ai/kling-video/v2.6/pro/image-to-video` | Image-to-Video | Animate images with Kling | +| `fal-ai/veo3.1` | Text-to-Video | Google Veo text-to-video | +| `fal-ai/veo3.1/image-to-video` | Image-to-Video | Google Veo image-to-video | +| `xai/grok-imagine-video/text-to-video` | Text-to-Video | xAI video from text | +| `xai/grok-imagine-video/image-to-video` | Image-to-Video | xAI animate images to video | +| `fal-ai/ltx-2/text-to-video/fast` | Text-to-Video | Fast text-to-video | +| `fal-ai/ltx-2/image-to-video/fast` | Image-to-Video | Fast image-to-video animation | + +### Text-to-Speech Models + +| Model | Description | +|-------|-------------| +| `fal-ai/gemini-3.1-flash-tts` | **New** — Google's flagship TTS with 80+ languages and expressive audio tags | +| `fal-ai/elevenlabs/tts/eleven-v3` | ElevenLabs v3 expressive multi-voice TTS | +| `fal-ai/elevenlabs/tts/turbo-v2.5` | Low-latency ElevenLabs TTS | +| `fal-ai/minimax/speech-2.6-hd` | MiniMax HD speech synthesis | +| `fal-ai/minimax/speech-2.6-turbo` | MiniMax low-latency variant | +| `fal-ai/kokoro/american-english` | Kokoro multilingual TTS — also `british-english`, `french`, `spanish`, `italian`, `japanese`, `mandarin-chinese`, `hindi`, `brazilian-portuguese` | +| `fal-ai/inworld-tts` | Inworld TTS-1.5 Max | +| `fal-ai/chatterbox/text-to-speech/multilingual` | Chatterbox multilingual TTS | +| `fal-ai/dia-tts` | Dia expressive dialogue TTS | +| `fal-ai/orpheus-tts` | Orpheus open-source TTS | +| `fal-ai/f5-tts` | F5-TTS voice cloning | +| `fal-ai/vibevoice/7b` | VibeVoice 7B conversational TTS | + +### Transcription Models + +| Model | Description | +|-------|-------------| +| `fal-ai/whisper` | OpenAI Whisper on fal infra | +| `fal-ai/wizper` | Faster-whisper variant with word-level timestamps | +| `fal-ai/speech-to-text/turbo` | Turbo STT with diarization | +| `fal-ai/elevenlabs/speech-to-text` | ElevenLabs STT | + +### Audio / Music Models + +| Model | Mode | Description | +|-------|------|-------------| +| `fal-ai/minimax-music/v2.6` | Music | **New** — MiniMax Music 2.6, full vocal + instrumental compositions from a prompt | +| `fal-ai/minimax-music/v2.5` | Music | MiniMax Music 2.5 | +| `fal-ai/minimax-music/v2` | Music | MiniMax Music v2 — supports `lyrics_prompt` | +| `fal-ai/diffrhythm` | Music | DiffRhythm — prompt + lyrics | +| `fal-ai/lyria2` | Music | Google Lyria 2 high-fidelity music | +| `fal-ai/stable-audio-25/text-to-audio` | Music / Audio | Stability AI Stable Audio 2.5 | +| `fal-ai/mmaudio-v2/text-to-audio` | Audio | MMAudio v2 text-to-audio | +| `fal-ai/elevenlabs/sound-effects/v2` | SFX | ElevenLabs sound-effect generation | +| `fal-ai/beatoven/sound-effect-generation` | SFX | Beatoven professional sound effects | +| `fal-ai/thinksound` | Audio | Thinksound reasoning-based audio generation | + +> **Very new models** (e.g. `gemini-3.1-flash-tts`, `minimax-music/v2.6`, `beatoven/sound-effect-generation`) may not yet appear in `@fal-ai/client`'s type map — they still work as plain string model IDs, you just won't get autocomplete for their `modelOptions`. + +## Environment Variables + +Create an API key at [fal.ai](https://fal.ai) and set it in your environment: + +```bash +FAL_KEY=your-fal-api-key +``` + +## API Reference + +### `falImage(model, config?)` + +Creates a fal.ai image adapter using the `FAL_KEY` environment variable or an explicit config. + +**Parameters:** + +- `model` - The fal.ai model ID (e.g., `"fal-ai/flux/dev"`) +- `config.apiKey?` - Your fal.ai API key (falls back to `FAL_KEY` env var) +- `config.proxyUrl?` - Proxy URL for client-side usage + +**Returns:** A `FalImageAdapter` instance for use with `generateImage()`. + +### `falVideo(model, config?)` + +Creates a fal.ai video adapter using the `FAL_KEY` environment variable or an explicit config. + +**Parameters:** + +- `model` - The fal.ai model ID (e.g., `"fal-ai/kling-video/v2.6/pro/text-to-video"`) +- `config.apiKey?` - Your fal.ai API key (falls back to `FAL_KEY` env var) +- `config.proxyUrl?` - Proxy URL for client-side usage + +**Returns:** A `FalVideoAdapter` instance for use with `generateVideo()` and `getVideoJobStatus()`. + +### `falSpeech(model, config?)` + +Creates a fal.ai text-to-speech adapter. + +**Parameters:** + +- `model` - The fal.ai TTS model ID (e.g., `"fal-ai/kokoro/american-english"`) +- `config.apiKey?` - Your fal.ai API key (falls back to `FAL_KEY` env var) +- `config.proxyUrl?` - Proxy URL for client-side usage + +**Returns:** A `FalSpeechAdapter` instance for use with `generateSpeech()`. The adapter fetches the generated audio URL from fal and returns it as base64 in `result.audio`. + +### `falTranscription(model, config?)` + +Creates a fal.ai transcription (speech-to-text) adapter. + +**Parameters:** + +- `model` - The fal.ai STT model ID (e.g., `"fal-ai/whisper"`) +- `config.apiKey?` - Your fal.ai API key (falls back to `FAL_KEY` env var) +- `config.proxyUrl?` - Proxy URL for client-side usage + +**Returns:** A `FalTranscriptionAdapter` instance for use with `generateTranscription()`. + +### `falAudio(model, config?)` + +Creates a fal.ai audio generation adapter (music and sound effects). + +**Parameters:** + +- `model` - The fal.ai audio model ID (e.g., `"fal-ai/diffrhythm"`, `"fal-ai/minimax-music/v2"`) +- `config.apiKey?` - Your fal.ai API key (falls back to `FAL_KEY` env var) +- `config.proxyUrl?` - Proxy URL for client-side usage + +**Returns:** A `FalAudioAdapter` instance for use with `generateAudio()`. The result contains a URL at `result.audio.url`. + +### `getFalApiKeyFromEnv()` + +Reads the `FAL_KEY` environment variable. Throws if not set. + +**Returns:** The API key string. + +### `configureFalClient(config?)` + +Configures the underlying `@fal-ai/client`. Called automatically by adapter constructors. Uses `proxyUrl` if provided, otherwise sets credentials from the API key. + +## Limitations + +- **No text/chat support** — Use OpenAI, Anthropic, Gemini, or another text adapter for `chat()` +- **No tools support** — Tool definitions are not applicable to media generation +- **No summarization** — Use a text adapter for `summarize()` +- **Video is experimental** — The video generation API may change in future releases + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Other Adapters](/ai/adapters/openai) - Explore other providers diff --git a/mintlify/ai/adapters/gemini.md b/mintlify/ai/adapters/gemini.md new file mode 100644 index 000000000..47865804c --- /dev/null +++ b/mintlify/ai/adapters/gemini.md @@ -0,0 +1,746 @@ +--- +title: Google Gemini +id: gemini-adapter +order: 3 +description: "Use Google Gemini with TanStack AI — text, image generation via Imagen and Gemini native (NanoBanana), and experimental TTS via @tanstack/ai-gemini." +keywords: + - tanstack ai + - gemini + - google gemini + - imagen + - nano banana + - image generation + - adapter + - google ai +--- + +The Google Gemini adapter provides access to Google's Gemini models, including text generation, image generation with both Imagen and Gemini native image models (NanoBanana), and experimental text-to-speech. + +For a full working example with image generation, see the [media generation example app](https://github.com/TanStack/ai/tree/main/examples/ts-react-media). + +## Installation + +```bash +npm install @tanstack/ai-gemini +``` + +## Basic Usage + +```typescript +import { chat } from "@tanstack/ai"; +import { geminiText } from "@tanstack/ai-gemini"; + +const stream = chat({ + adapter: geminiText("gemini-3.1-pro-preview"), + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Basic Usage - Custom API Key + +```typescript +import { chat } from "@tanstack/ai"; +import { createGeminiChat } from "@tanstack/ai-gemini"; + +const adapter = createGeminiChat("gemini-3.1-pro-preview", process.env.GEMINI_API_KEY!, { + // ... your config options +}); + +const stream = chat({ + adapter, + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Configuration + +```typescript +import { createGeminiChat, type GeminiTextConfig } from "@tanstack/ai-gemini"; + +const config: Omit = { + httpOptions: { + baseUrl: "https://generativelanguage.googleapis.com/v1beta", // Optional + }, +}; + +const adapter = createGeminiChat("gemini-3.1-pro-preview", process.env.GEMINI_API_KEY!, config); +``` + + +## Example: Chat Completion + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { geminiText } from "@tanstack/ai-gemini"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: geminiText("gemini-3.1-pro-preview"), + messages, + }); + + return toServerSentEventsResponse(stream); +} +``` + +## Example: With Tools + +```typescript +import { chat, toServerSentEventsResponse, toolDefinition } from "@tanstack/ai"; +import { geminiText } from "@tanstack/ai-gemini"; +import { z } from "zod"; + +const getCalendarEventsDef = toolDefinition({ + name: "get_calendar_events", + description: "Get calendar events for a date", + inputSchema: z.object({ + date: z.string(), + }), +}); + +const getCalendarEvents = getCalendarEventsDef.server(async ({ date }) => { + // Fetch calendar events + return { events: [] }; +}); + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: geminiText("gemini-3.1-pro-preview"), + messages, + tools: [getCalendarEvents], + }); + + return toServerSentEventsResponse(stream); +} +``` + +## Stateful Conversations — Interactions API (Experimental) + +Gemini's [Interactions API](https://ai.google.dev/gemini-api/docs/interactions) (currently in Beta) offers server-side conversation state — the Gemini equivalent of OpenAI's Responses API. Instead of replaying the full message history on every turn, you pass a `previous_interaction_id` and the server retains the transcript. This also improves cache hit rates for repeated prefixes. + +The `geminiTextInteractions` adapter routes through `client.interactions.create` and surfaces the server-assigned interaction id via an AG-UI `CUSTOM` event (`name: 'gemini.interactionId'`) emitted just before `RUN_FINISHED`, so you can chain turns. + +> **⚠️ Experimental.** Google marks the Interactions API as Beta and explicitly flags possible breaking changes until it reaches general availability. The adapter is exported from the `@tanstack/ai-gemini/experimental` subpath so the experimental status is load-bearing in your editor and bundle. Text output, function tools, and the built-in tools `google_search`, `code_execution`, `url_context`, `file_search`, and `computer_use` are supported. `google_search_retrieval`, `google_maps`, and `mcp_server` still throw on this adapter — use `geminiText()` for those or wait for follow-up work. + +### Basic Usage + +```typescript ignore +import { chat } from "@tanstack/ai"; +import { geminiTextInteractions } from "@tanstack/ai-gemini/experimental"; + +// Turn 1: introduce yourself, capture the interaction id. +let interactionId: string | undefined; + +for await (const chunk of chat({ + adapter: geminiTextInteractions("gemini-3.5-flash"), + messages: [{ role: "user", content: "Hi, my name is Amir." }], +})) { + if ( + chunk.type === "CUSTOM" && + chunk.name === "gemini.interactionId" && + chunk.value && + typeof chunk.value === "object" && + "interactionId" in chunk.value + ) { + interactionId = String(chunk.value.interactionId); + } +} + +// Turn 2: only send the new turn's content — the server has the history. +for await (const chunk of chat({ + adapter: geminiTextInteractions("gemini-3.5-flash"), + messages: [{ role: "user", content: "What is my name?" }], + modelOptions: { + previous_interaction_id: interactionId, + }, +})) { + // ...stream "Your name is Amir." back to the client. +} +``` + +### Wiring with `useChat` (React) + +The Interactions API is stateful and **does not accept multi-turn history without a `previous_interaction_id`** — if a chat client sends `[user, assistant, user]` to a fresh interaction the adapter throws `cannot send prior conversation history on a fresh interaction`. To make `useChat` work, persist the server-assigned id and send it back on the next turn: + +**Server route** (e.g. TanStack Start handler): + +```typescript +import { + chat, + chatParamsFromRequestBody, + toServerSentEventsResponse, +} from "@tanstack/ai"; +import { geminiTextInteractions } from "@tanstack/ai-gemini/experimental"; + +export async function POST({ request }: { request: Request }) { + const params = await chatParamsFromRequestBody(await request.json()); + + // The client sends body.previousInteractionId; AG-UI maps `body` into + // `forwardedProps` on the wire. + const previousInteractionId = + typeof params.forwardedProps.previousInteractionId === "string" + ? params.forwardedProps.previousInteractionId + : undefined; + + const stream = chat({ + adapter: geminiTextInteractions("gemini-3.5-flash"), + messages: params.messages, + modelOptions: { + previous_interaction_id: previousInteractionId, + store: true, // required for chaining on the next turn + }, + }); + + return toServerSentEventsResponse(stream); +} +``` + +**React client**: + +```tsx +import { useEffect, useMemo, useState } from "react"; +import { fetchServerSentEvents, useChat } from "@tanstack/ai-react"; + +function GeminiChat() { + const [interactionId, setInteractionId] = useState(); + + const body = useMemo( + () => (interactionId ? { previousInteractionId: interactionId } : {}), + [interactionId], + ); + + const { messages, setMessages, sendMessage } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + body, + onCustomEvent: (eventType, data) => { + if ( + eventType === "gemini.interactionId" && + typeof data === "object" && + data !== null && + "interactionId" in data + ) { + setInteractionId(String(data.interactionId)); + } + }, + }); + + // Switching provider/model resets the server-side chain — drop the id + // AND the local message history together, otherwise the next turn + // ships multi-turn messages with no previous_interaction_id and the + // adapter errors out. + const [provider, setProvider] = useState("gemini-interactions"); + useEffect(() => { + setInteractionId(undefined); + setMessages([]); + }, [provider]); + + // ...render messages, call sendMessage(input) +} +``` + +The full working example is in [`examples/ts-react-chat`](https://github.com/TanStack/ai/tree/main/examples/ts-react-chat) — see `src/routes/index.tsx` for the client and `src/routes/api.tanchat.ts` for the route. + +### How it differs from `geminiText` + +| Concern | `geminiText` | `geminiTextInteractions` | +| --- | --- | --- | +| Underlying endpoint | `models:generateContent` | `interactions:create` | +| Conversation state | Stateless — send full history each turn | Stateful — server retains transcript via `previous_interaction_id` | +| Provider options shape | camelCase (`stopSequences`, `responseModalities`, `safetySettings`) | snake_case (`generation_config`, `response_modalities`, `previous_interaction_id`) | +| Built-in tools | `google_search`, `code_execution`, `url_context`, `file_search`, `google_maps`, `google_search_retrieval`, `computer_use` | `google_search`, `code_execution`, `url_context`, `file_search`, `computer_use` (only the first four stream `CUSTOM` event activity; `computer_use` is accepted in the request but does not currently emit per-delta events) | +| Stability | GA | Experimental (Google Beta) | + +### Provider Options + +The adapter exposes Interactions-specific options on `modelOptions`: + +```typescript +import { chat } from "@tanstack/ai"; +import { geminiTextInteractions } from "@tanstack/ai-gemini/experimental"; + +const stream = chat({ + adapter: geminiTextInteractions("gemini-3.5-flash"), + messages: [{ role: "user", content: "Hello!" }], + modelOptions: { + // Stateful chaining — passed only on turn 2+. + previous_interaction_id: "int_abc123", + + // Persist the interaction server-side (default true). Must be true for + // previous_interaction_id to work on the *next* turn. + store: true, + + // Per-request system instruction (interaction-scoped — re-specify each turn). + system_instruction: "You are a helpful assistant.", + + // snake_case generation config distinct from geminiText's camelCase one. + generation_config: { + thinking_level: "low", + thinking_summaries: "auto", + stop_sequences: [""], + }, + + response_modalities: ["text"], + }, +}); +``` + +### Reading the interaction id + +The server's interaction id arrives as an AG-UI `CUSTOM` event emitted just before `RUN_FINISHED`: + +```typescript ignore +for await (const chunk of stream) { + if ( + chunk.type === "CUSTOM" && + chunk.name === "gemini.interactionId" && + typeof chunk.value === "object" && + chunk.value !== null && + "interactionId" in chunk.value + ) { + const id = String(chunk.value.interactionId); + // Persist `id` wherever you store per-user conversation pointers — + // pass it back on the next turn as `previous_interaction_id`. + } +} +``` + +### Caveats + +- **Multi-turn history requires `previous_interaction_id`.** The Interactions API has no stateless replay path — sending more than one message in `messages` without a `previous_interaction_id` throws. Chat UIs that maintain local history must capture the server-assigned id and chain (see [Wiring with `useChat`](#wiring-with-usechat-react)). On provider/model switch, also clear the local message buffer. +- **Tools, `system_instruction`, and `generation_config` are interaction-scoped.** Per Google's docs these are NOT inherited from a prior interaction via `previous_interaction_id` — pass them again on every turn you need them. +- `store: false` is incompatible with `previous_interaction_id` (no state to recall) and with `background: true`. +- Retention (as of the time of writing): **55 days on the Paid Tier, 1 day on the Free Tier.** See [Google's Interactions API docs](https://ai.google.dev/gemini-api/docs/interactions) for current retention policy. +- Built-in tools in scope (`google_search`, `code_execution`, `url_context`, `file_search`, `computer_use`) are wired through as request tools. Per-delta activity for the four search/exec tools streams back as AG-UI `CUSTOM` events — `gemini.googleSearchCall` / `gemini.googleSearchResult` (and the matching `codeExecutionCall`/`Result`, `urlContextCall`/`Result`, `fileSearchCall`/`Result`) — carrying the raw Interactions delta. `computer_use` is accepted in the request but the Interactions API does not currently emit per-delta `CUSTOM` events for it. Function-tool `TOOL_CALL_*` events are unchanged, and `finishReason` stays `stop` when only built-in tools ran. +- `google_search_retrieval`, `google_maps`, and `mcp_server` still throw a targeted error on this adapter. Use `geminiText()` for the first two, or wait for a dedicated follow-up for `mcp_server`. +- Image and audio output via Interactions aren't routed through this adapter yet — it's text-only. Use `geminiImage` / `geminiSpeech` for non-text generation for now. + +## Model Options + +Gemini supports various model-specific options. Sampling parameters live here too — `temperature`, `topP`, and `maxOutputTokens` — rather than as root-level props on `chat()`: + +```typescript +import { chat } from "@tanstack/ai"; +import { geminiText } from "@tanstack/ai-gemini"; + +const stream = chat({ + adapter: geminiText("gemini-3.1-pro-preview"), + messages: [{ role: "user", content: "Hello!" }], + modelOptions: { + maxOutputTokens: 2048, + temperature: 0.7, + topP: 0.9, + topK: 40, + stopSequences: ["END"], + }, +}); +``` + +> If you previously passed `temperature` / `topP` / `maxTokens` at the root of `chat()`, see [Moving Sampling Options into modelOptions](/ai/migration/sampling-options-to-model-options). + +### Thinking + +Enable thinking for models that support it: + +```typescript ignore +modelOptions: { + thinking: { + includeThoughts: true, + }, +} +``` + +### Structured Output + +Configure structured output format: + +```typescript ignore +modelOptions: { + responseMimeType: "application/json", +} +``` + +## Summarization + +Summarize long text content: + +```typescript +import { summarize } from "@tanstack/ai"; +import { geminiSummarize } from "@tanstack/ai-gemini"; + +const result = await summarize({ + adapter: geminiSummarize("gemini-3.1-pro-preview"), + text: "Your long text to summarize...", + maxLength: 100, + style: "concise", // "concise" | "bullet-points" | "paragraph" +}); + +console.log(result.summary); +``` + +## Image Generation + +The Gemini adapter supports two types of image generation: + +- **Gemini native image models** (NanoBanana) — Use the `generateContent` API with models like `gemini-3.1-flash-image-preview`. These support extended resolution tiers (1K, 2K, 4K) and aspect ratio control. +- **Imagen models** — Use the `generateImages` API with models like `imagen-4.0-generate-001`. These are dedicated image generation models with WIDTHxHEIGHT sizing. + +The adapter automatically routes to the correct API based on the model name — models starting with `gemini-` use `generateContent`, while `imagen-` models use `generateImages`. + +### Example: Gemini Native Image Generation (NanoBanana) + +From the [media generation example app](https://github.com/TanStack/ai/tree/main/examples/ts-react-media): + +```typescript +import { generateImage } from "@tanstack/ai"; +import { geminiImage } from "@tanstack/ai-gemini"; + +const result = await generateImage({ + adapter: geminiImage("gemini-3.1-flash-image-preview"), + prompt: "A futuristic cityscape at sunset", + numberOfImages: 1, + size: "16:9_4K", +}); + +console.log(result.images); +``` + +### Example: Imagen + +```typescript +import { generateImage } from "@tanstack/ai"; +import { geminiImage } from "@tanstack/ai-gemini"; + +const result = await generateImage({ + adapter: geminiImage("imagen-4.0-generate-001"), + prompt: "A futuristic cityscape at sunset", + numberOfImages: 1, +}); + +console.log(result.images); +``` + +### Image Size Options + +#### Gemini Native Models (NanoBanana) + +Gemini native image models use a template literal size format combining aspect ratio and resolution tier: + +```typescript ignore +// Format: "aspectRatio_resolution" +size: "16:9_4K" +size: "1:1_2K" +size: "9:16_1K" +``` + +| Component | Values | +|-----------|--------| +| Aspect Ratio | `1:1`, `2:3`, `3:2`, `3:4`, `4:3`, `9:16`, `16:9`, `21:9` | +| Resolution | `1K`, `2K`, `4K` | + +#### Imagen Models + +Imagen models use WIDTHxHEIGHT format, which maps to aspect ratios internally: + +| Size | Aspect Ratio | +|------|-------------| +| `1024x1024` | 1:1 | +| `1920x1080` | 16:9 | +| `1080x1920` | 9:16 | + +Alternatively, you can specify the aspect ratio directly in Model Options: + +```typescript +import { generateImage } from "@tanstack/ai"; +import { geminiImage } from "@tanstack/ai-gemini"; + +const result = await generateImage({ + adapter: geminiImage("imagen-4.0-generate-001"), + prompt: "A landscape photo", + modelOptions: { + aspectRatio: "16:9", + }, +}); +``` + +### Image Model Options + +```typescript ignore +import { generateImage } from "@tanstack/ai"; +import { geminiImage } from "@tanstack/ai-gemini"; + +const result = await generateImage({ + adapter: geminiImage("imagen-4.0-generate-001"), + prompt: "...", + modelOptions: { + aspectRatio: "16:9", // "1:1" | "3:4" | "4:3" | "9:16" | "16:9" + personGeneration: "DONT_ALLOW", // Control person generation + safetyFilterLevel: "BLOCK_SOME", // Safety filtering + }, +}); +``` + +## Text-to-Speech (Experimental) + +> **Note:** Gemini TTS is experimental and may require the Live API for full functionality. + +Generate speech from text: + +```typescript +import { generateSpeech } from "@tanstack/ai"; +import { geminiSpeech } from "@tanstack/ai-gemini"; + +const result = await generateSpeech({ + adapter: geminiSpeech("gemini-3.1-flash-tts-preview"), + text: "Hello from Gemini TTS!", +}); + +console.log(result.audio); // Base64 encoded audio +``` + +## Environment Variables + +Set your API key in environment variables: + +```bash +GEMINI_API_KEY=your-api-key-here +# or +GOOGLE_API_KEY=your-api-key-here +``` + +## Getting an API Key + +1. Go to [Google AI Studio](https://aistudio.google.com/apikey) +2. Create a new API key +3. Add it to your environment variables + +## Popular Image Models + +### Gemini Native Image Models (NanoBanana) + +These models use the `generateContent` API and support resolution tiers (1K, 2K, 4K). + +| Model | Description | +|-------|-------------| +| `gemini-3.1-flash-image-preview` | Latest and fastest Gemini native image generation | +| `gemini-3.1-flash-lite-image` | Nano Banana 2 Lite — ultra-low-latency, low-cost image generation | +| `gemini-3-pro-image-preview` | Higher quality Gemini native image generation | +| `gemini-2.5-flash-image` | Gemini 2.5 Flash with image generation | + +### Imagen Models + +These models use the dedicated `generateImages` API. + +| Model | Description | +|-------|-------------| +| `imagen-4.0-ultra-generate-001` | Best quality Imagen image generation | +| `imagen-4.0-generate-001` | High quality Imagen image generation | +| `imagen-4.0-fast-generate-001` | Fast Imagen image generation | + +## API Reference + +Every factory pair follows the same shape: the short factory (`geminiText`, `geminiImage`, …) reads `GEMINI_API_KEY` (or `GOOGLE_API_KEY`) from the environment, while the `create*` variant takes an explicit API key. Both take `model` as the first argument. + +### `geminiText(model, config?)` / `createGeminiChat(model, apiKey, config?)` + +Creates a Gemini text/chat adapter. + +**Parameters:** + +- `model` - Gemini chat model id (e.g. `"gemini-3.1-pro-preview"`) +- `config?.baseURL` - Custom base URL (optional) + +### `geminiTextInteractions(model, config?)` / `createGeminiTextInteractions(model, apiKey, config?)` (experimental) + +Creates a Gemini Interactions API text adapter. Backs the stateful conversation pattern via `previous_interaction_id`. Exported from `@tanstack/ai-gemini/experimental`. + +**Parameters:** + +- `model` - Gemini chat model id (e.g. `"gemini-3.5-flash"`) +- `config?.baseURL` - Custom base URL (optional) + +### `geminiSummarize(model, config?)` / `createGeminiSummarize(model, apiKey, config?)` + +Creates a Gemini summarization adapter. + +### `geminiImage(model, config?)` / `createGeminiImage(model, apiKey, config?)` + +Creates a Gemini image adapter. Automatically routes to the correct API based on the model name — `gemini-*` models use `generateContent`, `imagen-*` models use `generateImages`. + +### `geminiSpeech(model, config?)` / `createGeminiSpeech(model, apiKey, config?)` + +Creates a Gemini text-to-speech adapter. _Experimental._ + +### `geminiAudio(model, config?)` / `createGeminiAudio(model, apiKey, config?)` + +Creates a Gemini Lyria music generation adapter. _Experimental._ + +## Next Steps + +- [Image Generation Guide](/ai/media/image-generation) - Learn more about image generation +- [Media Generation Example](https://github.com/TanStack/ai/tree/main/examples/ts-react-media) - Full working example with Gemini and fal.ai +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about tools +- [Other Adapters](/ai/adapters/openai) - Explore other providers + +## Provider Tools + +Google Gemini exposes several native tools beyond user-defined function calls. +Import them from `@tanstack/ai-gemini/tools` and pass them into +`chat({ tools: [...] })`. + +> For the full concept, a comparison matrix, and type-gating details, see +> [Provider Tools](/ai/tools/provider-tools). + +### `codeExecutionTool` + +Enables Gemini to execute Python code in a sandboxed environment and return +results inline. Takes no arguments — include it in the `tools` array to +activate code execution. + +```typescript +import { chat } from "@tanstack/ai"; +import { geminiText } from "@tanstack/ai-gemini"; +import { codeExecutionTool } from "@tanstack/ai-gemini/tools"; + +const stream = chat({ + adapter: geminiText("gemini-3.1-pro-preview"), + messages: [{ role: "user", content: "Calculate the first 10 Fibonacci numbers" }], + tools: [codeExecutionTool()], +}); +``` + +**Supported models:** Gemini 1.5 Pro, Gemini 2.x, Gemini 2.5 and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `fileSearchTool` + +Searches files that have been uploaded to the Gemini File API. Pass a +`FileSearch` config object with the corpus and file IDs to scope the search. + +```typescript +import { chat } from "@tanstack/ai"; +import { geminiText } from "@tanstack/ai-gemini"; +import { fileSearchTool } from "@tanstack/ai-gemini/tools"; + +const stream = chat({ + adapter: geminiText("gemini-3.1-pro-preview"), + messages: [{ role: "user", content: "Find the quarterly revenue figures" }], + tools: [ + fileSearchTool({ + fileSearchStoreNames: ["fileSearchStores/my-file-search-store-123"], + }), + ], +}); +``` + +**Supported models:** Gemini 2.x and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `googleSearchTool` + +Enables Gemini to query Google Search and incorporate grounded search results +into its response. Pass an optional `GoogleSearch` config or call with no +arguments to use defaults. + +```typescript +import { chat } from "@tanstack/ai"; +import { geminiText } from "@tanstack/ai-gemini"; +import { googleSearchTool } from "@tanstack/ai-gemini/tools"; + +const stream = chat({ + adapter: geminiText("gemini-3.1-pro-preview"), + messages: [{ role: "user", content: "What's the weather in Tokyo right now?" }], + tools: [googleSearchTool()], +}); +``` + +**Supported models:** Gemini 1.5 Pro, Gemini 2.x, Gemini 2.5. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `googleSearchRetrievalTool` + +A retrieval-augmented variant of Google Search that returns ranked passages +from the web with configurable dynamic retrieval mode. Pass an optional +`GoogleSearchRetrieval` config. + +```typescript ignore +import { chat } from "@tanstack/ai"; +import { geminiText } from "@tanstack/ai-gemini"; +import { googleSearchRetrievalTool } from "@tanstack/ai-gemini/tools"; + +const stream = chat({ + adapter: geminiText("gemini-3.1-pro-preview"), + messages: [{ role: "user", content: "Explain the latest JavaScript proposals" }], + tools: [ + googleSearchRetrievalTool({ + dynamicRetrievalConfig: { mode: "MODE_DYNAMIC", dynamicThreshold: 0.7 }, + }), + ], +}); +``` + +**Supported models:** Gemini 1.5 Pro and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `googleMapsTool` + +Connects Gemini to the Google Maps API for location-aware queries such as +directions, place search, and geocoding. Pass an optional `GoogleMaps` config +or call with no arguments. + +```typescript +import { chat } from "@tanstack/ai"; +import { geminiText } from "@tanstack/ai-gemini"; +import { googleMapsTool } from "@tanstack/ai-gemini/tools"; + +const stream = chat({ + adapter: geminiText("gemini-2.5-pro"), + messages: [{ role: "user", content: "Find coffee shops near Union Square, SF" }], + tools: [googleMapsTool()], +}); +``` + +**Supported models:** Gemini 2.5 and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `urlContextTool` + +Fetches and includes the content of URLs mentioned in the conversation so +Gemini can reason over live web pages. Takes no arguments. + +```typescript +import { chat } from "@tanstack/ai"; +import { geminiText } from "@tanstack/ai-gemini"; +import { urlContextTool } from "@tanstack/ai-gemini/tools"; + +const stream = chat({ + adapter: geminiText("gemini-3.1-pro-preview"), + messages: [{ role: "user", content: "Summarise https://example.com/article" }], + tools: [urlContextTool()], +}); +``` + +**Supported models:** Gemini 2.x and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `computerUseTool` + +Allows Gemini to observe a virtual desktop via screenshots and interact with +it using predefined computer-use functions. Provide the `environment` and +optionally restrict callable functions via `excludedPredefinedFunctions`. + +```typescript ignore +import { chat } from "@tanstack/ai"; +import { geminiText } from "@tanstack/ai-gemini"; +import { computerUseTool } from "@tanstack/ai-gemini/tools"; + +const stream = chat({ + adapter: geminiText("gemini-3.1-pro-preview"), + messages: [{ role: "user", content: "Navigate to example.com in the browser" }], + tools: [ + computerUseTool({ + environment: "browser", + }), + ], +}); +``` + +**Supported models:** Gemini 2.5 and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). diff --git a/mintlify/ai/adapters/grok-build.md b/mintlify/ai/adapters/grok-build.md new file mode 100644 index 000000000..0e60f4041 --- /dev/null +++ b/mintlify/ai/adapters/grok-build.md @@ -0,0 +1,178 @@ +--- +title: Grok Build +id: grok-build-adapter +order: 15 +description: "Use xAI's Grok Build coding agent as a chat backend in TanStack AI — a sandbox harness that runs the grok CLI against a real workspace, with tool bridging via @tanstack/ai-grok-build." +keywords: + - tanstack ai + - grok + - grok build + - xai + - harness + - agent + - coding agent + - adapter +--- + +The Grok Build adapter runs xAI's **Grok Build** coding agent as a chat backend. +Unlike HTTP provider adapters, this is a **harness adapter**: Grok Build runs +its own agent loop and executes its own tools — shell commands, file edits, +search — by spawning the `grok` CLI **inside a sandbox**. Each `chat()` call runs +one full harness turn; the harness's tool activity streams back as +already-resolved tool-call events your UI can render. + +> **Requires a sandbox.** `grok-build` declares `requires: [SandboxCapability]`, +> so `chat()` errors at the call site unless you provide a sandbox with +> `withSandbox(...)` middleware. The sandbox — your laptop, a Docker container, +> or a cloud VM — is the filesystem and safety boundary the agent runs in. See +> the [Sandboxes overview](/ai/sandbox/overview) for the full picture. + +## Installation + +```bash +npm install @tanstack/ai-grok-build @tanstack/ai-sandbox +``` + +You also need a sandbox provider (e.g. `@tanstack/ai-sandbox-docker`) and the +`grok` CLI available inside the sandbox image. + +## Authentication + +Grok Build resolves credentials the same way the `grok` CLI does: + +- the `XAI_API_KEY` environment variable (headless / sandbox — inject it as a + workspace secret), or +- an existing grok.com browser login on the machine (local dev). + +The two auth modes expose the model under slightly different ids; the adapter +maps the short alias for you (see [Models](#models)). + +## Basic Usage + +```ts +import { chat } from '@tanstack/ai' +import { grokBuildText } from '@tanstack/ai-grok-build' +import { + createSecrets, + defineSandbox, + defineWorkspace, + githubRepo, + withSandbox, +} from '@tanstack/ai-sandbox' +import { dockerSandbox } from '@tanstack/ai-sandbox-docker' +import { messages, threadId } from './chat-context' + +const sandbox = defineSandbox({ + id: 'grok-build-agent', + provider: dockerSandbox({ image: 'node:22' }), + workspace: defineWorkspace({ + source: githubRepo({ repo: 'owner/app' }), + setup: ['corepack enable', 'pnpm install'], + secrets: createSecrets({ XAI_API_KEY: process.env.XAI_API_KEY ?? '' }), + }), +}) + +const stream = chat({ + threadId, + adapter: grokBuildText('grok-build'), + messages, + middleware: [withSandbox(sandbox)], +}) +``` + +## Models + +Grok Build accepts any xAI model id its backend supports; the known ids get +autocomplete (any string is allowed): + +| Model id | Notes | +| --- | --- | +| `grok-build` | The short alias. With a grok.com browser login the CLI lists it under this name. | +| `grok-build-0.1` | The fully-qualified id the CLI lists when authenticated with `XAI_API_KEY`. | +| `composer-2.5` | Also runnable through the Grok Build harness. | + +Pass any of these to `grokBuildText(...)` — the adapter resolves `grok-build` to +the CLI's `grok-build-0.1` automatically, so the same code works under both auth +modes. + +## Configuration + +Adapter config (second argument to `grokBuildText`): + +| Option | Description | +| ---------------- | --------------------------------------------------------------------------- | +| `cwd` | Working directory inside the sandbox. Defaults to `/workspace`. | +| `grokExecutable` | Path/name of the `grok` executable inside the sandbox. Defaults to `grok`. | +| `env` | Extra environment variables for the `grok` process inside the sandbox. | +| `emitDiff` | Emit a `file.changed` CUSTOM event with the working-tree `git diff` after the run. Defaults to `true`. | +| `extraArgs` | Extra raw CLI flags appended verbatim (advanced). | + +Per-call overrides go through `modelOptions`: + +| `modelOptions` | Description | +| --------------- | ------------------------------------------------------------ | +| `sessionId` | Resume an existing Grok Build session (see below). | +| `cwd` | Per-call override of the harness working directory. | +| `maxTurns` | Per-call cap on the number of harness turns. | + +## Stateful Sessions + +Grok Build sessions are stateful — the harness keeps the working context (files +read, commands run, conclusions reached) between turns. The adapter surfaces the +session id of every fresh run as a custom stream event named +`grok-build.session-id`; thread it back via `modelOptions.sessionId` to resume. +When resuming, only the latest user message is sent — the harness already holds +the prior context. + +```ts +import { chat, chatParamsFromRequest, toServerSentEventsResponse } from '@tanstack/ai' +import { grokBuildText } from '@tanstack/ai-grok-build' +import { withSandbox } from '@tanstack/ai-sandbox' +import { sandbox } from './sandbox' + +export async function POST(request: Request) { + const params = await chatParamsFromRequest(request) + const sessionId = + typeof params.forwardedProps.sessionId === 'string' + ? params.forwardedProps.sessionId + : undefined + + const stream = chat({ + adapter: grokBuildText('grok-build'), + messages: params.messages, + middleware: [withSandbox(sandbox)], + modelOptions: { sessionId }, + }) + + return toServerSentEventsResponse(stream) +} +``` + +## Tools + +Two kinds of tools flow through this adapter: + +1. **Built-in harness tools** are executed by Grok Build itself (shell, file + edits, search) and stream back as tool-call events with results already + attached. Your code never executes them. +2. **Your TanStack tools** are bridged *into* the harness over an authenticated + MCP tool-proxy: define them with `toolDefinition().server()` and pass them to + `chat({ tools })`. Tool-call events come back under the names you registered. + Because the harness runs in a sandbox, see [Sandbox tools](/ai/sandbox/tools) + for how the bridge reaches your host across providers (local/Docker vs cloud). + +**Client-side and approval-gated tools are not supported** — the harness runs +tools inside a live process and can't pause across an HTTP round-trip. A tool +without a server `execute()` (or marked `needsApproval`) fails fast; run those +with a regular provider adapter. + +## Limitations + +- **Requires a sandbox.** Always run it under `withSandbox(...)`; see the + [Sandboxes overview](/ai/sandbox/overview). +- **Server-only (Node).** The harness spawns the `grok` CLI in a sandbox. +- **The harness owns the agent loop.** TanStack's agent-loop strategies and + per-iteration middleware don't apply inside a harness turn. +- **No sampling controls.** `temperature`-style options don't exist here. +- **Cold starts.** Each call runs a full harness turn; expect higher first-token + latency than HTTP adapters. diff --git a/mintlify/ai/adapters/grok.md b/mintlify/ai/adapters/grok.md new file mode 100644 index 000000000..afa37e698 --- /dev/null +++ b/mintlify/ai/adapters/grok.md @@ -0,0 +1,422 @@ +--- +title: Grok (xAI) +id: grok-adapter +order: 5 +description: "Use xAI Grok models with TanStack AI — Grok 4.3, Grok Build 0.1, Grok Imagine image generation, and Grok Imagine video generation via @tanstack/ai-grok." +keywords: + - tanstack ai + - grok + - xai + - grok 4.3 + - grok build + - image generation + - video generation + - grok imagine + - adapter +--- + +The Grok text and summarization adapters provide access to xAI's Responses API for `grok-4.3` and `grok-build-0.1`, plus Grok Imagine image generation and Grok Imagine video generation. + +## Installation + +```bash +npm install @tanstack/ai-grok +``` + +## Basic Usage + +```typescript +import { chat } from "@tanstack/ai"; +import { grokText } from "@tanstack/ai-grok"; + +const stream = chat({ + adapter: grokText("grok-build-0.1"), + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Basic Usage - Custom API Key + +```typescript +import { chat } from "@tanstack/ai"; +import { createGrokText } from "@tanstack/ai-grok"; + +const adapter = createGrokText("grok-build-0.1", process.env.XAI_API_KEY!); + +const stream = chat({ + adapter, + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Configuration + +```typescript +import { createGrokText, type GrokTextConfig } from "@tanstack/ai-grok"; + +const config: Omit = { + baseURL: "https://api.x.ai/v1", // Optional, this is the default +}; + +const adapter = createGrokText("grok-build-0.1", process.env.XAI_API_KEY!, config); +``` + +## Example: Chat Completion + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { grokText } from "@tanstack/ai-grok"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: grokText("grok-build-0.1"), + messages, + }); + + return toServerSentEventsResponse(stream); +} +``` + +## Example: With Tools + +```typescript +import { chat, toServerSentEventsResponse, toolDefinition } from "@tanstack/ai"; +import { grokText } from "@tanstack/ai-grok"; +import { z } from "zod"; + +const getWeatherDef = toolDefinition({ + name: "get_weather", + description: "Get the current weather", + inputSchema: z.object({ + location: z.string(), + }), +}); + +const getWeather = getWeatherDef.server(async ({ location }) => { + // Fetch weather data + return { temperature: 72, conditions: "sunny" }; +}); + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: grokText("grok-build-0.1"), + messages, + tools: [getWeather], + }); + + return toServerSentEventsResponse(stream); +} +``` + +## Model Options + +Grok supports xAI Responses API options. Sampling parameters live here too — `temperature`, `top_p`, and `max_output_tokens` — rather than as root-level props on `chat()`: + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { grokText } from "@tanstack/ai-grok"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: grokText("grok-build-0.1"), + messages, + modelOptions: { + temperature: 0.7, + top_p: 0.9, + max_output_tokens: 1024, + store: false, + include: ["reasoning.encrypted_content"], + }, + }); + + return toServerSentEventsResponse(stream); +} +``` + +> If you previously passed `temperature` / `topP` / `maxTokens` at the root of `chat()`, see [Moving Sampling Options into modelOptions](/ai/migration/sampling-options-to-model-options). + +## Summarization + +Summarize long text content: + + + + +```typescript ignore +import { summarize } from "@tanstack/ai"; +import { grokSummarize } from "@tanstack/ai-grok"; + +const result = await summarize({ + adapter: grokSummarize("grok-4.3"), + text: "Your long text to summarize...", + maxLength: 100, + style: "concise", // "concise" | "bullet-points" | "paragraph" +}); + +console.log(result.summary); +``` + +## Image Generation + +Generate images with Grok 2 Image: + +```typescript +import { generateImage } from "@tanstack/ai"; +import { grokImage } from "@tanstack/ai-grok"; + +const result = await generateImage({ + adapter: grokImage("grok-2-image-1212"), + prompt: "A futuristic cityscape at sunset", + numberOfImages: 1, +}); + +console.log(result.images); +``` + +The grok-imagine models (`grok-imagine-image`, `grok-imagine-image-quality`) +are aspect-ratio sized — `size` takes an `aspectRatio_resolution` template +like `"16:9_2k"` (the `_2k` suffix is optional): + +```typescript +import { generateImage } from "@tanstack/ai"; +import { grokImage } from "@tanstack/ai-grok"; + +const result = await generateImage({ + adapter: grokImage("grok-imagine-image"), + prompt: "A futuristic cityscape at sunset", + size: "16:9_2k", +}); +``` + +### Image Editing (image-to-image) + +The grok-imagine models accept image prompt parts for image-conditioned +generation via xAI's `/v1/images/edits` endpoint — up to 3 source images, +addressed by xAI in the order they appear in the prompt. Per xAI's docs +there is no in-prompt referencing syntax; write the prompt naturally and +your text is sent verbatim: + +```typescript +import { generateImage } from "@tanstack/ai"; +import { grokImage } from "@tanstack/ai-grok"; + +const result = await generateImage({ + adapter: grokImage("grok-imagine-image"), + prompt: [ + { + type: "text", + content: "Render the product in the style of the second image", + }, + { + type: "image", + source: { type: "url", value: "https://example.com/product.png" }, + }, + { + type: "image", + source: { type: "url", value: "https://example.com/style.png" }, + }, + ], +}); +``` + +URL sources are fetched by xAI's servers, so they must be publicly +reachable; use a `data` source for private images. `grok-2-image-1212` is +text-to-image only — image prompt parts are a compile-time type error and +throw at runtime. + +## Video Generation (Experimental) + +Generate short video clips (1–15 seconds, with audio) with the Grok Imagine video models via xAI's asynchronous jobs/polling API. + +Available models: + +- `grok-imagine-video` (v1.0) — text-to-video and image-to-video, $0.05 per second of video. +- `grok-imagine-video-1.5` — **image-to-video only**, $0.08 per second of video. A text-only prompt is rejected by the API; the adapter fails fast with a clear error telling you to add a starting-frame image or use `grok-imagine-video`. + +Text-to-video with the base `grok-imagine-video` model: + +```typescript +import { generateVideo, getVideoJobStatus } from "@tanstack/ai"; +import { grokVideo } from "@tanstack/ai-grok"; + +const adapter = grokVideo("grok-imagine-video"); + +// 1. Create the job +const { jobId } = await generateVideo({ + adapter, + prompt: "A red panda balancing on a bamboo stalk in the rain", + size: "16:9_720p", // "aspectRatio" or "aspectRatio_resolution" + duration: 5, // integer seconds, 1–15 +}); + +// 2. Poll until complete, then read the video URL +let status = await getVideoJobStatus({ adapter, jobId }); +while (status.status !== "completed" && status.status !== "failed") { + await new Promise((r) => setTimeout(r, 5000)); + status = await getVideoJobStatus({ adapter, jobId }); +} + +console.log(status.url); // hosted .mp4 URL +``` + +For image-to-video (required for `grok-imagine-video-1.5`, optional for `grok-imagine-video`), include an `image` prompt part as the starting frame and describe the desired motion in the text part. URL sources are fetched by xAI's servers (so they must be publicly reachable); use a `data` source for a base64 starting frame: + +```typescript +import { generateVideo } from "@tanstack/ai"; +import { grokVideo } from "@tanstack/ai-grok"; + +const { jobId } = await generateVideo({ + adapter: grokVideo("grok-imagine-video-1.5"), + prompt: [ + { + type: "text", + content: "Make the waterfall crash down and slowly pan out the camera", + }, + { + type: "image", + source: { type: "url", value: "https://example.com/waterfall-still.png" }, + }, + ], + size: "16:9_720p", + duration: 10, +}); +``` + +Like the Grok Imagine image models, sizing is aspect-ratio based: the `size` option takes an `aspectRatio_resolution` template. Supported aspect ratios are `1:1`, `16:9`, `9:16`, `4:3`, `3:4`, `3:2`, and `2:3`; supported resolutions are `480p`, `720p`, and `1080p` (e.g. `"9:16_1080p"`). The resolution suffix is optional. + +When the job completes, the adapter reports usage on the result: `usage.unitsBilled` carries the billed seconds of video and `usage.cost` the exact cost in USD, both as returned by the xAI API. + +See [Video Generation](/ai/media/video-generation) for the full jobs/polling flow, streaming mode, and the `useGenerateVideo` hook. + +## Text-to-Speech + +Generate speech with Grok TTS: + +```typescript +import { generateSpeech } from "@tanstack/ai"; +import { grokSpeech } from "@tanstack/ai-grok"; + +const result = await generateSpeech({ + adapter: grokSpeech("grok-tts"), + text: "Hello from Grok!", + voice: "default", + format: "mp3", +}); + +console.log(result.audio); // Base64-encoded audio +``` + +## Transcription + +Transcribe audio with Grok STT: + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { grokTranscription } from "@tanstack/ai-grok"; +import { audioFile } from "./audio"; + +const result = await generateTranscription({ + adapter: grokTranscription("grok-stt"), + audio: audioFile, +}); + +console.log(result.text); +``` + +## Realtime Voice + +Grok also exposes a Realtime voice adapter (`grokRealtime`) and a token issuer (`grokRealtimeToken`) for low-latency voice conversations. See [Realtime Voice Chat](/ai/media/realtime-chat) for the end-to-end flow. + +## Environment Variables + +Set your API key in environment variables: + +```bash +XAI_API_KEY=xai-... +``` + +## Implementation Notes + +### Responses API + +The Grok text and summarize adapters use xAI's **Responses API** (`/v1/responses`). Requests default to `store: false` and include encrypted reasoning content with `include: ["reasoning.encrypted_content"]`; both can be overridden through `modelOptions`. + +The shared Responses implementation supports streaming text, reasoning events, structured output via `text.format`, and user-defined function tools. + +## API Reference + +### `grokText(model, config?)` + +Creates a Grok text adapter using environment variables. + +**Parameters:** + +- `model` - The model name (`'grok-4.3'` or `'grok-build-0.1'`) +- `config.baseURL?` - Custom base URL (optional) + +**Returns:** A Grok text adapter instance. + +### `createGrokText(model, apiKey, config?)` + +Creates a Grok text adapter with an explicit API key. + +**Parameters:** + +- `model` - The model name +- `apiKey` - Your xAI API key +- `config.baseURL?` - Custom base URL (optional) + +**Returns:** A Grok text adapter instance. + +### `grokSummarize(model, config?)` + +Creates a Grok summarization adapter using environment variables. + +**Returns:** A Grok summarize adapter instance. + +### `createGrokSummarize(model, apiKey, config?)` + +Creates a Grok summarization adapter with an explicit API key. + +**Returns:** A Grok summarize adapter instance. + +### `grokImage(model, config?)` / `createGrokImage(model, apiKey, config?)` + +Creates a Grok image generation adapter. + +### `grokVideo(model, config?)` / `createGrokVideo(model, apiKey, config?)` + +Creates a Grok video generation adapter (experimental) for the Grok Imagine video models (`'grok-imagine-video'`, `'grok-imagine-video-1.5'`). + +### `grokSpeech(model, config?)` / `createGrokSpeech(model, apiKey, config?)` + +Creates a Grok text-to-speech adapter. + +### `grokTranscription(model, config?)` / `createGrokTranscription(model, apiKey, config?)` + +Creates a Grok speech-to-text adapter. + +### `grokRealtime(...)` / `grokRealtimeToken(...)` + +Realtime voice adapter and token issuer. See [Realtime Voice Chat](/ai/media/realtime-chat) for usage. + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about tools +- [Other Adapters](/ai/adapters/openai) - Explore other providers + +## Provider Tools + +Grok does not currently expose provider-specific tool factories. +Define your own tools with `toolDefinition()` from `@tanstack/ai`. + +See [Tools](/ai/tools/tools) for the general tool-definition flow, or +[Provider Tools](/ai/tools/provider-tools) for other providers' +native-tool offerings. diff --git a/mintlify/ai/adapters/groq.md b/mintlify/ai/adapters/groq.md new file mode 100644 index 000000000..5d89484b8 --- /dev/null +++ b/mintlify/ai/adapters/groq.md @@ -0,0 +1,255 @@ +--- +title: Groq +id: groq-adapter +order: 6 +description: "Use Groq's fast inference API with TanStack AI for low-latency LLM responses and Whisper transcription — Llama and other open-weight models via @tanstack/ai-groq." +keywords: + - tanstack ai + - groq + - fast inference + - llama + - low latency + - adapter + - llm + - whisper + - transcription +--- + +The Groq adapter provides access to Groq's fast inference API, featuring the world's fastest LLM inference and Whisper-based audio transcription. + +## Installation + +```bash +npm install @tanstack/ai-groq +``` + +## Basic Usage + +```typescript +import { chat } from "@tanstack/ai"; +import { groqText } from "@tanstack/ai-groq"; + +const stream = chat({ + adapter: groqText("llama-3.3-70b-versatile"), + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Basic Usage - Custom API Key + +```typescript +import { chat } from "@tanstack/ai"; +import { createGroqText } from "@tanstack/ai-groq"; + +const adapter = createGroqText("llama-3.3-70b-versatile", process.env.GROQ_API_KEY!, { + // ... your config options +}); + +const stream = chat({ + adapter, + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Configuration + +```typescript +import { createGroqText, type GroqTextConfig } from "@tanstack/ai-groq"; + +const config: Omit = { + baseURL: "https://api.groq.com/openai/v1", // Optional, for custom endpoints +}; + +const adapter = createGroqText("llama-3.3-70b-versatile", process.env.GROQ_API_KEY!, config); +``` + +## Example: Chat Completion + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { groqText } from "@tanstack/ai-groq"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: groqText("llama-3.3-70b-versatile"), + messages, + }); + + return toServerSentEventsResponse(stream); +} +``` + +## Example: With Tools + +```typescript +import { chat, toolDefinition, type ModelMessage } from "@tanstack/ai"; +import { groqText } from "@tanstack/ai-groq"; +import { z } from "zod"; + +const searchDatabaseDef = toolDefinition({ + name: "search_database", + description: "Search the database", + inputSchema: z.object({ + query: z.string(), + }), +}); + +const searchDatabase = searchDatabaseDef.server(async ({ query }) => { + // Search database + return { results: [] }; +}); + +const messages: Array = [{ role: "user", content: "Search for something" }]; + +const stream = chat({ + adapter: groqText("llama-3.3-70b-versatile"), + messages, + tools: [searchDatabase], +}); +``` + +## Transcription + +Groq exposes Whisper-based speech-to-text via `groqTranscription()` and the `generateTranscription()` activity. The `audio` input accepts a `File`, `Blob`, `ArrayBuffer`, base64 string, data URL, or an `https://` URL (forwarded directly to Groq without re-uploading). + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { groqTranscription } from "@tanstack/ai-groq"; + +const result = await generateTranscription({ + adapter: groqTranscription("whisper-large-v3-turbo"), + audio: "https://example.com/recording.mp3", + language: "en", +}); + +console.log(result.text); + +// verbose_json (the default) populates language, duration, and timestamped segments +for (const segment of result.segments ?? []) { + console.log(`[${segment.start}s → ${segment.end}s] ${segment.text}`); +} +``` + +Supported models: `whisper-large-v3-turbo`, `whisper-large-v3`. Supported `responseFormat` values: `json`, `text`, `verbose_json` (default). `srt` and `vtt` are not supported by Groq. + +See [Transcription](/ai/media/transcription) for the full API. + +## Model Options + +Groq supports various provider-specific options. Sampling parameters live here too — `temperature`, `top_p`, and `max_completion_tokens` (Groq's token-limit key) — rather than as root-level props on `chat()`: + +```typescript +import { chat } from "@tanstack/ai"; +import { groqText } from "@tanstack/ai-groq"; + +const stream = chat({ + adapter: groqText("llama-3.3-70b-versatile"), + messages: [{ role: "user", content: "Hello!" }], + modelOptions: { + temperature: 0.7, + max_completion_tokens: 1024, + top_p: 0.9, + }, +}); +``` + +> If you previously passed `temperature` / `topP` / `maxTokens` at the root of `chat()`, see [Moving Sampling Options into modelOptions](/ai/migration/sampling-options-to-model-options). + +### Reasoning + +Enable reasoning for models that support it (e.g., `openai/gpt-oss-120b`, `qwen/qwen3-32b`). This allows the model to show its reasoning process, which is streamed as `thinking` chunks: + +```typescript ignore +modelOptions: { + reasoning_effort: "medium", // "none" | "default" | "low" | "medium" | "high" +} +``` + +## Supported Models + +Groq offers a diverse selection of models from multiple providers: + +### Meta Llama + +- `llama-3.3-70b-versatile` - Fast, capable model with 128K context +- `llama-3.1-8b-instant` - Fast, cost-effective model +- `meta-llama/llama-4-maverick-17b-128e-instruct` - Latest Llama 4 with vision support +- `meta-llama/llama-4-scout-17b-16e-instruct` - Efficient Llama 4 model + +### Security Models + +- `meta-llama/llama-guard-4-12b` - Content moderation +- `meta-llama/llama-prompt-guard-2-86m` - Prompt injection detection +- `meta-llama/llama-prompt-guard-2-22m` - Lightweight prompt guard + +### OpenAI GPT-OSS Models + +- `openai/gpt-oss-120b` - Large OSS model with reasoning support +- `openai/gpt-oss-20b` - Efficient OSS model +- `openai/gpt-oss-safeguard-20b` - Safety-tuned OSS model + +### Other Providers + +- `moonshotai/kimi-k2-instruct-0905` - Kimi K2 with 256K context +- `qwen/qwen3-32b` - Qwen 3 with reasoning support + +## Environment Variables + +Set your API key in environment variables: + +```bash +GROQ_API_KEY=gsk_... +``` + +## API Reference + +### `groqText(model, config?)` + +Creates a Groq chat adapter using environment variables. + +**Parameters:** + +- `model` - The model name (e.g., `llama-3.3-70b-versatile`) +- `config` (optional) - Optional configuration object. Supports the same options as `createGroqText` except `apiKey`, which is auto-detected from `GROQ_API_KEY` environment variable. Common options: + - `baseURL` - Custom base URL for API requests (optional) + +**Returns:** A Groq chat adapter instance. + +### `createGroqText(model, apiKey, config?)` + +Creates a Groq chat adapter with an explicit API key. + +**Parameters:** + +- `model` - The model name (e.g., `llama-3.3-70b-versatile`) +- `apiKey` - Your Groq API key +- `config` (optional) - Optional configuration object: + - `baseURL` - Custom base URL for API requests (optional) + +**Returns:** A Groq chat adapter instance. + +### `groqTranscription(model, config?)` / `createGroqTranscription(model, apiKey, config?)` + +Creates a Groq transcription (speech-to-text) adapter. The short form reads `GROQ_API_KEY` from the environment; the `create*` form takes an explicit API key. Supported models: `whisper-large-v3-turbo`, `whisper-large-v3`. + +## Limitations + +- **Text-to-Speech**: Groq does not currently expose a TTS adapter. Use OpenAI, Gemini, ElevenLabs, or fal for speech generation. +- **Image Generation**: Groq does not support image generation. Use OpenAI, Gemini, or fal for image generation. + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about tools +- [Other Adapters](/ai/adapters/openai) - Explore other providers + +## Provider Tools + +Groq does not currently expose provider-specific tool factories. +Define your own tools with `toolDefinition()` from `@tanstack/ai`. + +See [Tools](/ai/tools/tools) for the general tool-definition flow, or +[Provider Tools](/ai/tools/provider-tools) for other providers' +native-tool offerings. diff --git a/mintlify/ai/adapters/mistral.md b/mintlify/ai/adapters/mistral.md new file mode 100644 index 000000000..94839a631 --- /dev/null +++ b/mintlify/ai/adapters/mistral.md @@ -0,0 +1,352 @@ +--- +title: Mistral +id: mistral-adapter +order: 7 +description: "Use Mistral models with TanStack AI — Mistral Large, Mistral Medium, Pixtral vision models, Magistral reasoning models, and Codestral via @tanstack/ai-mistral." +keywords: + - tanstack ai + - mistral + - mistral large + - pixtral + - magistral + - codestral + - adapter + - llm +--- + +The Mistral adapter provides access to Mistral's chat models, including Mistral Large, the multimodal Pixtral family, the Magistral reasoning models, and the Codestral code-specialized model. + +## Installation + +```bash +npm install @tanstack/ai-mistral +``` + +## Basic Usage + +```typescript +import { chat } from "@tanstack/ai"; +import { mistralText } from "@tanstack/ai-mistral"; + +const stream = chat({ + adapter: mistralText("mistral-large-latest"), + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Basic Usage - Custom API Key + +```typescript +import { chat } from "@tanstack/ai"; +import { createMistralText } from "@tanstack/ai-mistral"; + +const adapter = createMistralText( + "mistral-large-latest", + process.env.MISTRAL_API_KEY!, +); + +const stream = chat({ + adapter, + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Configuration + +```typescript +import { + createMistralText, + type MistralTextConfig, +} from "@tanstack/ai-mistral"; + +const config: Omit = { + serverURL: "https://api.mistral.ai", // Optional, this is the default + defaultHeaders: { + "X-Custom-Header": "value", + }, +}; + +const adapter = createMistralText( + "mistral-large-latest", + process.env.MISTRAL_API_KEY!, + config, +); +``` + +## Example: Chat Completion + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { mistralText } from "@tanstack/ai-mistral"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: mistralText("mistral-large-latest"), + messages, + }); + + return toServerSentEventsResponse(stream); +} +``` + +## Example: With Tools + +```typescript +import { chat, toolDefinition } from "@tanstack/ai"; +import { mistralText } from "@tanstack/ai-mistral"; +import { z } from "zod"; + +const getWeatherDef = toolDefinition({ + name: "get_weather", + description: "Get the current weather for a location", + inputSchema: z.object({ + location: z.string(), + }), +}); + +const getWeather = getWeatherDef.server(async ({ location }) => { + return { temperature: 72, conditions: "sunny" }; +}); + +const stream = chat({ + adapter: mistralText("mistral-large-latest"), + messages: [{ role: "user", content: "What's the weather in Paris?" }], + tools: [getWeather], +}); +``` + +## Example: Multimodal (Vision) + +Use a vision-capable model — `pixtral-large-latest`, `pixtral-12b-2409`, `mistral-medium-latest`, or `mistral-small-latest` — to send images alongside text: + +```typescript +import { chat } from "@tanstack/ai"; +import { mistralText } from "@tanstack/ai-mistral"; + +const stream = chat({ + adapter: mistralText("pixtral-large-latest"), + messages: [ + { + role: "user", + content: [ + { type: "text", content: "What's in this image?" }, + { + type: "image", + source: { + type: "url", + value: "https://example.com/photo.jpg", + }, + }, + ], + }, + ], +}); +``` + +For data-URL or base64 images, set `source.type` to `"data"` and provide `mimeType`: + +```typescript +import { chat } from "@tanstack/ai"; +import { mistralText } from "@tanstack/ai-mistral"; + +const base64String = "..."; // your base64-encoded image bytes + +const stream = chat({ + adapter: mistralText("pixtral-large-latest"), + messages: [ + { + role: "user", + content: [ + { type: "text", content: "What's in this image?" }, + { + type: "image", + source: { + type: "data", + mimeType: "image/png", + value: base64String, + }, + }, + ], + }, + ], +}); +``` + +See [Multimodal Content](/ai/advanced/multimodal-content) for the full content-part shape. + +## Example: Reasoning (Magistral) + +Magistral models (`magistral-medium-latest`, `magistral-small-latest`) stream their reasoning as separate events before the final answer. The adapter emits AG-UI `REASONING_*` chunks for the thinking content and `TEXT_MESSAGE_*` chunks for the answer: + +```typescript ignore +// ignore: narrowing the raw AG-UI stream by `chunk.type` relies on @ag-ui/core's +// discriminated-union `type` field, which kiira can't resolve in a source-only +// check. The runtime behaviour is exactly as shown. +import { chat } from "@tanstack/ai"; +import { mistralText } from "@tanstack/ai-mistral"; + +const stream = chat({ + adapter: mistralText("magistral-medium-latest"), + messages: [{ role: "user", content: "Why is the sky blue?" }], +}); + +for await (const chunk of stream) { + if (chunk.type === "REASONING_MESSAGE_CONTENT") { + process.stdout.write(`[thinking] ${chunk.delta}`); + } else if (chunk.type === "TEXT_MESSAGE_CONTENT") { + process.stdout.write(chunk.delta); + } +} +``` + +Reasoning events are always closed before any text or tool output begins, so consumers see a complete `REASONING_START → REASONING_MESSAGE_START → REASONING_MESSAGE_CONTENT* → REASONING_MESSAGE_END → REASONING_END` sequence first. + +See [Thinking & Reasoning](/ai/chat/thinking-content) for the cross-provider event spec. + +## Example: Structured Output + +Generate JSON that conforms to a Zod schema using Mistral's `json_schema` response format: + +```typescript +import { chat } from "@tanstack/ai"; +import { mistralText } from "@tanstack/ai-mistral"; +import { z } from "zod"; + +const recipeSchema = z.object({ + name: z.string(), + ingredients: z.array(z.string()), + steps: z.array(z.string()), +}); + +const recipe = await chat({ + adapter: mistralText("mistral-large-latest"), + messages: [ + { role: "user", content: "Give me a chocolate chip cookie recipe." }, + ], + outputSchema: recipeSchema, +}); + +console.log(recipe.name); // typed as z.infer +``` + +See [Structured Outputs](/ai/chat/structured-outputs) for the full guide. + +## Model Options + +Mistral exposes provider-specific options via `modelOptions`: + +```typescript +import { chat } from "@tanstack/ai"; +import { mistralText } from "@tanstack/ai-mistral"; + +const stream = chat({ + adapter: mistralText("mistral-large-latest"), + messages: [{ role: "user", content: "Hello!" }], + modelOptions: { + temperature: 0.7, + top_p: 0.9, + max_tokens: 1024, + random_seed: 42, + stop: ["END"], + safe_prompt: true, + frequency_penalty: 0.5, + presence_penalty: 0.5, + parallel_tool_calls: true, + tool_choice: "auto", + }, +}); +``` + +> All sampling parameters — including `temperature`, `top_p`, and `max_tokens` — +> go inside `modelOptions` using Mistral's native (snake_case) names. + +## Environment Variables + +Set your API key in environment variables: + +```bash +MISTRAL_API_KEY=... +``` + +Get a key from the [Mistral Console](https://console.mistral.ai/). + +## Supported Models + +### Chat + +- `mistral-large-latest` — Flagship general-purpose model (128k context) +- `mistral-medium-latest` — Multimodal mid-tier model with vision +- `mistral-small-latest` — Fast, affordable multimodal model with vision +- `ministral-8b-latest` — 8B edge model +- `ministral-3b-latest` — 3B edge model +- `open-mistral-nemo` — Open 12B model + +### Code + +- `codestral-latest` — Code-specialized model (256k context) + +### Vision + +- `pixtral-large-latest` — Large vision model +- `pixtral-12b-2409` — 12B vision model + +### Reasoning + +Reasoning content is streamed as `REASONING_*` events before the final answer. + +- `magistral-medium-latest` — Mid-tier reasoning model +- `magistral-small-latest` — Small reasoning model + +See [Mistral's model comparison](https://docs.mistral.ai/getting-started/models/compare) for full details. + +## API Reference + +### `mistralText(model, config?)` + +Creates a Mistral text adapter using the `MISTRAL_API_KEY` environment variable. + +**Parameters:** + +- `model` — The model name (e.g., `'mistral-large-latest'`) +- `config.serverURL?` — Custom base URL (optional) +- `config.defaultHeaders?` — Headers to attach to every request (optional) + +**Returns:** A Mistral text adapter instance. + +### `createMistralText(model, apiKey, config?)` + +Creates a Mistral text adapter with an explicit API key. + +**Parameters:** + +- `model` — The model name +- `apiKey` — Your Mistral API key +- `config.serverURL?` — Custom base URL (optional) +- `config.defaultHeaders?` — Headers to attach to every request (optional) + +**Returns:** A Mistral text adapter instance. + +## Limitations + +- **Embeddings**: Use the [Mistral SDK](https://github.com/mistralai/client-ts) directly for `mistral-embed`. +- **Image / Audio / Video Generation**: Mistral does not provide these endpoints. Use OpenAI, Gemini, or fal.ai. +- **Text-to-Speech / Transcription**: Not supported. Use OpenAI or ElevenLabs. + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) — Learn the basics +- [Tools Guide](/ai/tools/tools) — Define and call tools +- [Structured Outputs](/ai/chat/structured-outputs) — Generate typed JSON +- [Multimodal Content](/ai/advanced/multimodal-content) — Send images and other modalities +- [Other Adapters](/ai/adapters/openai) — Explore other providers + +## Provider Tools + +Mistral does not currently expose provider-specific tool factories. +Define your own tools with `toolDefinition()` from `@tanstack/ai`. + +See [Tools](/ai/tools/tools) for the general tool-definition flow, or +[Provider Tools](/ai/tools/provider-tools) for other providers' +native-tool offerings. diff --git a/mintlify/ai/adapters/ollama.md b/mintlify/ai/adapters/ollama.md new file mode 100644 index 000000000..4ac91b4bc --- /dev/null +++ b/mintlify/ai/adapters/ollama.md @@ -0,0 +1,312 @@ +--- +title: Ollama +id: ollama-adapter +order: 4 +description: "Run local LLMs with Ollama in TanStack AI for private, no-cost AI on your own hardware via the @tanstack/ai-ollama adapter." +keywords: + - tanstack ai + - ollama + - local llm + - self-hosted + - privacy + - llama + - offline ai + - adapter +--- + +The Ollama adapter provides access to local models running via Ollama, allowing you to run AI models on your own infrastructure with full privacy and no API costs. + +## Installation + +```bash +npm install @tanstack/ai-ollama +``` + +## Basic Usage + +```typescript +import { chat } from "@tanstack/ai"; +import { ollamaText } from "@tanstack/ai-ollama"; + +const stream = chat({ + adapter: ollamaText("llama3"), + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Basic Usage - Custom Host + +```typescript +import { chat } from "@tanstack/ai"; +import { createOllamaChat } from "@tanstack/ai-ollama"; + +const adapter = createOllamaChat("llama3", "http://your-server:11434"); + +const stream = chat({ + adapter, + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Configuration + +```typescript +import { createOllamaChat } from "@tanstack/ai-ollama"; + +// Custom host (URL string) +const adapter = createOllamaChat("llama3", "http://your-server:11434"); + +// Custom client config (e.g., custom headers, fetch) +const adapter2 = createOllamaChat("llama3", { + host: "http://your-server:11434", + headers: { Authorization: "Bearer ..." }, +}); +``` + +## Available Models + +To see available models on your Ollama instance: + +```bash +ollama list +``` + +### Popular Models + +- `llama3` / `llama3.1` / `llama3.2` - Meta's Llama models +- `mistral` / `mistral:7b` - Mistral AI models +- `mixtral` - Mixtral MoE model +- `codellama` - Code-focused Llama +- `phi3` - Microsoft's Phi models +- `gemma` / `gemma2` - Google's Gemma models +- `qwen2` / `qwen2.5` - Alibaba's Qwen models +- `deepseek-coder` - DeepSeek coding model + +## Example: Chat Completion + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { ollamaText } from "@tanstack/ai-ollama"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: ollamaText("llama3"), + messages, + }); + + return toServerSentEventsResponse(stream); +} +``` + +## Example: With Tools + +```typescript +import { chat, toServerSentEventsResponse, toolDefinition } from "@tanstack/ai"; +import { ollamaText } from "@tanstack/ai-ollama"; +import { z } from "zod"; + +const getLocalDataDef = toolDefinition({ + name: "get_local_data", + description: "Get data from local storage", + inputSchema: z.object({ + key: z.string(), + }), +}); + +const getLocalData = getLocalDataDef.server(async ({ key }) => { + // Access local data + return { data: "..." }; +}); + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: ollamaText("llama3"), + messages, + tools: [getLocalData], + }); + + return toServerSentEventsResponse(stream); +} +``` + +**Note:** Tool support varies by model. Models like `llama3`, `mistral`, and `qwen2` generally have good tool calling support. + +## Model Options + +Ollama supports various provider-specific options. Unlike the other providers, Ollama nests its sampling and runner parameters inside an `options` object **within** `modelOptions` — `temperature`, `top_p`, and `num_predict` (the token-limit key) all live under `modelOptions.options`: + +```typescript +import { chat } from "@tanstack/ai"; +import { ollamaText } from "@tanstack/ai-ollama"; + +const stream = chat({ + adapter: ollamaText("llama3:latest"), + messages: [{ role: "user", content: "Hello!" }], + modelOptions: { + options: { + temperature: 0.7, + top_p: 0.9, + top_k: 40, + num_predict: 1000, // Max tokens to generate + repeat_penalty: 1.1, + num_ctx: 4096, // Context window size + num_gpu: -1, // GPU layers (-1 = auto) + }, + }, +}); +``` + +> If you previously passed `temperature` / `topP` / `maxTokens` at the root of `chat()`, note that for Ollama they map to `modelOptions.options.temperature`, `modelOptions.options.top_p`, and `modelOptions.options.num_predict`. See [Moving Sampling Options into modelOptions](/ai/migration/sampling-options-to-model-options). + +### Advanced Options + +All sampling and runner parameters are nested under `modelOptions.options`: + +```typescript ignore +modelOptions: { + options: { + // Sampling + temperature: 0.7, + top_p: 0.9, + top_k: 40, + min_p: 0.05, + typical_p: 1.0, + + // Generation + num_predict: 1000, + repeat_penalty: 1.1, + repeat_last_n: 64, + penalize_newline: false, + + // Performance + num_ctx: 4096, + num_batch: 512, + num_gpu: -1, + num_thread: 0, // 0 = auto + + // Memory + use_mmap: true, + use_mlock: false, + + // Mirostat sampling + mirostat: 0, // 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0 + mirostat_tau: 5.0, + mirostat_eta: 0.1, + }, +} +``` + +## Summarization + +Summarize long text content locally: + +```typescript ignore +import { summarize } from "@tanstack/ai"; +import { ollamaSummarize } from "@tanstack/ai-ollama"; + +const result = await summarize({ + adapter: ollamaSummarize("llama3"), + text: "Your long text to summarize...", + maxLength: 100, + style: "concise", // "concise" | "bullet-points" | "paragraph" +}); + +console.log(result.summary); +``` + +## Setting Up Ollama + +### 1. Install Ollama + +```bash +# macOS +brew install ollama + +# Linux +curl -fsSL https://ollama.com/install.sh | sh + +# Windows +# Download from https://ollama.com +``` + +### 2. Pull a Model + +```bash +ollama pull llama3 +``` + +### 3. Start Ollama Server + +```bash +ollama serve +``` + +The server runs on `http://localhost:11434` by default. + +## Running on a Remote Server + +```typescript +import { createOllamaChat } from "@tanstack/ai-ollama"; + +const adapter = createOllamaChat("llama3", "http://your-server:11434"); +``` + +To expose Ollama on a network interface: + +```bash +OLLAMA_HOST=0.0.0.0:11434 ollama serve +``` + +## Environment Variables + +Optionally set the host in environment variables: + +```bash +OLLAMA_HOST=http://localhost:11434 +``` + +## API Reference + +### `ollamaText(model)` + +Creates an Ollama text/chat adapter using `OLLAMA_HOST` from the environment (defaults to `http://localhost:11434`). + +**Parameters:** + +- `model` - Model name (e.g. `"llama3"`, `"mistral:7b"`) + +### `createOllamaChat(model, hostOrConfig?)` + +Creates an Ollama text/chat adapter with an explicit host or client config. + +**Parameters:** + +- `model` - Model name +- `hostOrConfig?` - Either an `OLLAMA_HOST`-style URL string, or an `OllamaClientConfig` object (e.g. `{ host, headers, fetch }`). + +### `ollamaSummarize(model)` / `createOllamaSummarize(model, hostOrConfig?)` + +Creates an Ollama summarization adapter — same signature shape as the chat adapter. + +## Benefits of Ollama + +- ✅ **Privacy** - Data stays on your infrastructure +- ✅ **Cost** - No API costs after hardware +- ✅ **Customization** - Use any compatible model +- ✅ **Offline** - Works without internet +- ✅ **Speed** - No network latency for local deployment + +## Limitations + +- **Image Generation**: Ollama does not support image generation. Use OpenAI or Gemini for image generation. +- **Performance**: Depends on your hardware (GPU recommended for larger models) + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about tools +- [Other Adapters](/ai/adapters/openai) - Explore other providers diff --git a/mintlify/ai/adapters/openai-compatible.md b/mintlify/ai/adapters/openai-compatible.md new file mode 100644 index 000000000..2513c44aa --- /dev/null +++ b/mintlify/ai/adapters/openai-compatible.md @@ -0,0 +1,243 @@ +--- +title: OpenAI-Compatible Adapter +id: openai-compatible-adapter +description: "Use any OpenAI-compatible provider (DeepSeek, Moonshot/Kimi, Together, Fireworks, Cerebras, Qwen, Perplexity, local servers, and more) in TanStack AI with one generic adapter." +keywords: + - tanstack ai + - openai compatible + - deepseek + - moonshot + - kimi + - together + - fireworks + - cerebras + - qwen + - perplexity + - lm studio + - vllm + - adapter +--- + +Many providers expose the OpenAI **Chat Completions** API (`/chat/completions`) — DeepSeek, Moonshot/Kimi, Together, Fireworks, Cerebras, Alibaba Qwen, Perplexity, NVIDIA NIM, and local servers like LM Studio, Ollama, and vLLM. Instead of a dedicated package per provider, TanStack AI ships one generic adapter: point it at any compatible `baseURL`, give it your models, and you get the same type-safe `chat()` experience as the first-class adapters. + +Use this when your provider speaks the OpenAI Chat Completions wire format but doesn't have its own `@tanstack/ai-*` package. If a dedicated adapter exists (OpenAI, Grok, Groq, OpenRouter), prefer it — those carry curated per-model metadata. + +## Installation + +The adapter ships inside `@tanstack/ai-openai` under the `/compatible` subpath — no extra install: + +```bash +npm install @tanstack/ai-openai +``` + +## Basic Usage + +Configure the provider once with `openaiCompatible({ baseURL, apiKey, models })`, then select a model per call. The returned model name is a type-safe union of the models you declared: + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiCompatible } from "@tanstack/ai-openai/compatible"; + +const deepseek = openaiCompatible({ + name: "deepseek", // optional label shown in devtools/errors (default: "openai-compatible") + baseURL: "https://api.deepseek.com/v1", + apiKey: process.env.DEEPSEEK_API_KEY!, + models: ["deepseek-chat", "deepseek-reasoner"], +}); + +const stream = chat({ + adapter: deepseek("deepseek-chat"), + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +`deepseek("deepseek-reasoner")` is valid; `deepseek("gpt-4o")` is a type error — only declared models are accepted. + +## One-Shot Usage + +For a single model, skip the provider-factory and build the adapter inline with `openaiCompatibleText`: + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiCompatibleText } from "@tanstack/ai-openai/compatible"; + +const stream = chat({ + adapter: openaiCompatibleText("deepseek-chat", { + baseURL: "https://api.deepseek.com/v1", + apiKey: process.env.DEEPSEEK_API_KEY!, + }), + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Declaring Models + +The `models` array accepts two forms, which you can mix: + +- **A bare string** — gets optimistic defaults: `text` + `image` input, with `streaming`, `function_calling`, and `structured_outputs` support. Good for mainstream chat models. +- **A `createModel(name, capabilities)` definition** — declares precise per-model capabilities so the types match reality (e.g. a reasoning model with no image input). + +```typescript +import { openaiCompatible } from "@tanstack/ai-openai/compatible"; +import { createModel } from "@tanstack/ai"; + +const provider = openaiCompatible({ + baseURL: "https://api.deepseek.com/v1", + apiKey: process.env.DEEPSEEK_API_KEY!, + models: [ + "deepseek-chat", // string → optimistic defaults + createModel("deepseek-reasoner", { + input: ["text"], // text only + features: ["reasoning", "structured_outputs"], + }), + ], +}); +``` + +> Capabilities are enforced at the type level. If a provider rejects a feature at runtime (e.g. tools on a model that doesn't support them), declare that model with `createModel` and omit the unsupported feature so the types stop you from calling it. + +## Configuration + +`openaiCompatible` accepts every OpenAI SDK `ClientOptions` field besides `apiKey`/`baseURL` (which are required and promoted to the top level). The most useful are `defaultHeaders` and `defaultQuery`, for providers that need extra auth or routing parameters: + +```typescript +import { openaiCompatible } from "@tanstack/ai-openai/compatible"; + +const provider = openaiCompatible({ + baseURL: "https://api.example.com/v1", + apiKey: process.env.EXAMPLE_API_KEY!, + models: ["some-model"], + defaultHeaders: { "X-Custom-Header": "value" }, + defaultQuery: { "api-version": "2026-01-01" }, +}); +``` + +## Chat Completions vs Responses + +By default the adapter targets the **Chat Completions** API (`/chat/completions`) — the surface virtually every compatible provider implements. For the rare provider that also implements OpenAI's **Responses** API (e.g. Azure OpenAI), opt in with `api: "responses"`: + +```typescript +import { openaiCompatible } from "@tanstack/ai-openai/compatible"; + +const provider = openaiCompatible({ + baseURL: "https://my-resource.openai.azure.com/openai/v1", + apiKey: process.env.AZURE_OPENAI_API_KEY!, + models: ["gpt-4o"], + api: "responses", // default is "chat-completions" +}); +``` + +## Supported Providers + +Any provider implementing the OpenAI Chat Completions API works. Common ones are below — **verify the `baseURL` and model ids against each provider's current docs**, since they change over time. Set the API key via the provider's own environment variable and pass it as `apiKey`. + +| Provider | `baseURL` | Example model | +| --- | --- | --- | +| DeepSeek | `https://api.deepseek.com/v1` | `deepseek-chat`, `deepseek-reasoner` | +| Moonshot / Kimi | `https://api.moonshot.ai/v1` | `kimi-k2-0711-preview` | +| Alibaba Qwen (DashScope, intl) | `https://dashscope-intl.aliyuncs.com/compatible-mode/v1` | `qwen-max`, `qwen-plus` | +| Alibaba Qwen (DashScope, China) | `https://dashscope.aliyuncs.com/compatible-mode/v1` | `qwen-max` | +| Together AI | `https://api.together.xyz/v1` | `meta-llama/Llama-3.3-70B-Instruct-Turbo` | +| Fireworks AI | `https://api.fireworks.ai/inference/v1` | `accounts/fireworks/models/llama-v3p3-70b-instruct` | +| Cerebras | `https://api.cerebras.ai/v1` | `llama-3.3-70b` | +| DeepInfra | `https://api.deepinfra.com/v1/openai` | `meta-llama/Llama-3.3-70B-Instruct` | +| Perplexity | `https://api.perplexity.ai` | `sonar`, `sonar-pro` | +| Requesty | `https://router.requesty.ai/v1` | `openai/gpt-4o-mini` | +| Mistral | `https://api.mistral.ai/v1` | `mistral-large-latest` | +| Nebius | `https://api.studio.nebius.ai/v1` | `meta-llama/Llama-3.3-70B-Instruct` | +| Z.AI (GLM) | `https://api.z.ai/api/paas/v4` | `glm-4.6` | +| Baseten | `https://inference.baseten.co/v1` | model-dependent | +| Hugging Face (router) | `https://router.huggingface.co/v1` | `meta-llama/Llama-3.3-70B-Instruct` | +| NVIDIA NIM | `https://integrate.api.nvidia.com/v1` | `meta/llama-3.3-70b-instruct` | + +## Local & Self-Hosted Servers + +Point the adapter at any local OpenAI-compatible server. The API key is usually a placeholder: + +```typescript +import { openaiCompatible } from "@tanstack/ai-openai/compatible"; + +// LM Studio +const lmstudio = openaiCompatible({ + name: "lmstudio", + baseURL: "http://localhost:1234/v1", + apiKey: "lm-studio", + models: ["local-model"], +}); + +// vLLM +const vllm = openaiCompatible({ + name: "vllm", + baseURL: "http://localhost:8000/v1", + apiKey: "not-needed", + models: ["meta-llama/Llama-3.3-70B-Instruct"], +}); + +// Ollama's OpenAI-compatible endpoint +const ollama = openaiCompatible({ + name: "ollama", + baseURL: "http://localhost:11434/v1", + apiKey: "ollama", + models: ["llama3.3"], +}); +``` + +> Ollama also has a dedicated adapter, [`@tanstack/ai-ollama`](/ai/adapters/ollama), which understands its native API. Use `openaiCompatible` only if you specifically want Ollama's OpenAI-compatible surface. + +## Azure OpenAI + +Azure uses a resource-scoped URL and a separate API-version. Use the `/openai/v1` endpoint with `defaultQuery` for the version and `defaultHeaders` for the `api-key` header: + +```typescript +import { openaiCompatible } from "@tanstack/ai-openai/compatible"; + +const azure = openaiCompatible({ + name: "azure", + baseURL: "https://YOUR_RESOURCE.openai.azure.com/openai/v1", + apiKey: process.env.AZURE_OPENAI_API_KEY!, // also sent as Bearer; Azure accepts the api-key header below + models: ["gpt-4o"], // your Azure deployment name + defaultQuery: { "api-version": "2026-01-01-preview" }, + defaultHeaders: { "api-key": process.env.AZURE_OPENAI_API_KEY! }, +}); +``` + +> Confirm the current `api-version` and endpoint shape in Azure's documentation — Azure's API surface evolves independently of OpenAI's. + +## Example: With Tools + +Tools work exactly as they do with any other adapter, for models that support function calling: + +```typescript +import { chat, toolDefinition } from "@tanstack/ai"; +import { openaiCompatible } from "@tanstack/ai-openai/compatible"; +import { z } from "zod"; + +const getWeatherDef = toolDefinition({ + name: "get_weather", + description: "Get the current weather", + inputSchema: z.object({ location: z.string() }), +}); + +const getWeather = getWeatherDef.server(async ({ location }) => { + return { temperature: 72, conditions: "sunny" }; +}); + +const deepseek = openaiCompatible({ + baseURL: "https://api.deepseek.com/v1", + apiKey: process.env.DEEPSEEK_API_KEY!, + models: ["deepseek-chat"], +}); + +const stream = chat({ + adapter: deepseek("deepseek-chat"), + messages: [{ role: "user", content: "What's the weather in Tokyo?" }], + tools: [getWeather], +}); +``` + +## Next Steps + +- [OpenAI Adapter](/ai/adapters/openai) - The first-class OpenAI adapter +- [OpenRouter Adapter](/ai/adapters/openrouter) - Access 300+ models through one gateway +- [Tools Guide](/ai/tools/tools) - Learn about tools +- [Extending Adapters](/ai/advanced/extend-adapter) - Add custom models to any adapter diff --git a/mintlify/ai/adapters/openai.md b/mintlify/ai/adapters/openai.md new file mode 100644 index 000000000..02b638455 --- /dev/null +++ b/mintlify/ai/adapters/openai.md @@ -0,0 +1,717 @@ +--- +title: OpenAI +id: openai-adapter +order: 1 +description: "Use OpenAI models with TanStack AI — GPT-4o, GPT-5, DALL-E image generation, TTS, and Whisper transcription via @tanstack/ai-openai." +keywords: + - tanstack ai + - openai + - gpt-4o + - gpt-5 + - dall-e + - whisper + - openai tts + - adapter + - chatgpt +--- + +The OpenAI adapter provides access to OpenAI's models, including GPT-4o, GPT-5, image generation (DALL-E), text-to-speech (TTS), and audio transcription (Whisper). + +> Using a third-party provider that speaks the OpenAI API (DeepSeek, Moonshot/Kimi, Together, Fireworks, a local LM Studio/vLLM server, …)? See the [OpenAI-Compatible Adapter](/ai/adapters/openai-compatible) for a generic `openaiCompatible({ baseURL, apiKey, models })` factory. + +## Installation + +```bash +npm install @tanstack/ai-openai +``` + +## Basic Usage + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Chat Completions API + +`@tanstack/ai-openai` ships two text adapters that hit different OpenAI endpoints. `openaiText` (default) calls the Responses API (`/v1/responses`). `openaiChatCompletions` calls the older Chat Completions API (`/v1/chat/completions`). + +Pick whichever fits your wire format and feature needs: + +| | `openaiText` (Responses) | `openaiChatCompletions` (Chat Completions) | +|---|---|---| +| Endpoint | `/v1/responses` | `/v1/chat/completions` | +| Reasoning summaries | Yes — set `modelOptions.reasoning.summary: 'auto'` to surface reasoning text via `REASONING_*` events | No — reasoning tokens are still consumed but cannot be exposed | +| Wire-format compatibility | OpenAI-only | Matches the older de-facto industry shape (Grok, Groq, OpenRouter, many local model servers) | +| Structured output streaming | `text.format: { type: 'json_schema', strict: true }` + `stream: true` | `response_format: { type: 'json_schema', strict: true }` + `stream: true` | + +Use `openaiText` when you want reasoning-summary streaming or OpenAI-specific Responses features. Use `openaiChatCompletions` when you're migrating off a Chat-Completions-style provider, share request-building code with other Chat-Completions adapters in your stack, or want the more battle-tested wire format. + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiChatCompletions } from "@tanstack/ai-openai"; + +const stream = chat({ + adapter: openaiChatCompletions("gpt-5.2"), + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +With an explicit API key: + +```typescript +import { chat } from "@tanstack/ai"; +import { createOpenaiChatCompletions } from "@tanstack/ai-openai"; + +const adapter = createOpenaiChatCompletions("gpt-5.2", process.env.OPENAI_API_KEY!, { + // organization, baseURL, headers — all optional +}); + +const stream = chat({ + adapter, + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +Both adapters work identically with [Structured Outputs](/ai/structured-outputs/overview) — including `stream: true` — and accept the same `modelOptions` (temperature, top_p, max_tokens, stop, …). The reasoning section below applies to `openaiText`; `openaiChatCompletions` accepts `modelOptions.reasoning.effort` but cannot stream summary text. + +## Basic Usage - Custom API Key + +```typescript +import { chat } from "@tanstack/ai"; +import { createOpenaiChat } from "@tanstack/ai-openai"; + +const adapter = createOpenaiChat("gpt-5.2", process.env.OPENAI_API_KEY!, { + // ... your config options +}); + +const stream = chat({ + adapter, + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Configuration + +```typescript +import { createOpenaiChat, type OpenAITextConfig } from "@tanstack/ai-openai"; + +const config: Omit = { + organization: "org-...", // Optional + baseURL: "https://api.openai.com/v1", // Optional, for custom endpoints +}; + +const adapter = createOpenaiChat("gpt-5.2", process.env.OPENAI_API_KEY!, config); +``` + +## Example: Chat Completion + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages, + }); + + return toServerSentEventsResponse(stream); +} +``` + +## Example: With Tools + +```typescript +import { chat, toServerSentEventsResponse, toolDefinition } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { z } from "zod"; + +const getWeatherDef = toolDefinition({ + name: "get_weather", + description: "Get the current weather", + inputSchema: z.object({ + location: z.string(), + }), +}); + +const getWeather = getWeatherDef.server(async ({ location }) => { + // Fetch weather data + return { temperature: 72, conditions: "sunny" }; +}); + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages, + tools: [getWeather], + }); + + return toServerSentEventsResponse(stream); +} +``` + +## Model Options + +OpenAI supports various provider-specific options. Sampling parameters live here too — `temperature`, `top_p`, and `max_output_tokens` (the Responses API token-limit key) — rather than as root-level props on `chat()`: + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Hello!" }], + modelOptions: { + temperature: 0.7, + max_output_tokens: 1000, + top_p: 0.9, + }, +}); +``` + +> The `openaiChatCompletions` adapter targets `/v1/chat/completions`, where the token-limit key is `max_tokens` (not `max_output_tokens`). If you previously passed `temperature` / `topP` / `maxTokens` at the root of `chat()`, see [Moving Sampling Options into modelOptions](/ai/migration/sampling-options-to-model-options). + +### Reasoning + +Enable reasoning for models that support it (e.g., GPT-5, O3). This allows the model to show its reasoning process, which is streamed as `thinking` chunks: + +```typescript ignore +modelOptions: { + reasoning: { + effort: "medium", // "none" | "minimal" | "low" | "medium" | "high" + summary: "detailed", // "auto" | "detailed" (optional) + }, +} +``` + +When reasoning is enabled, the model's reasoning process is streamed separately from the response text and appears as a collapsible thinking section in the UI. + +## Summarization + +Summarize long text content: + +```typescript +import { summarize } from "@tanstack/ai"; +import { openaiSummarize } from "@tanstack/ai-openai"; + +const result = await summarize({ + adapter: openaiSummarize("gpt-5-mini"), + text: "Your long text to summarize...", + maxLength: 100, + style: "concise", // "concise" | "bullet-points" | "paragraph" +}); + +console.log(result.summary); +``` + +## Image Generation + +Generate images with DALL-E: + +```typescript +import { generateImage } from "@tanstack/ai"; +import { openaiImage } from "@tanstack/ai-openai"; + +const result = await generateImage({ + adapter: openaiImage("gpt-image-1"), + prompt: "A futuristic cityscape at sunset", + numberOfImages: 1, + size: "1024x1024", +}); + +console.log(result.images); +``` + +### Image Model Options + +```typescript +import { generateImage } from "@tanstack/ai"; +import { openaiImage } from "@tanstack/ai-openai"; + +const result = await generateImage({ + adapter: openaiImage("gpt-image-1"), + prompt: "...", + modelOptions: { + quality: "high", // "high" | "medium" | "low" | "auto" + }, +}); +``` + +## Text-to-Speech + +Generate speech from text: + +```typescript +import { generateSpeech } from "@tanstack/ai"; +import { openaiSpeech } from "@tanstack/ai-openai"; + +const result = await generateSpeech({ + adapter: openaiSpeech("tts-1"), + text: "Hello, welcome to TanStack AI!", + voice: "alloy", + format: "mp3", +}); + +// result.audio contains base64-encoded audio +console.log(result.format); // "mp3" +``` + +### TTS Voices + +Available voices: `alloy`, `echo`, `fable`, `onyx`, `nova`, `shimmer`, `ash`, `ballad`, `coral`, `sage`, `verse` + +### TTS Model Options + +```typescript +import { generateSpeech } from "@tanstack/ai"; +import { openaiSpeech } from "@tanstack/ai-openai"; + +const result = await generateSpeech({ + adapter: openaiSpeech("tts-1-hd"), + text: "High quality speech", + modelOptions: { + instructions: "Speak slowly and clearly.", // voice instructions (not supported by tts-1/tts-1-hd) + }, +}); +``` + +## Transcription + +Transcribe audio to text: + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { openaiTranscription } from "@tanstack/ai-openai"; +import { audioFile } from "./audio"; + +const result = await generateTranscription({ + adapter: openaiTranscription("whisper-1"), + audio: audioFile, // File object or base64 string + language: "en", +}); + +console.log(result.text); // Transcribed text +``` + +### Transcription Model Options + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { openaiTranscription } from "@tanstack/ai-openai"; +import { audioFile } from "./audio"; + +const result = await generateTranscription({ + adapter: openaiTranscription("whisper-1"), + audio: audioFile, + responseFormat: "verbose_json", + prompt: "Technical terms: API, SDK", + modelOptions: { + temperature: 0, + timestamp_granularities: ["word", "segment"], + }, +}); + +// Access the transcribed text +console.log(result.text); +``` + +### Speaker Diarization + +Use `gpt-4o-transcribe-diarize` for speaker-labeled transcripts: + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { openaiTranscription } from "@tanstack/ai-openai"; +import { meetingAudioFile } from "./audio"; + +const result = await generateTranscription({ + adapter: openaiTranscription("gpt-4o-transcribe-diarize"), + audio: meetingAudioFile, + modelOptions: { + known_speaker_names: ["agent", "customer"], + known_speaker_references: [ + "data:audio/wav;base64,...", + "data:audio/wav;base64,...", + ], + }, +}); + +for (const segment of result.segments ?? []) { + console.log(segment.speaker, segment.start, segment.end, segment.text); +} +``` + +When no response format is specified, `gpt-4o-transcribe-diarize` requests default to `response_format: "diarized_json"` and `chunking_strategy: "auto"`; passing a top-level `responseFormat` of `"json"` or `"text"` opts out of speaker segments. `known_speaker_names` and `known_speaker_references` must be provided together (up to 4, matching lengths). OpenAI does not support `prompt`, `include`, or `timestamp_granularities` with diarized transcription. + +## Environment Variables + +Set your API key in environment variables: + +```bash +OPENAI_API_KEY=sk-... +``` + +## API Reference + +Every factory pair follows the same shape: the short factory (`openaiText`, `openaiImage`, …) reads `OPENAI_API_KEY` from the environment, while the `create*` variant takes an explicit API key. Both take `model` as the first argument. + +### `openaiText(model, config?)` + +Creates an OpenAI text adapter against the Responses API (`/v1/responses`) using `OPENAI_API_KEY` from the environment. + +**Parameters:** + +- `model` - OpenAI chat model id (e.g. `"gpt-5.2"`, `"gpt-4o-mini"`) +- `config?.organization` - Organization ID (optional) +- `config?.baseURL` - Custom base URL (optional) + +### `createOpenaiChat(model, apiKey, config?)` + +Creates an OpenAI text adapter (Responses API) with an explicit API key. + +### `openaiChatCompletions(model, config?)` + +Creates an OpenAI text adapter that targets `/v1/chat/completions` instead of the Responses API. See [Chat Completions API](#chat-completions-api) for when to use this over `openaiText`. + +### `createOpenaiChatCompletions(model, apiKey, config?)` + +Creates an OpenAI chat-completions adapter with an explicit API key. + +### `openaiSummarize(model, config?)` / `createOpenaiSummarize(model, apiKey, config?)` + +Creates an OpenAI summarization adapter. + +### `openaiImage(model, config?)` / `createOpenaiImage(model, apiKey, config?)` + +Creates an OpenAI image generation adapter (DALL-E, gpt-image). + +### `openaiSpeech(model, config?)` / `createOpenaiSpeech(model, apiKey, config?)` + +Creates an OpenAI text-to-speech adapter. + +### `openaiTranscription(model, config?)` / `createOpenaiTranscription(model, apiKey, config?)` + +Creates an OpenAI transcription adapter for Whisper, GPT-4o transcription, and GPT-4o diarized transcription models. + +### `openaiVideo(model, config?)` / `createOpenaiVideo(model, apiKey, config?)` + +Creates an OpenAI video generation adapter (Sora). _Experimental._ + +### `openaiRealtime(...)` / `openaiRealtimeToken(...)` + +Realtime voice adapters. See [Realtime Voice Chat](/ai/media/realtime-chat) for usage. + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about tools +- [Other Adapters](/ai/adapters/anthropic) - Explore other providers + +## Provider Tools + +OpenAI exposes several native tools beyond user-defined function calls. +Import them from `@tanstack/ai-openai/tools` and pass them into +`chat({ tools: [...] })`. + +> For the full concept, a comparison matrix, and type-gating details, see +> [Provider Tools](/ai/tools/provider-tools). + +### `webSearchTool` + +Enables the model to run a web search and return grounded results with +citations. Pass a `WebSearchToolConfig` object (typed from the OpenAI SDK) +to configure the tool. + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { webSearchTool } from "@tanstack/ai-openai/tools"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "What's new in AI this week?" }], + tools: [webSearchTool({ type: "web_search" })], +}); +``` + +**Supported models:** GPT-4o, GPT-5, and Responses API-capable models. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `webSearchPreviewTool` + +The preview variant of web search with additional options for controlling +search context size and user location. Use this when you want fine-grained +control over the search context sent to the model. + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { webSearchPreviewTool } from "@tanstack/ai-openai/tools"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Latest news about TypeScript" }], + tools: [ + webSearchPreviewTool({ + type: "web_search_preview_2025_03_11", + search_context_size: "high", + }), + ], +}); +``` + +**Supported models:** GPT-4o and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `fileSearchTool` + +Searches OpenAI vector stores that you have pre-populated, letting the model +retrieve relevant document chunks. Provide the `vector_store_ids` to search +and optionally limit results with `max_num_results` (1–50). + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { fileSearchTool } from "@tanstack/ai-openai/tools"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "What does the handbook say about PTO?" }], + tools: [ + fileSearchTool({ + type: "file_search", + vector_store_ids: ["vs_abc123"], + max_num_results: 5, + }), + ], +}); +``` + +**Supported models:** GPT-4o and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `imageGenerationTool` + +Allows the model to generate images inline during a conversation using +DALL-E/GPT-Image. Pass quality, size, and style options via the config object. + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { imageGenerationTool } from "@tanstack/ai-openai/tools"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Draw a logo for my app" }], + tools: [ + imageGenerationTool({ + quality: "high", + size: "1024x1024", + }), + ], +}); +``` + +**Supported models:** GPT-5 and GPT-Image-capable models. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `codeInterpreterTool` + +Gives the model a sandboxed Python execution environment. The `container` +field configures the execution environment; pass the full +`CodeInterpreterToolConfig` object. + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { codeInterpreterTool } from "@tanstack/ai-openai/tools"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Analyse this CSV and plot a chart" }], + tools: [ + codeInterpreterTool({ type: "code_interpreter", container: { type: "auto" } }), + ], +}); +``` + +**Supported models:** GPT-4o and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `mcpTool` + +Connects the model to a remote MCP (Model Context Protocol) server, exposing +all its capabilities as callable tools. Provide either `server_url` or +`connector_id` — not both. + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { mcpTool } from "@tanstack/ai-openai/tools"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "List my GitHub issues" }], + tools: [ + mcpTool({ + server_url: "https://mcp.example.com", + server_label: "github", + }), + ], +}); +``` + +**Supported models:** GPT-4o and above. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `computerUseTool` + +Lets the model observe a virtual desktop via screenshots and interact with +it using keyboard and mouse events. Provide the display dimensions and the +execution environment type. + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { computerUseTool } from "@tanstack/ai-openai/tools"; + +const stream = chat({ + adapter: openaiText("computer-use-preview"), + messages: [{ role: "user", content: "Open Chrome and navigate to example.com" }], + tools: [ + computerUseTool({ + type: "computer_use_preview", + display_width: 1024, + display_height: 768, + environment: "browser", + }), + ], +}); +``` + +**Supported models:** `computer-use-preview`. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `localShellTool` + +Provides the model with a local shell for executing system commands. Takes no +arguments — the tool is enabled simply by including it in the `tools` array. + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { localShellTool } from "@tanstack/ai-openai/tools"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Run the test suite and summarise failures" }], + tools: [localShellTool()], +}); +``` + +**Supported models:** GPT-5.x and other agent-capable models. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `shellTool` + +A function-style shell tool that exposes shell execution as a structured +function call. Pass an `environment` object to attach container config and +hosted skills. + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { shellTool } from "@tanstack/ai-openai/tools"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Count lines in all JS files" }], + tools: [shellTool()], +}); +``` + +**Supported models:** GPT-5.x and other agent-capable models. Responses API +only — Chat Completions does not support the shell tool. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +#### Attaching hosted skills + +Pass `environment.skills` to load provider-managed skill bundles into the +shell's container (Responses API only). + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { shellTool } from "@tanstack/ai-openai/tools"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages, + tools: [ + shellTool({ + environment: { + type: "container_auto", + skills: [ + { type: "skill_reference", skill_id: "skill_abc", version: "2" }, + ], + }, + }), + ], + }); + + return toServerSentEventsResponse(stream); +} +``` + +For the full reference — skill shape, `version` string format, and the +Anthropic equivalent — see [Provider Skills](/ai/tools/provider-skills). + +### `applyPatchTool` + +Lets the model apply unified-diff patches to modify files directly. Takes no +arguments — include it in the `tools` array to enable patch application. + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { applyPatchTool } from "@tanstack/ai-openai/tools"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Fix the import paths in src/index.ts" }], + tools: [applyPatchTool()], +}); +``` + +**Supported models:** GPT-5.x and other agent-capable models. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `customTool` + +Defines a custom Responses API tool with an explicit name, description, and +format. Use this when none of the structured tool types fits your use case. +Unlike branded provider tools, `customTool` returns a plain `Tool` and is +accepted by any chat model. + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { customTool } from "@tanstack/ai-openai/tools"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Look up order #1234" }], + tools: [ + customTool({ + type: "custom", + name: "lookup_order", + description: "Look up the status of a customer order by order ID", + }), + ], +}); +``` + +**Supported models:** all Responses API models. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). diff --git a/mintlify/ai/adapters/opencode.md b/mintlify/ai/adapters/opencode.md new file mode 100644 index 000000000..0ea49d540 --- /dev/null +++ b/mintlify/ai/adapters/opencode.md @@ -0,0 +1,186 @@ +--- +title: OpenCode +id: opencode-adapter +order: 14 +description: "Use OpenCode as a chat backend in TanStack AI — agent harness with local tool execution, token-level streaming, stateful sessions, and tool bridging via @tanstack/ai-opencode." +keywords: + - tanstack ai + - opencode + - opencode sdk + - harness + - agent + - coding agent + - adapter +--- + +The OpenCode adapter runs [OpenCode](https://opencode.ai) as a chat backend, driving it over its local HTTP server (`@opencode-ai/sdk`). Unlike HTTP provider adapters, this is a **harness adapter**: OpenCode runs its own agent loop and executes its own tools — shell commands, file reads and edits, search — locally on your server. Each `chat()` call runs one full harness turn; assistant text and reasoning stream as true token-level deltas, and the harness's tool activity streams back as already-resolved tool-call events your UI can render. + +> **Server-only.** The adapter spawns (or attaches to) an `opencode serve` process, so it only works in a Node.js server environment — never in the browser. Treat it like giving OpenCode a shell on the machine it runs on, and configure permissions accordingly. + +## Installation + +```bash +npm install @tanstack/ai-opencode +``` + +The `opencode` CLI must be installed and its providers authenticated on the host: + +```bash +npm install -g opencode-ai +opencode auth login +``` + +A runnable demo lives at [`examples/sandbox-web`](https://github.com/TanStack/ai/tree/main/examples/sandbox-web) — switch the harness (Claude Code, Codex, OpenCode, Grok Build) and sandbox provider per run, with session resume, the harness tool timeline, permission modes, and tool bridging, wired into a TanStack Start app. + +## Models + +OpenCode is provider-agnostic: it resolves any `provider/model` id its configured providers support. Address models as `provider/model` (the adapter splits on the first `/`): + +```typescript +import { chat } from "@tanstack/ai"; +import { opencodeText } from "@tanstack/ai-opencode"; + +const stream = chat({ + adapter: opencodeText("anthropic/claude-sonnet-4-5", { + directory: "/path/to/project", + permissionMode: "acceptEdits", + }), + messages: [{ role: "user", content: "Fix the failing test in utils.test.ts" }], +}); +``` + +## Configuration + +| Option | Description | +| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `directory` | Working directory for the harness session. Defaults to `process.cwd()`. | +| `baseUrl` | Attach to an already-running `opencode serve` (e.g. `http://127.0.0.1:4096`) instead of spawning a new server per turn. | +| `hostname` | Hostname for the spawned server. Defaults to the SDK default (`127.0.0.1`). | +| `port` | Port for the spawned server. Defaults to the SDK default (`4096`). | +| `permissionMode` | `'default'` (bridged tools run, everything else that prompts is rejected), `'acceptEdits'` (also auto-approves file edits), or `'bypassPermissions'` (allow all). | +| `onPermissionRequest` | Custom permission handler; replaces the default policy entirely. | +| `config` | Extra OpenCode config merged with the adapter's MCP and permission config. | + +Per-call overrides — `sessionId`, `permissionMode`, `directory` — go through `modelOptions`. + +## Permissions + +OpenCode asks for permission before mutating files or running commands. A headless server has no one to answer those prompts, so the adapter applies a policy automatically — it never hangs a turn: + +- **`'default'`** — bridged TanStack tools run; anything else that would prompt (edits, shell, web fetch) is rejected. +- **`'acceptEdits'`** — additionally auto-approves file-mutation requests (edit / write / patch). +- **`'bypassPermissions'`** — approves everything. Only use this against a sandbox or scratch directory. + +Provide `onPermissionRequest` to implement your own policy (e.g. allow-list specific commands). + +## Stateful Sessions + +OpenCode sessions are stateful — the harness keeps the full working context (files read, commands run, conclusions reached) between turns. The adapter surfaces the session id of every fresh run as a custom stream event named `opencode.session-id`; thread it back via `modelOptions.sessionId` to resume. When resuming, only the latest user message is sent — the harness already holds the prior context. + +Server endpoint: + +```typescript +import { + chat, + chatParamsFromRequest, + toServerSentEventsResponse, +} from "@tanstack/ai"; +import { opencodeText } from "@tanstack/ai-opencode"; + +export async function POST(request: Request) { + const params = await chatParamsFromRequest(request); + + // Extra fields the client puts in the connection `body` arrive here. + const sessionId = + typeof params.forwardedProps.sessionId === "string" + ? params.forwardedProps.sessionId + : undefined; + + const stream = chat({ + adapter: opencodeText("anthropic/claude-sonnet-4-5", { + directory: "/path/to/project", + permissionMode: "acceptEdits", + }), + messages: params.messages, + modelOptions: { sessionId }, + }); + + return toServerSentEventsResponse(stream); +} +``` + +Client (React) — capture the session id from the custom event and send it back on subsequent requests: + +```typescript +import { useState } from "react"; +import { useChat } from "@tanstack/ai-react"; +import { fetchServerSentEvents } from "@tanstack/ai-client"; + +function CodingAssistant() { + const [sessionId, setSessionId] = useState(undefined); + + const { messages, sendMessage } = useChat({ + connection: fetchServerSentEvents("/api/chat", () => ({ + body: { sessionId }, + })), + onCustomEvent: (name, value) => { + if ( + name === "opencode.session-id" && + typeof value === "object" && + value !== null && + "sessionId" in value && + typeof value.sessionId === "string" + ) { + setSessionId(value.sessionId); + } + }, + }); + + // ... render messages; harness tool activity (bash, edit, read, ...) + // arrives as regular tool-call parts with results. +} +``` + +Sessions live on the server that ran them, so resuming only works against the same server instance (or a shared `baseUrl`). + +## Tools + +Two kinds of tools flow through this adapter: + +1. **Built-in harness tools** are executed by OpenCode itself and stream back as tool-call events with results already attached: `bash`, `edit`, `write`, `read`, `grep`, and the agent's running todo plan (surfaced as an `opencode.todo` custom event). Your code never executes them. + +2. **Your TanStack tools** are bridged *into* the harness: the adapter starts a short-lived Streamable-HTTP MCP server on `127.0.0.1` for the duration of the turn and registers it with OpenCode. Define tools as usual with `toolDefinition().server()`; tool-call events come back under the names you registered (OpenCode prefixes MCP tools `tanstack_…` internally, which the adapter strips). + +```typescript +import { z } from "zod"; +import { chat, toolDefinition } from "@tanstack/ai"; +import { opencodeText } from "@tanstack/ai-opencode"; + +const lookupTicket = toolDefinition({ + name: "lookup_ticket", + description: "Look up an issue ticket by id", + inputSchema: z.object({ ticketId: z.string() }), +}).server(async ({ ticketId }) => { + return { ticketId, status: "open", title: "Crash on startup" }; +}); + +const stream = chat({ + adapter: opencodeText("anthropic/claude-sonnet-4-5"), + messages: [{ role: "user", content: "What's the status of ticket T-123?" }], + tools: [lookupTicket], +}); +``` + +**Client-side and approval-gated tools are not supported.** The harness executes tools inside a live process, which cannot pause across HTTP requests to wait for a browser round-trip or a human approval. Passing a tool without a server `execute()` implementation — or one marked `needsApproval` — fails fast with a descriptive error. Run those tools outside the harness with a regular provider adapter. + +## Structured Output + +`structuredOutput()` is best-effort: OpenCode's prompt API has no native JSON-schema channel, so the schema is embedded as a prompt instruction in a fresh, one-shot session and the final text is parsed (markdown fences are stripped when present). It works for finalization after a chat, but a plain provider adapter (e.g. `@tanstack/ai-openai`) is the better choice when structured extraction is the primary job — it's faster, deterministic, and doesn't spawn a harness. + +## Limitations + +- **Server-only (Node).** The adapter spawns or attaches to an `opencode serve` process. +- **The harness owns the agent loop.** TanStack's agent-loop strategies and per-iteration middleware don't apply inside a harness turn. +- **No sampling controls.** `temperature`-style options don't exist here. +- **Sessions are server-local.** Resume requires hitting the same server instance (or a shared `baseUrl`). +- **Cold starts.** Spawning a server per turn adds first-token latency; point the adapter at a long-lived `baseUrl` to avoid it. diff --git a/mintlify/ai/adapters/openrouter.md b/mintlify/ai/adapters/openrouter.md new file mode 100644 index 000000000..6845fc582 --- /dev/null +++ b/mintlify/ai/adapters/openrouter.md @@ -0,0 +1,334 @@ +--- +title: OpenRouter Adapter +id: openrouter-adapter +description: "Access 300+ LLMs from OpenAI, Anthropic, Google, Meta, Mistral, and more through a single API with OpenRouter in TanStack AI." +keywords: + - tanstack ai + - openrouter + - multi-provider + - unified api + - llm gateway + - 300 models + - adapter +--- + +OpenRouter is TanStack AI's first official AI partner and the recommended starting point for most projects. It provides access to 300+ models from OpenAI, Anthropic, Google, Meta, Mistral, and many more — all through a single API key and unified interface. + +## Installation + +```bash +npm install @tanstack/ai-openrouter +``` + +## Basic Usage + +```typescript +import { chat } from "@tanstack/ai"; +import { openRouterText } from "@tanstack/ai-openrouter"; + +const stream = chat({ + adapter: openRouterText("openai/gpt-5"), + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +## Configuration + +```typescript +import { createOpenRouterText } from "@tanstack/ai-openrouter"; + +const adapter = createOpenRouterText( + "openai/gpt-5", + process.env.OPENROUTER_API_KEY!, + { + serverURL: "https://openrouter.ai/api/v1", // Optional + httpReferer: "https://your-app.com", // Optional, for rankings + appTitle: "Your App Name", // Optional, for rankings + }, +); +``` + +## Available Models + +OpenRouter provides access to 300+ models from various providers. Models use the format `provider/model-name`: + +```text +model: "openai/gpt-5.1" +model: "anthropic/claude-sonnet-4.5" +model: "google/gemini-3.1-pro-preview" +model: "meta-llama/llama-4-maverick" +model: "deepseek/deepseek-v3.2" +``` + +See the full list at [openrouter.ai/models](https://openrouter.ai/models). + +## Example: Chat Completion + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openRouterText } from "@tanstack/ai-openrouter"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: openRouterText("openai/gpt-5"), + messages, + }); + + return toServerSentEventsResponse(stream); +} +``` + +## Example: With Tools + +```typescript +import { chat, toServerSentEventsResponse, toolDefinition } from "@tanstack/ai"; +import { openRouterText } from "@tanstack/ai-openrouter"; +import { z } from "zod"; + +const getWeatherDef = toolDefinition({ + name: "get_weather", + description: "Get the current weather", + inputSchema: z.object({ + location: z.string(), + }), +}); + +const getWeather = getWeatherDef.server(async ({ location }) => { + return { temperature: 72, conditions: "sunny" }; +}); + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: openRouterText("openai/gpt-5"), + messages, + tools: [getWeather], + }); + + return toServerSentEventsResponse(stream); +} +``` + + + +## Environment Variables + +Set your API key in environment variables: + +```bash +OPENROUTER_API_KEY=sk-or-... +``` + +## Model Routing + +OpenRouter can automatically route requests to the best available provider: + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openRouterText } from "@tanstack/ai-openrouter"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: openRouterText("openrouter/auto"), + messages, + modelOptions: { + models: [ + "openai/gpt-5.5", + "anthropic/claude-sonnet-4.5", + "google/gemini-3.1-pro-preview", + ], + }, + }); + + return toServerSentEventsResponse(stream); +} +``` + +## Model Options + +OpenRouter supports various provider-specific options. Sampling parameters live here too — `temperature`, `topP`, and `maxCompletionTokens` (OpenRouter's token-limit key for the chat adapter) — rather than as root-level props on `chat()`: + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openRouterText } from "@tanstack/ai-openrouter"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: openRouterText("anthropic/claude-sonnet-4.5"), + messages, + modelOptions: { + temperature: 0.7, + topP: 0.9, + maxCompletionTokens: 1024, + }, + }); + + return toServerSentEventsResponse(stream); +} +``` + +> If you previously passed `temperature` / `topP` / `maxTokens` at the root of `chat()`, see [Moving Sampling Options into modelOptions](/ai/migration/sampling-options-to-model-options). + +## Chat Completions vs Responses (beta) + +OpenRouter exposes two OpenAI-compatible wire formats, and the adapter +package ships one of each: + +| Adapter | Endpoint | Status | When to use | +| -------------------------- | ------------------------- | -------- | ---------------------------------------------------------------------------- | +| `openRouterText` | `/v1/chat/completions` | Stable | Default for almost everything. Broadest model + tool support. | +| `openRouterResponsesText` | `/v1/responses` | Beta | OpenAI Responses-shaped request/response; richer multi-turn state on OpenAI-style models. | + +Both adapters route to any underlying model OpenRouter supports +(`anthropic/...`, `google/...`, `meta-llama/...`, etc.) — the wire format +describes how your client talks to OpenRouter, not which provider answers. +`/v1/responses` is OpenAI's newer API surface; OpenRouter implements it so +clients that prefer that wire format can use it across the same 300+ +model catalogue. + +```typescript +import { chat } from "@tanstack/ai"; +import { openRouterResponsesText } from "@tanstack/ai-openrouter"; + +const stream = chat({ + adapter: openRouterResponsesText("anthropic/claude-sonnet-4.5"), + messages: [{ role: "user", content: "Hello!" }], +}); +``` + +Caveats while the Responses adapter is in beta: + +- Function tools are supported; OpenRouter's branded server-tools (web + search, file search, …) are not yet wired through this path — use + `openRouterText` if you need those. +- If in doubt, prefer `openRouterText`. The Chat Completions endpoint has + broader provider coverage and feature parity today. + +## Cost Tracking + +OpenRouter reports the actual cost of each request inline on the streamed +response. When present, the adapter forwards it on the terminal `RUN_FINISHED` +event under `usage.cost`, with OpenRouter's per-request breakdown under +`usage.costDetails`. This is the cost OpenRouter itself reports for the +request — it is **not** computed locally from token counts, so it already +accounts for routing, fallback providers, BYOK, and cached-token pricing. See +OpenRouter's [Usage Accounting](https://openrouter.ai/docs/use-cases/usage-accounting) +docs for the meaning and units of these fields. + +```typescript ignore +import { chat, type RunFinishedEvent, type StreamChunk } from "@tanstack/ai"; +import { openRouterText } from "@tanstack/ai-openrouter"; + +function isRunFinished(chunk: StreamChunk): chunk is RunFinishedEvent { + return "finishReason" in chunk; +} + +for await (const chunk of chat({ + adapter: openRouterText("openai/gpt-5"), + messages: [{ role: "user", content: "Hello!" }], +})) { + if (isRunFinished(chunk)) { + console.log("cost:", chunk.usage?.cost); + console.log("breakdown:", chunk.usage?.costDetails); + } +} +``` + +The same `usage` (including `cost` / `costDetails`) is passed to middleware via +the `onUsage` and `onFinish` hooks. When OpenRouter does not report a cost, the +fields are simply absent and the stream completes normally. Both +`openRouterText` and `openRouterResponsesText` populate cost when OpenRouter +returns it. + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about tools + +## Provider Tools + +> **Migrated from `createWebSearchTool`?** This factory was renamed to +> `webSearchTool` and moved to the `/tools` subpath in this release. +> See [Migration Guide §6](/ai/migration/migration#6-provider-tools-moved-to-tools-subpath) +> for the exact before/after. + +OpenRouter's gateway exposes web search via a plugin that works across +any proxied chat model. Import it from `@tanstack/ai-openrouter/tools`. + +> For the full concept, a comparison matrix, and type-gating details, see +> [Provider Tools](/ai/tools/provider-tools). + +### `webSearchTool` + +Adds web search capability to any OpenRouter-proxied chat model. The factory +accepts OpenRouter's `WebSearchConfig` directly — pick the `engine` +(`auto`, `native`, `exa`, `firecrawl`, or `parallel`), cap results with +`maxResults` / `maxTotalResults`, restrict which sites can appear in results +with `allowedDomains` / `excludedDomains`, and optionally pass +`searchContextSize` or `userLocation` for finer control. + +```typescript +import { chat } from "@tanstack/ai"; +import { openRouterText } from "@tanstack/ai-openrouter"; +import { webSearchTool } from "@tanstack/ai-openrouter/tools"; + +const stream = chat({ + adapter: openRouterText("openai/gpt-5"), + messages: [{ role: "user", content: "What's new in AI this week?" }], + tools: [ + webSearchTool({ + engine: "exa", + maxResults: 5, + allowedDomains: ["arxiv.org", "openai.com"], + }), + ], +}); +``` + +**Supported models:** all OpenRouter chat models. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + +### `webFetchTool` + +Lets any OpenRouter-proxied chat model fetch the full contents of a URL the +model chooses, instead of running a search. The factory accepts OpenRouter's +`WebFetchServerToolConfig` directly — pick the fetch `engine` (`auto` — the +default, `native`, `openrouter`, `exa`, or `firecrawl`), cap how much page +content the model receives with `maxContentTokens`, cap how many fetches the +model can make per request with `maxUses`, and restrict which URLs the model +can fetch with `allowedDomains` / `blockedDomains`. + +> The `native` engine routes to the underlying provider's own fetch (for +> example, Anthropic's `web_fetch` on Claude models). Native fetch +> capabilities vary, so `allowedDomains` and `blockedDomains` may be +> ignored. Use `openrouter`, `exa`, or `firecrawl` if you need consistent +> behaviour across models. + +```typescript +import { chat } from "@tanstack/ai"; +import { openRouterText } from "@tanstack/ai-openrouter"; +import { webFetchTool } from "@tanstack/ai-openrouter/tools"; + +const stream = chat({ + adapter: openRouterText("openai/gpt-5"), + messages: [ + { role: "user", content: "Summarize https://example.com/article" }, + ], + tools: [ + webFetchTool({ + engine: "openrouter", + maxContentTokens: 4000, + allowedDomains: ["example.com"], + }), + ], +}); +``` + +**Supported models:** all OpenRouter chat models. See [Provider Tools](/ai/tools/provider-tools#which-models-support-which-tools). + diff --git a/mintlify/ai/advanced/built-in-middleware.md b/mintlify/ai/advanced/built-in-middleware.md new file mode 100644 index 000000000..7c9bf1cb1 --- /dev/null +++ b/mintlify/ai/advanced/built-in-middleware.md @@ -0,0 +1,247 @@ +--- +title: Built-in Middleware +id: built-in-middleware +order: 2 +description: "Ready-made TanStack AI chat() middleware — toolCacheMiddleware for caching tool results, contentGuardMiddleware for redacting streamed text, and otelMiddleware for OpenTelemetry tracing." +keywords: + - tanstack ai + - middleware + - built-in middleware + - tool cache + - content guard + - redaction + - opentelemetry +--- + +TanStack AI ships ready-made middleware so you don't have to hand-roll the common cases. Each one is an ordinary [`ChatMiddleware`](/ai/advanced/middleware) — drop it into the `middleware` array of any `chat()` call. This page documents every built-in. + +| Middleware | Import | What it does | +|------------|--------|--------------| +| `toolCacheMiddleware` | `@tanstack/ai/middlewares` | Cache tool-call results by name + arguments | +| `contentGuardMiddleware` | `@tanstack/ai/middlewares` | Redact / transform / block streamed text content | +| `otelMiddleware` | `@tanstack/ai/middlewares/otel` | Emit OpenTelemetry spans + GenAI metrics | + +> `toolCacheMiddleware` and `contentGuardMiddleware` are exported from the main `@tanstack/ai/middlewares` barrel. `otelMiddleware` lives on its own subpath (`@tanstack/ai/middlewares/otel`) so that importing the barrel never eagerly pulls in `@opentelemetry/api` (an optional peer dependency). + +## toolCacheMiddleware + +Caches tool call results based on tool name and arguments. When a tool is called with the same name and arguments as a previous call, the cached result is returned immediately without re-executing the tool. + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { toolCacheMiddleware } from "@tanstack/ai/middlewares"; +import { weatherTool, stockTool } from "./tools"; + +const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: [{ role: "user", content: "What's the weather in Paris?" }], + tools: [weatherTool, stockTool], + middleware: [ + toolCacheMiddleware({ + ttl: 60_000, // Cache entries expire after 60 seconds + maxSize: 50, // Keep at most 50 entries (LRU eviction) + toolNames: ["getWeather"], // Only cache specific tools + }), + ], +}); +``` + +**Options:** + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `maxSize` | `number` | `100` | Maximum cache entries. Oldest evicted first (LRU). Only applies to the default in-memory storage. | +| `ttl` | `number` | `Infinity` | Time-to-live in milliseconds. Expired entries are not served. | +| `toolNames` | `string[]` | All tools | Only cache these tools. Others pass through. | +| `keyFn` | `(toolName, args) => string` | `JSON.stringify([toolName, args])` | Custom cache key derivation. | +| `storage` | `ToolCacheStorage` | In-memory Map | Custom storage backend. When provided, `maxSize` is ignored — the storage manages its own capacity. | + +**Behaviors:** + +- Only successful tool calls are cached — errors are never stored +- Cache hits trigger `{ type: 'skip', result }` via `onBeforeToolCall` +- LRU eviction: when `maxSize` is reached, the oldest entry is removed (default storage only) +- Cache hits refresh the entry's LRU position (moved to most-recently-used) + +**Custom key function** — useful when you want to ignore certain arguments: + +```typescript +import { toolCacheMiddleware } from "@tanstack/ai/middlewares"; + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null; +} + +toolCacheMiddleware({ + keyFn: (toolName, args) => { + // Ignore pagination, cache by query only. `args` is `unknown`, so + // narrow it with a type guard before destructuring. + if (!isRecord(args)) return JSON.stringify([toolName, args]); + const { page, ...rest } = args; + return JSON.stringify([toolName, rest]); + }, +}); +``` + +### Custom Storage + +By default the cache lives in-memory and is scoped to a single `toolCacheMiddleware()` instance. Pass a `storage` option to use an external backend like Redis, localStorage, or a database. This also enables **sharing a cache across multiple `chat()` calls**. + +The storage interface: + +```typescript +import { type ToolCacheEntry, type ToolCacheStorage } from "@tanstack/ai/middlewares"; + +// Implement this interface (exported from `@tanstack/ai/middlewares`): +interface MyStorage extends ToolCacheStorage { + getItem: (key: string) => ToolCacheEntry | undefined | Promise; + setItem: (key: string, value: ToolCacheEntry) => void | Promise; + deleteItem: (key: string) => void | Promise; +} + +// ToolCacheEntry is { result: unknown; timestamp: number } +``` + +All methods may return a `Promise` for async backends. The middleware handles TTL checking — your storage just needs to store and retrieve entries. + +**Redis example:** + +```typescript +import { chat } from "@tanstack/ai"; +import { createClient } from "redis"; +import { toolCacheMiddleware, type ToolCacheStorage } from "@tanstack/ai/middlewares"; +import { adapter, messages } from "./server"; +import { weatherTool } from "./tools"; + +const redis = createClient(); + +const redisStorage: ToolCacheStorage = { + getItem: async (key) => { + const raw = await redis.get(`tool-cache:${key}`); + return raw ? JSON.parse(raw) : undefined; + }, + setItem: async (key, value) => { + await redis.set(`tool-cache:${key}`, JSON.stringify(value)); + }, + deleteItem: async (key) => { + await redis.del(`tool-cache:${key}`); + }, +}; + +const stream = chat({ + adapter, + messages, + tools: [weatherTool], + middleware: [toolCacheMiddleware({ storage: redisStorage, ttl: 60_000 })], +}); +``` + +**Sharing a cache across requests:** + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { toolCacheMiddleware, type ToolCacheStorage } from "@tanstack/ai/middlewares"; +import { globalCache, app, adapter } from "./server"; +import { weatherTool } from "./tools"; + +// Create storage once, reuse across chat() calls +const sharedStorage: ToolCacheStorage = { + getItem: (key) => globalCache.get(key), + setItem: (key, value) => { globalCache.set(key, value); }, + deleteItem: (key) => { globalCache.delete(key); }, +}; + +// Both requests share the same cache +app.post("/api/chat", async (req: { body: { messages: unknown[] } }) => { + const stream = chat({ + adapter, + messages: req.body.messages, + tools: [weatherTool], + middleware: [toolCacheMiddleware({ storage: sharedStorage })], + }); + return toServerSentEventsResponse(stream); +}); +``` + +## contentGuardMiddleware + +Filters or transforms streamed text content as it flows through `onChunk`. Use it to redact sensitive data (SSNs, emails, API keys), enforce a profanity filter, or rewrite text on the fly. Rules are applied to `TEXT_MESSAGE_CONTENT` chunks; all other chunk types pass through untouched. + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { contentGuardMiddleware } from "@tanstack/ai/middlewares"; + +const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: [{ role: "user", content: "Tell me about customer 123-45-6789" }], + middleware: [ + contentGuardMiddleware({ + rules: [ + // Regex + replacement + { pattern: /\b\d{3}-\d{2}-\d{4}\b/g, replacement: "[SSN REDACTED]" }, + // Custom transform function + { fn: (text) => text.replaceAll("badword", "****") }, + ], + strategy: "buffered", + }), + ], +}); +``` + +**Options:** + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `rules` | `ContentGuardRule[]` | — | **Required.** Applied in order; each rule receives the previous rule's output. A rule is either `{ pattern: RegExp; replacement: string }` or `{ fn: (text: string) => string }`. | +| `strategy` | `'delta' \| 'buffered'` | `'buffered'` | How content is matched. See below. | +| `bufferSize` | `number` | `50` | (Buffered only) Characters held back before emitting, so patterns spanning chunk boundaries still match. Set it ≥ the longest pattern you expect. Flushed at stream end. | +| `blockOnMatch` | `boolean` | `false` | When `true`, drop the entire chunk if any rule changes the content (instead of emitting the filtered version). | +| `onFiltered` | `(info: ContentFilteredInfo) => void` | — | Callback fired whenever a rule changes content. Receives `{ messageId, original, filtered, strategy }`. | + +**Matching strategies:** + +- **`'buffered'` (default)** — Accumulates content and applies rules to the settled portion, holding back a `bufferSize` look-behind window so a pattern split across two chunks (`"...123-45"` then `"-6789..."`) is still caught. The buffer is flushed when the message or run ends. Use this for anything that can span deltas — which is most redaction. +- **`'delta'`** — Applies rules to each delta in isolation as it arrives. Fastest and lowest-latency, but a pattern split across a chunk boundary may slip through. Use only when your patterns are guaranteed to fit within a single delta. + +**Behaviors:** + +- Only `TEXT_MESSAGE_CONTENT` chunks are inspected; every other chunk type passes through. +- A rule that doesn't change the text is a no-op — the chunk passes through unchanged. +- With `blockOnMatch: true`, a matched chunk is dropped entirely (returns `null` from `onChunk`) rather than emitting the redacted text. +- The `onFiltered` callback is for observability/audit — it fires with the before/after text but does not alter what is emitted. + +## otelMiddleware + +Emits vendor-neutral OpenTelemetry traces and metrics for every `chat()` call — a root span per call, a child span per agent-loop iteration, and a grandchild span per tool execution, all tagged with [GenAI semantic-convention attributes](https://opentelemetry.io/docs/specs/semconv/gen-ai/). + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { otelMiddleware } from "@tanstack/ai/middlewares/otel"; +import { trace, metrics } from "@opentelemetry/api"; +import { messages } from "./server"; + +const otel = otelMiddleware({ + tracer: trace.getTracer("my-app"), + meter: metrics.getMeter("my-app"), // optional — enables GenAI histograms +}); + +const result = await chat({ + adapter: openaiText("gpt-5.5"), + messages, + middleware: [otel], +}); +``` + +`otelMiddleware` has its own configuration surface (content capture, redaction, span-name formatting, attribute enrichment, lifecycle callbacks) and requires the optional `@opentelemetry/api` peer dependency. See the dedicated [OpenTelemetry](/ai/advanced/otel) guide for full setup, the span/metric catalogue, and all options. + +## Writing your own + +These built-ins are just `ChatMiddleware` objects — nothing about them is privileged. To build your own, see the [Middleware](/ai/advanced/middleware) guide for the full hook reference, the context object, and composition rules. + +## Next Steps + +- [Middleware](/ai/advanced/middleware) — the full lifecycle and hook reference +- [OpenTelemetry](/ai/advanced/otel) — `otelMiddleware` in depth \ No newline at end of file diff --git a/mintlify/ai/advanced/debug-logging.md b/mintlify/ai/advanced/debug-logging.md new file mode 100644 index 000000000..d08ae49b1 --- /dev/null +++ b/mintlify/ai/advanced/debug-logging.md @@ -0,0 +1,198 @@ +--- +title: Debug Logging +id: debug-logging +order: 3 +description: "Turn on structured, category-toggleable debug logging to see every chunk, middleware transform, and tool call inside TanStack AI." +keywords: + - tanstack ai + - debug + - logging + - logger + - pino + - troubleshooting + - chunks + - middleware debugging +--- + +# Debug Logging + +You have a `chat()` that isn't behaving as expected — a missing chunk, a middleware that doesn't seem to fire, a tool call with wrong args. By the end of this guide, you'll have turned on debug logging and will see every chunk, middleware transform, and tool call flowing through your call. + +## Turn it on + +Add `debug: true` to any activity call: + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: [{ role: "user", content: "Hello" }], + debug: true, +}); +``` + +Every internal event now prints to the console with a `[tanstack-ai:]` prefix: + +``` +[tanstack-ai:request] activity=chat provider=openai model=gpt-5.5 messages=1 tools=0 stream=true +[tanstack-ai:agentLoop] run started +[tanstack-ai:provider] provider=openai type=response.output_text.delta +[tanstack-ai:output] type=TEXT_MESSAGE_CONTENT +... +``` + +## Narrow what's printed + +Pass a `DebugConfig` object instead of `true`. Every unspecified category defaults to `true`, so toggle by setting specific flags to `false`: + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +chat({ + adapter: openaiText("gpt-5.5"), + messages: [{ role: "user", content: "Hello" }], + debug: { middleware: false }, // everything except middleware +}); +``` + +If you want to see ONLY a specific set of categories, set the rest to `false` explicitly. Errors default to `true` — keep them on unless you really want total silence: + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +chat({ + adapter: openaiText("gpt-5.5"), + messages: [{ role: "user", content: "Hello" }], + debug: { + provider: true, + output: true, + middleware: false, + tools: false, + agentLoop: false, + config: false, + errors: true, // keep errors on — they're cheap and important + request: false, + }, +}); +``` + +## Pipe into your own logger + +Pass a `Logger` implementation and all debug output flows through it instead of `console`: + +```typescript +import { chat, type Logger } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import pino from "pino"; +import { messages } from "./server"; + +const pinoLogger = pino(); +const logger: Logger = { + debug: (msg, meta) => pinoLogger.debug(meta, msg), + info: (msg, meta) => pinoLogger.info(meta, msg), + warn: (msg, meta) => pinoLogger.warn(meta, msg), + error: (msg, meta) => pinoLogger.error(meta, msg), +}; + +chat({ + adapter: openaiText("gpt-5.5"), + messages, + debug: { logger }, // all categories on, piped to pino +}); +``` + +The default logger is exported as `ConsoleLogger` if you want to wrap it: + +```typescript +import { ConsoleLogger } from "@tanstack/ai"; +``` + +### Your `Logger` is wrapped in a try/catch + +If your `Logger` implementation throws — a cyclic-meta `JSON.stringify`, a transport that rejects synchronously, a typo in a bound `this` — the exception is swallowed so it never masks the real error that triggered the log call (for example, a provider SDK failure inside the chat stream). You won't see the log line, but the pipeline error still surfaces through thrown exceptions and `RUN_ERROR` chunks. + +If you need to know when your own logger is failing, guard inside your implementation: + +```typescript +import { type Logger } from "@tanstack/ai"; +import pino from "pino"; + +const pinoLogger = pino(); +const logger: Logger = { + debug: (msg, meta) => { + try { + pinoLogger.debug(meta, msg); + } catch (err) { + // surface to wherever you track infra errors + process.stderr.write(`logger failed: ${String(err)}\n`); + } + }, + info: (msg, meta) => pinoLogger.info(meta, msg), + warn: (msg, meta) => pinoLogger.warn(meta, msg), + error: (msg, meta) => pinoLogger.error(meta, msg), +}; +``` + +## Categories reference + +| Category | Logs | Applies to | +|----------|------|------------| +| `request` | Outgoing call to a provider (model, message count, tool count) | All activities | +| `provider` | Every raw chunk/frame received from a provider SDK | Streaming activities (`chat`, `realtime`, and streaming `generateAudio`/`generateSpeech`/`generateTranscription`) | +| `output` | Every chunk or result yielded to the caller | All activities | +| `middleware` | Inputs and outputs around every middleware hook | `chat()` only | +| `tools` | Before/after tool call execution | `chat()` only | +| `agentLoop` | Agent-loop iterations and phase transitions | `chat()` only | +| `config` | Config transforms returned by middleware `onConfig` hooks | `chat()` only | +| `errors` | Every caught error anywhere in the pipeline | All activities | + +## Errors are always logged + +Errors flow through the logger unconditionally — even when you omit `debug`: + +```typescript +import { chat } from "@tanstack/ai"; +import { adapter } from "./server"; + +chat({ adapter, messages: [{ role: "user", content: "Hello" }] }); // still prints [tanstack-ai:errors] ... on failure +``` + +To fully silence (including errors), set `debug: false` or `debug: { errors: false }`. Errors also always reach the caller via thrown exceptions or `RUN_ERROR` stream chunks — the logger is additive, not the only surface. + +## Non-chat activities + +The same `debug` option works on every activity: + +```typescript +import { + summarize, + generateImage, + generateSpeech, + generateAudio, + generateTranscription, + type Logger, +} from "@tanstack/ai"; +import { adapter } from "./server"; +import { logger } from "./logger"; + +const text = "Long article to summarize..."; +const audio = new File([""], "recording.mp3", { type: "audio/mpeg" }); + +summarize({ adapter, text, debug: true }); +generateImage({ adapter, prompt: "a cat", debug: { logger } }); +generateSpeech({ adapter, text, debug: { request: true } }); +generateAudio({ adapter, prompt: "ambient piano", debug: true }); +generateTranscription({ adapter, audio, debug: { provider: true } }); +``` + +When streaming any of these (`generateAudio`, `generateSpeech`, `generateTranscription` with `stream: true`), the `provider` category emits the raw SDK chunks and `output` emits the AG-UI-shaped chunks yielded to the caller — useful when a media pipeline looks stuck or the bytes arriving don't match what you expected. + +The chat-only categories (`middleware`, `tools`, `agentLoop`, `config`) simply never fire for these activities because those concepts don't exist in their pipelines. + +## Related + +If you're building middleware and want to see chunks flow through it, `debug: { middleware: true }` is faster than writing a logging middleware. See [Middleware](/ai/advanced/middleware) for writing your own middleware. diff --git a/mintlify/ai/advanced/extend-adapter.md b/mintlify/ai/advanced/extend-adapter.md new file mode 100644 index 000000000..bad7fc136 --- /dev/null +++ b/mintlify/ai/advanced/extend-adapter.md @@ -0,0 +1,213 @@ +--- +title: Extend Adapter +id: extend-adapter +order: 8 +description: "Extend TanStack AI adapter factories with custom model IDs and fine-tuned models while keeping full type safety for input modalities and provider options." +keywords: + - tanstack ai + - extendAdapter + - custom models + - fine-tuned models + - createModel + - type safety + - adapter factory +--- + +# Extending Adapters with Custom Models + +The `extendAdapter` utility allows you to extend existing adapter factories (like `openaiText`, `anthropicText`) with custom model names while maintaining full type safety for input modalities and provider options. + +## Basic Usage + +```typescript +import { createModel, extendAdapter } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +// Define your custom models using createModel helper +const myOpenaiModel = createModel('my-fine-tuned-gpt4',['text', 'image']); +const myOpenaiModelButCooler = createModel('my-fine-tuned-gpt5',['text', 'image']); + + +// Create an extended adapter factory - simple API, no type parameters needed +const myOpenai = extendAdapter(openaiText, [ + myOpenaiModel, + myOpenaiModelButCooler +]) + +// Use with original models - full type inference preserved +const gpt5Adapter = myOpenai('gpt-5.5') + +// Use with custom models - your custom types are applied +const customAdapter = myOpenai('my-fine-tuned-gpt4') + +// Works seamlessly with chat() +import { chat } from '@tanstack/ai' + +const stream = chat({ + adapter: myOpenai('my-fine-tuned-gpt4'), + messages: [{ role: 'user', content: 'Hello!' }] +}) +``` + +## The `createModel` Helper + +The `createModel` function provides a clean way to define custom models with full type inference: + +```typescript +import { createModel } from '@tanstack/ai' + +// Arguments define name and input modalities +const model = createModel( + 'my-model', // model name (literal type inferred) + ['text', 'image'] // input modalities (tuple type inferred) +) +``` + + +## Model Definition Structure + +A custom model definition (`ExtendedModelDef`) has the required properties `name`, `input`, and `modelOptions`, plus optional `features` and `tools`. The two `createModel` overloads let you fill these in two ways. + +### Defining Input Modalities + +The positional form takes a model name and an `input` array specifying which content types your model supports: + +```typescript +import { createModel } from '@tanstack/ai' + +const models = [ + createModel('text-only-model', ['text']), + createModel('multimodal-model', ['text', 'image', 'audio']), +] as const +``` + +Available modalities: `'text'`, `'image'`, `'audio'`, `'video'`, `'document'` + +### Capabilities-object form + +To attach typed `modelOptions`, declared `features`, or provider `tools` to a custom model, use the second `createModel` overload, which takes a capabilities object as its second argument: + +```typescript +import { createModel } from '@tanstack/ai' +import type { OpenAITextProviderOptions } from '@tanstack/ai-openai' + +// Type brand for provider options — the value is unused at runtime. +const modelOptions: OpenAITextProviderOptions = {} + +const reasoner = createModel('my-reasoner', { + input: ['text'], + features: ['reasoning', 'structured_outputs'], + tools: ['web_search'], + modelOptions, +}) +``` + +- `input` — supported input modalities (same as the positional form). +- `features` — declared feature flags (e.g. `'reasoning'`, `'structured_outputs'`). +- `tools` — declared provider tools (e.g. `'web_search'`). +- `modelOptions` — a type brand for the provider options accepted by this model; the value is unused at runtime, so declare an empty object typed as the provider options (e.g. `const modelOptions: OpenAITextProviderOptions = {}`). + +## Preserving Original Factory Behavior + +`extendAdapter` fully preserves the original factory's signature, including any configuration parameters: + +```typescript +import { createModel, extendAdapter } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +const customModels = [createModel('my-fine-tuned-gpt4', ['text', 'image'])] as const + +const myOpenai = extendAdapter(openaiText, customModels) + +// Config parameter is preserved +const adapter = myOpenai('my-fine-tuned-gpt4', { + baseURL: 'https://my-proxy.com/v1', + timeout: 30000 +}) +``` + +## Type Safety + +The extended adapter provides full type safety: + +```typescript ignore +import { extendAdapter, createModel } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +const myOpenai = extendAdapter(openaiText, [createModel('custom-model', ['text'])]) + +// ✅ Original models work with their original types +const a1 = myOpenai('gpt-5.5') + +// ✅ Custom models work with your defined types +const a2 = myOpenai('custom-model') + +// ❌ Type error: invalid model name +// Note: Type checking works when you assign the result to a variable +const invalid = myOpenai('nonexistent-model') // TypeScript error! +``` + + +## Runtime Behavior + +At runtime, `extendAdapter` simply passes through to the original factory: + +- No validation is performed on custom model names +- The original factory receives exactly what you pass +- This allows the original provider's API to handle the model name + +This design is intentional - it allows you to: +- Use fine-tuned model names that the provider accepts but TypeScript doesn't know about +- Proxy requests to different backends that accept custom model identifiers +- Add type safety without runtime overhead + +## Example: OpenAI-Compatible Proxy + +A common use case is typing models for an OpenAI-compatible proxy: + +```typescript +import { extendAdapter, createModel } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +// Models available through your proxy +const proxyModels = [ + createModel( + 'llama-3.1-70b', + ['text'] + ), + createModel( + 'mixtral-8x7b', + ['text'] + ), +] as const + +const proxyAdapter = extendAdapter(openaiText, proxyModels) + +// Use with your proxy's base URL +const adapter = proxyAdapter('llama-3.1-70b', { + baseURL: 'https://my-llm-proxy.com/v1' +}) +``` + +## Example: Fine-tuned Models + +Adding type safety for your fine-tuned models: + +```typescript +import { chat, createModel, extendAdapter } from '@tanstack/ai' +import { anthropicText } from '@tanstack/ai-anthropic' + +const fineTunedModels = [ + createModel( + 'ft:claude-3-opus:my-org:custom-task:abc123', + ['text', 'image'] + ), +] as const + +const myAnthropic = extendAdapter(anthropicText, fineTunedModels) + +chat({ + adapter: myAnthropic('ft:claude-3-opus:my-org:custom-task:abc123'), + messages: [{ role: 'user', content: 'Analyze this...' }] +}) +``` diff --git a/mintlify/ai/advanced/middleware.md b/mintlify/ai/advanced/middleware.md new file mode 100644 index 000000000..b619e5632 --- /dev/null +++ b/mintlify/ai/advanced/middleware.md @@ -0,0 +1,919 @@ +--- +title: Middleware +id: middleware +order: 1 +description: "Hook into every stage of TanStack AI's chat() lifecycle with middleware — logging, analytics, stream transforms, tool interception, and side effects." +keywords: + - tanstack ai + - middleware + - chat middleware + - lifecycle hooks + - observability + - logging + - tool interception + - stream transform +--- + +Middleware lets you hook into every stage of the `chat()` lifecycle — from configuration to streaming, tool execution, usage tracking, and completion. You can observe, transform, or short-circuit behavior at each stage without modifying your adapter or tool implementations. + +Common use cases include: + +- **Logging and observability** — track token usage, tool execution timing, errors +- **Configuration transforms** — inject system prompts, adjust temperature per iteration, filter tools +- **Stream processing** — redact sensitive content, transform chunks, drop unwanted events +- **Tool call interception** — validate arguments, cache results, abort on dangerous calls +- **Side effects** — send analytics, update databases, trigger notifications + +## Quick Start + +Pass an array of middleware to the `chat()` function: + +```typescript +import { chat, type ChatMiddleware } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +const logger: ChatMiddleware = { + name: "logger", + onStart: (ctx) => { + console.log(`[${ctx.requestId}] Chat started`); + }, + onFinish: (ctx, info) => { + console.log(`[${ctx.requestId}] Finished in ${info.duration}ms`); + }, +}; + +const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: [{ role: "user", content: "Hello" }], + middleware: [logger], +}); +``` + +> **Just want to see chunks flowing through your middleware during development?** +> Use `debug: { middleware: true }` on your `chat()` call — no custom middleware required. See [Debug Logging](/ai/advanced/debug-logging). + +## Lifecycle Overview + +Every `chat()` invocation follows a predictable lifecycle. Middleware hooks fire at specific phases: + +```mermaid +graph TD + A["chat() called"] --> B["onConfig (phase: init)"] + B --> C[onStart] + C --> D["onConfig (phase: beforeModel)"] + D --> E["Adapter streams response"] + E --> F["onChunk (for each chunk)"] + F --> G{Tool calls?} + G -->|No| H[onUsage] + G -->|Yes| I[onBeforeToolCall] + I --> J[Tool executes] + J --> K[onAfterToolCall] + K --> L{Continue loop?} + L -->|Yes| D + L -->|No| H + H --> SO{outputSchema?} + SO -->|No| M{Outcome} + SO -->|Yes| SOC[onStructuredOutputConfig] + SOC --> SOM["onConfig (phase: structuredOutput)"] + SOM --> SOS["Structured-output finalization (onChunk, onUsage)"] + SOS --> M + M -->|Success| N[onFinish] + M -->|Abort| O[onAbort] + M -->|Error| P[onError] + + style I fill:#e1f5ff + style J fill:#ffe1e1 + style SOC fill:#e1f5ff + style SOM fill:#e1f5ff + style SOS fill:#e1f5ff + style N fill:#e1ffe1 + style O fill:#fff4e1 + style P fill:#ffe1e1 +``` + +### Phase Transitions + +The context's `phase` field tracks where you are in the lifecycle: + +| Phase | When | Hooks Called | +|-------|------|-------------| +| `init` | Once at startup | `onConfig` | +| `beforeModel` | Before each model call (per iteration) | `onConfig` | +| `modelStream` | While adapter streams chunks | `onChunk`, `onUsage` | +| `beforeTools` | Before tool execution | `onBeforeToolCall` | +| `afterTools` | After tool execution | `onAfterToolCall` | +| `structuredOutput` | During the final structured-output adapter call (when `outputSchema` is set **and** the adapter does not declare `supportsCombinedToolsAndSchema()`). Chunks from `adapter.structuredOutputStream` (or the synthesized non-streaming fallback) flow through `onChunk` with this phase, and `onUsage` fires for the final call's tokens. **Does not fire** for adapters that natively combine tools + schema in one streaming call (modern OpenAI Chat Completions, OpenAI Responses, Claude 4.5+, Gemini 3.x, Grok 4.x family — see issue #605); on that path middleware observes the run through `beforeModel` / `modelStream` as usual. | `onStructuredOutputConfig`, `onConfig`, `onChunk`, `onUsage` | + +## Hooks Reference + +### onConfig + +Called once during `init` (startup) and once per iteration during `beforeModel` (before each model call). When `chat()` was invoked with `outputSchema`, `onConfig` additionally re-fires at the structured-output boundary with `ctx.phase === 'structuredOutput'`, receiving the post-`onStructuredOutputConfig` view of the config — so a single-iteration run with `outputSchema` fires `onConfig` three times (`init` + `beforeModel` + `structuredOutput`). Use it to transform the configuration that the model receives. + +Return a **partial** config object with only the fields you want to change — they are shallow-merged with the current config automatically. No need to spread the existing config. + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; + +const dynamicTemperature: ChatMiddleware = { + name: "dynamic-temperature", + onConfig: (ctx, config) => { + if (ctx.phase === "init") { + // Add a system prompt at startup — only systemPrompts is overwritten + return { + systemPrompts: [ + ...config.systemPrompts, + "You are a helpful assistant.", + ], + }; + } + + if (ctx.phase === "beforeModel" && ctx.iteration > 0) { + // Increase temperature on retries. Sampling params live in the + // provider-native modelOptions object — `temperature` is universal, + // so it's the same key across providers. Spread the existing + // modelOptions so other model options stay unchanged. + const current = + typeof config.modelOptions?.temperature === "number" + ? config.modelOptions.temperature + : 0.7; + return { + modelOptions: { + ...config.modelOptions, + temperature: Math.min(current + 0.1, 1.0), + }, + }; + } + }, +}; +``` + +> Sampling parameters (`temperature`, `top_p` / `topP`, the various `max*Tokens` keys) live inside `modelOptions` under each provider's native name — they are no longer root config fields. `temperature` happens to be spelled the same across every provider, so the example above is provider-agnostic; if you mutate a token limit instead, use the provider-native key (e.g. `max_output_tokens` for OpenAI, `num_predict` nested under `modelOptions.options` for Ollama). See [Moving Sampling Options into modelOptions](/ai/migration/sampling-options-to-model-options). + +**Config fields you can transform:** + +| Field | Type | Description | +|-------|------|-------------| +| `messages` | `ModelMessage[]` | Conversation history | +| `systemPrompts` | `string[]` | System prompts | +| `tools` | `Tool[]` | Available tools | +| `metadata` | `Record` | Request metadata | +| `modelOptions` | `Record` | Provider-native options — this is where sampling params (`temperature`, `top_p` / `topP`, the provider's `max*Tokens` key) now live, alongside every other model-specific knob. See [Moving Sampling Options into modelOptions](/ai/migration/sampling-options-to-model-options). | + +When multiple middleware define `onConfig`, the config is **piped** through them in order — each receives the merged config from the previous middleware. + +### onStructuredOutputConfig + +Called once at the start of the final structured-output adapter call — only when `chat()` was invoked with `outputSchema` **and** the adapter takes the legacy finalization path (i.e. does not declare `supportsCombinedToolsAndSchema()`). Pipes through middleware in order, like `onConfig`, but with access to the **JSON Schema** being sent to the provider. Use this hook when you need to transform the schema (e.g., inject `$defs`, strip vendor-incompatible keywords) or apply structured-output-specific behavior (e.g., suppress system prompts on the final call). + +> Native-combined adapters (modern OpenAI, Claude 4.5+, Gemini 3.x, Grok 4.x — see issue #605) skip the separate finalization call and never invoke this hook. If you need to mutate the schema for a native-combined adapter, do it in `onConfig` (the schema is on `config.modelOptions` / the request — adapter-specific). + +Return a **partial** `StructuredOutputMiddlewareConfig` with only the fields you want to change — they are shallow-merged with the current config. Return `void` to pass through. + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; +import { sharedDefs } from "./defs"; + +const injectDefs: ChatMiddleware = { + name: "inject-defs", + onStructuredOutputConfig: (_ctx, config) => { + // `config.outputSchema` is the JSON Schema being sent to the provider + return { + outputSchema: { + ...config.outputSchema, + $defs: { ...sharedDefs }, + }, + }; + }, +}; +``` + +**Config fields you can transform:** + +| Field | Type | Description | +|-------|------|-------------| +| `messages` | `ModelMessage[]` | Conversation history sent to the final call | +| `systemPrompts` | `SystemPrompt[]` | System prompts on the final call | +| `metadata` | `Record` | Request metadata | +| `modelOptions` | `Record` | Provider-native options — this is where sampling params (`temperature`, `top_p` / `topP`, the provider's `max*Tokens` key) now live, alongside every other model-specific knob. See [Moving Sampling Options into modelOptions](/ai/migration/sampling-options-to-model-options). | +| `outputSchema` | `JSONSchema` | JSON Schema being sent to the provider for structured output | + +**Ordering at the structured-output boundary:** + +1. `onStructuredOutputConfig` fires first, piping through every middleware in array order. +2. `onConfig` then re-fires at the same boundary with `ctx.phase === 'structuredOutput'`, receiving the post-`onStructuredOutputConfig` view of the config (minus `outputSchema`). Use `onConfig` for general-purpose transforms that apply to every adapter call; use `onStructuredOutputConfig` when you need access to the schema. + +When multiple middleware define `onStructuredOutputConfig`, the config is **piped** through them in order — each receives the merged config from the previous middleware. + +### onStart + +Called once after the initial `onConfig` completes. Use it for setup tasks like initializing timers or logging. + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; + +const timer: ChatMiddleware = { + name: "timer", + onStart: (ctx) => { + console.log(`Request ${ctx.requestId} started at iteration ${ctx.iteration}`); + }, +}; +``` + +### onChunk + +Called for every chunk streamed from the adapter. You can observe, transform, expand, or drop chunks. + +```typescript ignore +import { type ChatMiddleware } from "@tanstack/ai"; + +const redactor: ChatMiddleware = { + name: "redactor", + onChunk: (ctx, chunk) => { + if (chunk.type === "TEXT_MESSAGE_CONTENT") { + // Transform: redact sensitive content + return { + ...chunk, + delta: chunk.delta.replace(/\b\d{3}-\d{2}-\d{4}\b/g, "[REDACTED]"), + }; + } + // Return void to pass through unchanged + }, +}; +``` + +**Return values:** + +| Return | Effect | +|--------|--------| +| `void` / `undefined` | Chunk passes through unchanged | +| `StreamChunk` | Replaces the original chunk | +| `StreamChunk[]` | Expands into multiple chunks | +| `null` | Drops the chunk entirely | + +When multiple middleware define `onChunk`, chunks flow through them in order. If one middleware drops a chunk (returns `null`), subsequent middleware never see it. + +#### Chunk types you'll see + +`onChunk` receives every [AG-UI event](https://docs.ag-ui.com/introduction) the run produces — not just text. Narrow on `chunk.type` (a discriminated union) before reading type-specific fields. The common ones: + +| `chunk.type` | Meaning | Key fields | +|--------------|---------|-----------| +| `RUN_STARTED` / `RUN_FINISHED` / `RUN_ERROR` | Run lifecycle boundaries | `runId`, `finishReason`, `usage` (on finish), `message` (on error) | +| `TEXT_MESSAGE_START` / `TEXT_MESSAGE_CONTENT` / `TEXT_MESSAGE_END` | Assistant text streaming | `messageId`, `delta` (content) | +| `TOOL_CALL_START` / `TOOL_CALL_ARGS` / `TOOL_CALL_END` | Tool invocation streaming | `toolCallId`, `toolCallName`, `delta` (args), result on end | +| `STEP_STARTED` / `STEP_FINISHED` | Thinking / reasoning steps | `delta`, `signature` | +| `STATE_SNAPSHOT` / `STATE_DELTA` | Agent state sync | `snapshot`, `delta` | +| `CUSTOM` | Extensibility events (incl. structured-output — see below) | `name`, `value` | + +See the [AG-UI protocol docs](https://docs.ag-ui.com/introduction) for the full event catalogue and exact field shapes. + +#### Transforming structured-output chunks + +There is **no separate `onStructuredOutputChunk` hook** — and you don't need one. When `chat()` is invoked with `outputSchema`, the structured-output chunks (the JSON `TEXT_MESSAGE_CONTENT` deltas, plus the `structured-output.start` / `structured-output.complete` CUSTOM events and any finalization `RUN_ERROR`) flow through the **same `onChunk` hook** as everything else. You transform, expand, or drop them exactly like any other chunk. + +How you distinguish them depends on which finalization path the adapter takes: + +- **Separate-finalization adapters** (the legacy path — adapters that don't declare `supportsCombinedToolsAndSchema()`): `ctx.phase === 'structuredOutput'` during the finalization call. Discriminate on the phase. +- **Native-combined adapters** (modern OpenAI Chat Completions / Responses, Claude 4.5+, Gemini 3.x, Grok 4.x — see issue #605): the schema-constrained JSON is produced on the model's natural final turn, so **`ctx.phase` stays `'modelStream'`** — the `'structuredOutput'` phase never fires. Discriminate on the CUSTOM event name (`structured-output.start` / `structured-output.complete`) instead. + +```typescript ignore +import { type ChatMiddleware } from "@tanstack/ai"; + +const redactStructuredOutput: ChatMiddleware = { + name: "redact-structured-output", + onChunk: (ctx, chunk) => { + // Separate-finalization path: the JSON streams as TEXT_MESSAGE_CONTENT + // during the 'structuredOutput' phase. Transform the delta like any + // other text chunk — here, redact anything that looks like an SSN before + // it reaches the client. + if ( + ctx.phase === "structuredOutput" && + chunk.type === "TEXT_MESSAGE_CONTENT" + ) { + return { + ...chunk, + delta: chunk.delta.replace(/\b\d{3}-\d{2}-\d{4}\b/g, "[REDACTED]"), + }; + } + + // Both paths: the validated object arrives as a CUSTOM + // `structured-output.complete` event. On the native-combined path this is + // your only signal (ctx.phase never flips to 'structuredOutput'), so key + // off the event name, not the phase. `chunk.value` carries { object, raw }. + if (chunk.type === "CUSTOM" && chunk.name === "structured-output.complete") { + console.log("final structured output:", chunk.value); + } + + // Return void to pass everything else through unchanged. + }, +}; +``` + +> Why is there `onStructuredOutputConfig` but no `onStructuredOutputChunk`? Because the **config** shape genuinely differs at the structured-output boundary — it carries an `outputSchema` field that plain `ChatMiddlewareConfig` doesn't (see [onStructuredOutputConfig](#onstructuredoutputconfig)). **Chunks** are all just `StreamChunk` regardless of phase, so one `onChunk` plus `ctx.phase` (or the CUSTOM event name) covers every case — a parallel chunk hook would be redundant. + +### onBeforeToolCall + +Called before each tool executes. The first middleware that returns a non-void decision short-circuits — remaining middleware are skipped for that tool call. + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; + +function isRecord(v: unknown): v is Record { + return typeof v === "object" && v !== null; +} + +const guard: ChatMiddleware = { + name: "guard", + onBeforeToolCall: (ctx, hookCtx) => { + // Block dangerous tools + if (hookCtx.toolName === "deleteDatabase") { + return { type: "abort", reason: "Dangerous operation blocked" }; + } + + // Validate and transform arguments + if (hookCtx.toolName === "search" && isRecord(hookCtx.args) && !hookCtx.args.limit) { + return { + type: "transformArgs", + args: { ...hookCtx.args, limit: 10 }, + }; + } + }, +}; +``` + +**Decision types:** + +| Decision | Effect | +|----------|--------| +| `void` / `undefined` | Continue normally, next middleware can decide | +| `{ type: 'transformArgs', args }` | Replace tool arguments before execution | +| `{ type: 'skip', result }` | Skip execution entirely, use provided result | +| `{ type: 'abort', reason? }` | Abort the entire chat run | + +The `hookCtx` provides: + +| Field | Type | Description | +|-------|------|-------------| +| `toolCall` | `ToolCall` | Raw tool call object | +| `tool` | `Tool \| undefined` | Resolved tool definition | +| `args` | `unknown` | Parsed arguments | +| `toolName` | `string` | Tool name | +| `toolCallId` | `string` | Tool call ID | + +### onAfterToolCall + +Called after each tool execution (or skip). All middleware run — there is no short-circuiting. + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; + +const toolLogger: ChatMiddleware = { + name: "tool-logger", + onAfterToolCall: (ctx, info) => { + if (info.ok) { + console.log(`${info.toolName} completed in ${info.duration}ms`); + } else { + console.error(`${info.toolName} failed:`, info.error); + } + }, +}; +``` + +The `info` object provides: + +| Field | Type | Description | +|-------|------|-------------| +| `toolCall` | `ToolCall` | Raw tool call object | +| `tool` | `Tool \| undefined` | Resolved tool definition | +| `toolName` | `string` | Tool name | +| `toolCallId` | `string` | Tool call ID | +| `ok` | `boolean` | Whether execution succeeded | +| `duration` | `number` | Execution time in milliseconds | +| `result` | `unknown` | Result (when `ok` is true) | +| `error` | `unknown` | Error (when `ok` is false) | + +### onUsage + +Called once per model iteration when the `RUN_FINISHED` chunk includes usage data. Receives the usage object directly. + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; + +const usageTracker: ChatMiddleware = { + name: "usage-tracker", + onUsage: (ctx, usage) => { + console.log( + `Iteration ${ctx.iteration}: ${usage.totalTokens} tokens` + ); + }, +}; +``` + +The `usage` object: + +| Field | Type | Description | +|-------|------|-------------| +| `promptTokens` | `number` | Input tokens | +| `completionTokens` | `number` | Output tokens | +| `totalTokens` | `number` | Total tokens | + +### Terminal Hooks: onFinish, onAbort, onError + +Exactly **one** terminal hook fires per `chat()` invocation. They are mutually exclusive: + +| Hook | When it fires | +|------|--------------| +| `onFinish` | Run completed normally | +| `onAbort` | Run was aborted (via `ctx.abort()`, an external `AbortSignal`, or a `{ type: 'abort' }` decision from `onBeforeToolCall`) | +| `onError` | An unhandled error occurred | + +> **Structured-output lifecycle ordering:** When `chat()` is invoked with `outputSchema`, `onFinish` fires **after** the structured-output finalization call completes — not at the end of the agent loop. `onIteration` does **not** fire for the finalization step; it only fires for agent-loop iterations. +> +> **`onFinish` info fields and structured-output runs:** the `info` object reflects the **agent loop's** terminal state — finalization state is intentionally segregated to keep agent-loop semantics clean. +> +> - `info.content` — the agent loop's accumulated text. Finalization JSON deltas are **not** included here. The structured-output result is delivered via the `structured-output.complete` CUSTOM event, which middleware observes via `onChunk` (with `ctx.phase === 'structuredOutput'`). +> - `info.usage` — the agent loop's last `RUN_FINISHED.usage`. For a tools-less structured-output run (no agent-loop iteration produces `RUN_FINISHED`), this is `undefined`. To capture finalization tokens, use `onUsage` — that hook fires for **every** `RUN_FINISHED` carrying usage, including the finalization call. +> - `info.finishReason` — the agent loop's last `finishReason`. `null` when no agent-loop iteration produced `RUN_FINISHED` (e.g. a tools-less structured-output run). +> - `info.duration` — wall-clock duration of the entire `chat()` invocation, including finalization. +> +> To aggregate usage across the whole run, accumulate from `onUsage` callbacks rather than relying on `info.usage`. + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; + +const terminal: ChatMiddleware = { + name: "terminal", + onFinish: (ctx, info) => { + console.log(`Finished: ${info.finishReason}, ${info.duration}ms`); + console.log(`Content: ${info.content}`); + if (info.usage) { + console.log(`Tokens: ${info.usage.totalTokens}`); + } + }, + onAbort: (ctx, info) => { + console.log(`Aborted: ${info.reason}, ${info.duration}ms`); + }, + onError: (ctx, info) => { + console.error(`Error after ${info.duration}ms:`, info.error); + }, +}; +``` + +The `info` object for `onFinish` (`FinishInfo`): + +| Field | Type | Description | +|-------|------|-------------| +| `finishReason` | `string \| null` | The agent loop's last `finishReason`. `null` when no agent-loop iteration produced `RUN_FINISHED` (e.g. a tools-less `chat({ outputSchema })` run). | +| `duration` | `number` | Total run duration in milliseconds, including any structured-output finalization. | +| `content` | `string` | The agent loop's accumulated text content. Does **not** include finalization JSON deltas — for that, observe the `structured-output.complete` CUSTOM event via `onChunk`. | +| `usage` | `{ promptTokens; completionTokens; totalTokens } \| undefined` | **Optional.** The agent loop's last `RUN_FINISHED.usage`. **Does not include finalization tokens** — use `onUsage` to observe those. Always guard with `if (info.usage)` or `info.usage?.`. | + +## Context Object + +Every hook receives a `ChatMiddlewareContext` as its first argument. It provides request-scoped information and control functions: + +| Field | Type | Description | +|-------|------|-------------| +| `requestId` | `string` | Unique ID for this chat request | +| `streamId` | `string` | Unique ID for this stream | +| `threadId` | `string` | AG-UI thread identifier. Resolves to caller-provided `threadId` (or legacy `conversationId`), or an auto-generated value if neither is supplied. Use this for event correlation. | +| `conversationId` | `string \| undefined` | **Deprecated** alias of `threadId`. Always equals `ctx.threadId`; retained so middleware written before the AG-UI rename keeps working. New middleware should read `ctx.threadId`. | +| `phase` | `ChatMiddlewarePhase` | Current lifecycle phase | +| `iteration` | `number` | Agent loop iteration (0-indexed) | +| `chunkIndex` | `number` | Running count of chunks yielded | +| `signal` | `AbortSignal \| undefined` | External abort signal | +| `abort(reason?)` | `function` | Abort the run from within middleware | +| `context` | `TContext` | User-provided runtime context value | +| `defer(promise)` | `function` | Register a non-blocking side-effect | + +## Typed Runtime Context + +`ChatMiddleware` accepts a context generic. This lets reusable middleware declared outside `chat()` access the same typed runtime context as your tools. + +```typescript +import { chat, type ChatMiddleware } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { session, audit } from "./server"; + +type AppContext = { + userId: string; + audit: { + write(event: { userId: string; requestId: string }): Promise; + }; +}; + +export const auditMiddleware: ChatMiddleware = { + name: "audit", + onStart(ctx) { + ctx.defer( + ctx.context.audit.write({ + userId: ctx.context.userId, + requestId: ctx.requestId, + }) + ); + }, +}; + +chat({ + adapter: openaiText("gpt-5.5"), + messages: [{ role: "user", content: "Hello" }], + middleware: [auditMiddleware], + context: { + userId: session.user.id, + audit, + }, +}); +``` + +When typed middleware or typed tools are present, `chat()` checks that the provided `context` matches the required shape. Existing middleware typed as plain `ChatMiddleware` still works; its `ctx.context` remains `unknown` and does not force a `context` option. + +Runtime context is process-local application state. It is separate from AG-UI `RunAgentInput.context`, which is protocol metadata parsed by `chatParamsFromRequest`. See [Runtime Context](/ai/advanced/runtime-context) for server, client, and client-to-server handoff patterns. + +### Aborting from Middleware + +Call `ctx.abort()` to gracefully stop the run. This triggers the `onAbort` terminal hook: + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; + +const timeout: ChatMiddleware = { + name: "timeout", + onChunk: (ctx) => { + if (ctx.chunkIndex > 1000) { + ctx.abort("Too many chunks"); + } + }, +}; +``` + +### Deferred Side Effects + +Use `ctx.defer()` to register promises that run after the terminal hook without blocking the stream: + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; + +const analytics: ChatMiddleware = { + name: "analytics", + onFinish: (ctx, info) => { + ctx.defer( + fetch("/api/analytics", { + method: "POST", + body: JSON.stringify({ + requestId: ctx.requestId, + duration: info.duration, + tokens: info.usage?.totalTokens, + }), + }) + ); + }, +}; +``` + +## Composing Multiple Middleware + +Middleware execute in array order. The ordering matters for hooks that pipe or short-circuit: + +```typescript +import { chat, type ChatMiddleware } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { authMiddleware, loggingMiddleware, cachingMiddleware } from "./middleware"; + +const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: [{ role: "user", content: "Hello" }], + middleware: [authMiddleware, loggingMiddleware, cachingMiddleware], +}); +``` + +### Composition Rules + +| Hook | Composition | Effect of Order | +|------|------------|----------------| +| `onConfig` | **Piped** — each receives previous output | Earlier middleware transforms first | +| `onStructuredOutputConfig` | **Piped** — each receives previous output | Earlier middleware transforms first | +| `onStart` | Sequential | All run in order | +| `onChunk` | **Piped** — chunks flow through each middleware | If first drops a chunk, later middleware never see it | +| `onBeforeToolCall` | **First-win** — first non-void decision wins | Earlier middleware has priority | +| `onAfterToolCall` | Sequential | All run in order | +| `onUsage` | Sequential | All run in order | +| `onFinish/onAbort/onError` | Sequential | All run in order | + +## Capabilities + +Middleware often need to **share state**. A provider middleware sets something up (a database handle, a per-request counter, a sandbox), and a consumer middleware reads it back later in the same run. Capabilities make that hand-off **type-safe and order-checked**: the consumer declares what it needs, the provider declares what it offers, and `chat()` refuses to run (at compile time _and_ at runtime) if a required capability was never provided. + +### Creating a capability + +A capability is created with `createCapability()('name')` — a **curried** call: + +```typescript +import { createCapability } from "@tanstack/ai"; + +const counterCapability = createCapability<{ value: number }>()("counter"); +const [getCounter, provideCounter] = counterCapability; +``` + +The currying is deliberate: you supply the **value type** explicitly (`<{ value: number }>`) while the **name literal** is inferred from the argument (`"counter"`). A single `createCapability('name')` call can't do both — supplying `T` explicitly stops TypeScript inferring the name, collapsing it to `string` and defeating the compile-time coverage check that keys on the literal name. + +The returned `counterCapability` is a hybrid value: + +- It **destructures to `[get, provide]`** — the two accessors you use inside hooks. +- It **is itself the identity** you list in `requires` / `provides`. There is no separate token to import. + +The accessors: + +| Accessor | Behavior | +|----------|----------| +| `getCounter(ctx)` | Returns the value. **Throws** if the capability was never provided. | +| `getCounter(ctx, { optional: true })` | Returns `TValue \| undefined` — no throw when absent. | +| `provideCounter(ctx, value)` | Sets the value for this run. Call it from `setup`. | + +Equivalently, the context exposes `ctx.get(capability)`, `ctx.getOptional(capability)`, and `ctx.provide(capability, value)` — pass the capability handle directly. These are typed by the handle you pass (`ctx.get(counterCapability)` returns the value type), so `getCounter(ctx)` and `ctx.get(counterCapability)` are interchangeable — use whichever reads better in your hook. + +> **Capability names must be unique across your app.** The compile-time coverage check keys on the name literal (runtime keys on the handle reference), so two capabilities sharing a name will conflate in the type-level check. + +### The `setup` hook + +Provisioning happens in a dedicated `setup(ctx)` hook. It **runs first** — before any `onConfig` (init), across all middleware in array order — so that by the time the rest of the lifecycle begins, every capability is in place. `setup` receives the stable `ChatMiddlewareContext` (not the mutable config), and may be async. + +### `requires` / `provides` / `optionalRequires` + +Three array fields on a middleware declare its capability contract. Each is a `ReadonlyArray` — you list the capability handles themselves: + +| Field | Meaning | +|-------|---------| +| `provides` | Capabilities this middleware sets up. Each one **must** be `provide`d inside `setup`, or `chat()` throws after the setup phase. | +| `requires` | Capabilities this middleware reads. `chat()` validates (compile time + runtime) that some earlier middleware provides each one. | +| `optionalRequires` | Capabilities used **if present** but not required. Non-gating — never causes a validation error. Read with `getX(ctx, { optional: true })`. | + +### Array example + +Author middleware with `defineChatMiddleware` — it sharpens the `requires` / `provides` tuple types so the coverage check and builder can read them precisely. Here a **provider** sets up a counter in `setup`, and a **consumer** reads it in a hook: + +```typescript +import { + chat, + createCapability, + createChatMiddleware, + defineChatMiddleware, +} from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +const counterCapability = createCapability<{ value: number }>()("counter"); +const [getCounter, provideCounter] = counterCapability; + +// Provider: declares `provides` and provisions the value in `setup`. +const withCounter = defineChatMiddleware({ + name: "with-counter", + provides: [counterCapability], + setup(ctx) { + provideCounter(ctx, { value: 0 }); + }, +}); + +// Consumer: declares `requires` and reads the value via `get` in a hook. +const countsChunks = defineChatMiddleware({ + name: "counts-chunks", + requires: [counterCapability], + onChunk(ctx) { + getCounter(ctx).value++; + }, + onFinish(ctx) { + console.log(`Saw ${getCounter(ctx).value} chunks`); + }, +}); + +const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: [{ role: "user", content: "Hello" }], + // Provider must come before the consumer. + middleware: [withCounter, countsChunks], +}); +``` + +If you drop `withCounter` from the array, `chat()` reports a compile-time error at the `middleware` option naming the missing `"counter"` capability — and throws at runtime before the adapter is ever called. + +### Builder example + +`createChatMiddleware()` builds the array through chained `.use()` calls and enforces **provider-before-consumer ordering at compile time**: each `.use()` requires that the middleware's `requires` are already covered by capabilities provided by earlier `.use()` calls. + +```typescript +import { chat, createChatMiddleware } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { withCounter, countsChunks } from "./counter-middleware"; + +const middleware = createChatMiddleware() + .use(withCounter) // provides "counter" + .use(countsChunks) // requires "counter" — OK, already provided above + .build(); + +const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: [{ role: "user", content: "Hello" }], + middleware, +}); +``` + +Swap the two `.use()` calls (`.use(countsChunks).use(withCounter)`) and the builder rejects it at the `.use(countsChunks)` line — the consumer is ordered before its provider, so `"counter"` isn't in the provided set yet. + +### Validation guarantees + +The capability system fails loudly and early: + +- **Compile-time coverage.** A required capability that nothing provides surfaces as a type error at the `middleware` option. This is enforced two ways: an **array coverage check** on `middleware: [...]`, and the order-aware **`createChatMiddleware()` builder** (which additionally enforces ordering). +- **Runtime coverage.** Even if types are bypassed, `chat()` validates coverage and **throws before the adapter runs** if a required capability is missing. +- **Post-`setup` assertion.** If a middleware declares a capability in `provides` but never calls its `provide` accessor during `setup`, `chat()` throws after the setup phase — you can't silently forget to provision. +- **Duplicate provide → last-wins + warning.** If two middleware provide the same capability, the last write wins and a development warning is emitted. +- **Unique names.** Capability `name`s must be unique across your app; the compile-time coverage check keys on the name literal (runtime keys on the handle reference). + +## Built-in Middleware + +TanStack AI ships ready-made middleware for common cases — caching tool results, redacting streamed text, and OpenTelemetry tracing: + +| Middleware | Import | What it does | +|------------|--------|--------------| +| `toolCacheMiddleware` | `@tanstack/ai/middlewares` | Cache tool-call results by name + arguments | +| `contentGuardMiddleware` | `@tanstack/ai/middlewares` | Redact / transform / block streamed text content | +| `otelMiddleware` | `@tanstack/ai/middlewares/otel` | Emit OpenTelemetry spans + GenAI metrics | + +See [Built-in Middleware](/ai/advanced/built-in-middleware) for full options and examples for each. The recipes below show how to build your own. + +## Recipes + +### Rate Limiting + +Limit the number of tool calls per request: + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; + +function rateLimitMiddleware(maxCalls: number): ChatMiddleware { + let toolCallCount = 0; + return { + name: "rate-limit", + onBeforeToolCall: (ctx, hookCtx) => { + toolCallCount++; + if (toolCallCount > maxCalls) { + return { + type: "abort", + reason: `Rate limit: exceeded ${maxCalls} tool calls`, + }; + } + }, + }; +} +``` + +### Audit Trail + +Log every action for compliance: + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; +import { db } from "./db"; + +const auditTrail: ChatMiddleware = { + name: "audit-trail", + onStart: (ctx) => { + ctx.defer( + db.auditLog.create({ + requestId: ctx.requestId, + event: "chat_started", + timestamp: Date.now(), + }) + ); + }, + onAfterToolCall: (ctx, info) => { + ctx.defer( + db.auditLog.create({ + requestId: ctx.requestId, + event: "tool_executed", + toolName: info.toolName, + success: info.ok, + duration: info.duration, + timestamp: Date.now(), + }) + ); + }, + onFinish: (ctx, info) => { + ctx.defer( + db.auditLog.create({ + requestId: ctx.requestId, + event: "chat_finished", + duration: info.duration, + tokens: info.usage?.totalTokens, + timestamp: Date.now(), + }) + ); + }, +}; +``` + +### Per-Iteration Tool Swapping + +Expose different tools at different stages of the agent loop: + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; + +const toolSwapper: ChatMiddleware = { + name: "tool-swapper", + onConfig: (ctx, config) => { + if (ctx.phase !== "beforeModel") return; + + if (ctx.iteration === 0) { + // First iteration: only allow search + return { + tools: config.tools.filter((t) => t.name === "search"), + }; + } + // Later iterations: allow all tools + }, +}; +``` + +### Content Filtering + +Drop or transform chunks before they reach the consumer: + +```typescript ignore +import { type ChatMiddleware } from "@tanstack/ai"; +import { containsProfanity } from "./filters"; + +const contentFilter: ChatMiddleware = { + name: "content-filter", + onChunk: (ctx, chunk) => { + if (chunk.type === "TEXT_MESSAGE_CONTENT") { + if (containsProfanity(chunk.delta)) { + // Drop the chunk entirely + return null; + } + } + }, +}; +``` + +### Error Recovery with Retry Logging + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; +import { alertService } from "./services"; + +const errorRecovery: ChatMiddleware = { + name: "error-recovery", + onError: (ctx, info) => { + ctx.defer( + alertService.send({ + level: "error", + message: `Chat ${ctx.requestId} failed after ${info.duration}ms`, + error: String(info.error), + }) + ); + }, +}; +``` + +## TypeScript Types + +The core middleware types are exported from `@tanstack/ai`: + +```typescript +import type { + ChatMiddleware, + ChatMiddlewareContext, + ChatMiddlewarePhase, + ChatMiddlewareConfig, + StructuredOutputMiddlewareConfig, + ToolCallHookContext, + BeforeToolCallDecision, + AfterToolCallInfo, + IterationInfo, + ToolPhaseCompleteInfo, + UsageInfo, + FinishInfo, + AbortInfo, + ErrorInfo, +} from "@tanstack/ai"; +``` + +The option/type surfaces for the [built-in middleware](/ai/advanced/built-in-middleware) are exported from the `@tanstack/ai/middlewares` subpath (not the main barrel): + +```typescript +import type { + ToolCacheMiddlewareOptions, + ToolCacheStorage, + ToolCacheEntry, + ContentGuardMiddlewareOptions, + ContentGuardRule, + ContentFilteredInfo, +} from "@tanstack/ai/middlewares"; +``` + +## Next Steps + +- [Built-in Middleware](/ai/advanced/built-in-middleware) — `toolCacheMiddleware`, `contentGuardMiddleware`, `otelMiddleware` +- [OpenTelemetry](/ai/advanced/otel) — emit traces and metrics via `otelMiddleware`- [Tools](/ai/tools/tools) — Learn about the isomorphic tool system +- [Agentic Cycle](/ai/chat/agentic-cycle) — Understand the multi-step agent loop +- [Streaming](/ai/chat/streaming) — How streaming works in TanStack AI diff --git a/mintlify/ai/advanced/multimodal-content.md b/mintlify/ai/advanced/multimodal-content.md new file mode 100644 index 000000000..0bea1a795 --- /dev/null +++ b/mintlify/ai/advanced/multimodal-content.md @@ -0,0 +1,515 @@ +--- +title: Multimodal Content +id: multimodal-content +order: 4 +description: "Send images, audio, video, and documents alongside text in TanStack AI messages with typed ContentPart primitives for multimodal models." +keywords: + - tanstack ai + - multimodal + - vision + - images + - audio + - video + - documents + - ContentPart + - ImagePart +--- + +TanStack AI supports multimodal content in messages, allowing you to send images, audio, video, and documents alongside text to AI models that support these modalities. + +When sending messages to AI models, you can include different types of content: +- **Text** - Plain text messages +- **Images** - JPEG, PNG, GIF, WebP images +- **Audio** - Audio files (model-dependent support) +- **Video** - Video files (model-dependent support) +- **Documents** - PDFs and other document types + +## Content Parts + +Multimodal messages use the `ContentPart` type to represent different content types: + +```typescript +import type { ContentPart, ImagePart, TextPart } from '@tanstack/ai' + +// Text content +const textPart: TextPart = { + type: 'text', + content: 'What do you see in this image?' +} + +// Image from base64 data (mimeType is required for data sources) +const imagePart: ImagePart = { + type: 'image', + source: { + type: 'data', + value: 'base64EncodedImageData...', + mimeType: 'image/jpeg' // Required for data sources + }, + metadata: { + // Provider-specific metadata + detail: 'high' // OpenAI detail level + } +} + +// Image from URL (mimeType is optional for URL sources) +const imageUrlPart: ImagePart = { + type: 'image', + source: { + type: 'url', + value: 'https://example.com/image.jpg', + mimeType: 'image/jpeg' // Optional hint for URL sources + } +} +``` + +## Using Multimodal Content in Messages + +Messages can have `content` as either a string or an array of `ContentPart`: + +```typescript +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +const response = await chat({ + adapter: openaiText('gpt-5.5'), + messages: [ + { + role: 'user', + content: [ + { type: 'text', content: 'What is in this image?' }, + { + type: 'image', + source: { + type: 'url', + value: 'https://example.com/photo.jpg' + } + } + ] + } + ] +}) +``` + +## Provider Support + +### OpenAI + +OpenAI supports images and audio in their vision and audio models: + +```typescript +import { openaiText } from '@tanstack/ai-openai' +import { imageBase64 } from './data' + +const adapter = openaiText('gpt-5.5') + +// Image with detail level metadata +const message = { + role: 'user' , + content: [ + { type: 'text' , content: 'Describe this image' }, + { + type: 'image' , + source: { type: 'data' , value: imageBase64, mimeType: 'image/jpeg' }, + metadata: { detail: 'high' } // 'auto' | 'low' | 'high' + } + ] +} +``` + +**Supported modalities by model:** +- `gpt-5.2`, `gpt-5-mini`: text, image +- `gpt-4o-audio`: text, audio + +### Anthropic + +Anthropic's Claude models support images and PDF documents: + +```typescript +import { anthropicText } from '@tanstack/ai-anthropic' +import { imageBase64, pdfBase64 } from './data' + +const adapter = anthropicText('claude-sonnet-4-6') + +// Image with mimeType in source +const imageMessage = { + role: 'user' , + content: [ + { type: 'text' , content: 'What do you see?' }, + { + type: 'image' , + source: { type: 'data' , value: imageBase64, mimeType: 'image/jpeg' } + } + ] +} + +// PDF document +const docMessage = { + role: 'user', + content: [ + { type: 'text', content: 'Summarize this document' }, + { + type: 'document', + source: { type: 'data', value: pdfBase64, mimeType: 'application/pdf' } + } + ] +} +``` + +**Supported modalities:** +- All Claude models (e.g. `claude-haiku-4-5`, `claude-sonnet-5`, `claude-opus-4-8`, `claude-fable-5`): text, image, and document (PDF) + +Check each model's `supports.input` in `@tanstack/ai-anthropic`'s `model-meta.ts` for the authoritative per-model list. + +### Gemini + +Google's Gemini models support a wide range of modalities: + +```typescript +import { geminiText } from '@tanstack/ai-gemini' +import { imageBase64 } from './data' + +const adapter = geminiText('gemini-3-flash-preview') + +// Image with mimeType in source +const message = { + role: 'user', + content: [ + { type: 'text', content: 'Analyze this image' }, + { + type: 'image', + source: { type: 'data', value: imageBase64, mimeType: 'image/png' } + } + ] +} +``` + +**Supported modalities:** +- `gemini-2.5-flash`: text, image, audio, video + +### Ollama + +Ollama supports images in compatible models: + +```typescript +import { ollamaText } from '@tanstack/ai-ollama' +import { imageBase64 } from './data' + +// `ollamaText(model)` takes a model name. The host is read from the +// `OLLAMA_HOST` environment variable (defaults to http://localhost:11434). +const adapter = ollamaText('llama3.2-vision') + +// Image as base64 +const message = { + role: 'user', + content: [ + { type: 'text', content: 'What is in this image?' }, + { + type: 'image', + source: { type: 'data', value: imageBase64, mimeType: 'image/jpeg' } + } + ] +} +``` + +**Note:** Ollama support varies by model. Check the specific model documentation for multimodal capabilities. + +## Source Types + +Content can be provided as either inline data or a URL: + +### Data (Base64) + +Use `type: 'data'` for inline base64-encoded content. **The `mimeType` field is required** to ensure providers receive proper content type information: + +```typescript +const imagePart = { + type: 'image', + source: { + type: 'data', + value: 'iVBORw0KGgoAAAANSUhEUgAAAAUA...', // Base64 string + mimeType: 'image/png' // Required for data sources + } +} + +const audioPart = { + type: 'audio', + source: { + type: 'data', + value: 'base64AudioData...', + mimeType: 'audio/mp3' // Required for data sources + } +} +``` + +### URL + +Use `type: 'url'` for content hosted at a URL. The `mimeType` field is **optional** as providers can often infer it from the URL or response headers: + +```typescript +const imagePart = { + type: 'image' , + source: { + type: 'url' , + value: 'https://example.com/image.jpg', + mimeType: 'image/jpeg' // Optional hint + } +} +``` + +**Note:** Not all providers support URL-based content for all modalities. Check provider documentation for specifics. + +## Backward Compatibility + +String content continues to work as before: + +```typescript +// This still works +const message = { + role: 'user', + content: 'Hello, world!' +} + +// And this works for multimodal +const multimodalMessage = { + role: 'user', + content: [ + { type: 'text', content: 'Hello, world!' }, + { type: 'image', source: { type: 'url', value: '...' } } + ] +} +``` + +## Type Safety + +The multimodal types are fully typed. Provider-specific metadata types are available: + +```typescript +import type { + ContentPart, + ImagePart, + DocumentPart, + AudioPart, + VideoPart, + TextPart +} from '@tanstack/ai' + +// Provider-specific metadata types +import type { OpenAIImageMetadata } from '@tanstack/ai-openai' +import type { AnthropicImageMetadata } from '@tanstack/ai-anthropic' +import type { GeminiImageMetadata } from '@tanstack/ai-gemini' +``` + +### Validating Dynamic Messages + +When receiving messages from external sources (like `request.json()`), the data is typed as `any`. TanStack AI does not ship a runtime message validator — define a schema with your preferred Standard-Schema library (Zod, Valibot, ArkType, …) and parse the body before handing it to `chat()`. + +```typescript ignore +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { z } from 'zod' + +const ContentPartSchema = z.discriminatedUnion('type', [ + z.object({ type: z.literal('text'), content: z.string() }), + z.object({ + type: z.literal('image'), + source: z.object({ type: z.enum(['url', 'data']), value: z.string() }), + }), +]) + +const MessageSchema = z.object({ + // `ModelMessage.role` is 'user' | 'assistant' | 'tool' — there is no + // 'system' role. System instructions are passed separately via the + // `systemPrompts` option on `chat()`, not as messages. + role: z.enum(['user', 'assistant', 'tool']), + content: z.union([z.string(), z.array(ContentPartSchema)]), +}) + +const BodySchema = z.object({ messages: z.array(MessageSchema) }) + +// In an API route handler +const { messages } = BodySchema.parse(await request.json()) + +const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages, +}) +``` + +The TypeScript types on `chat()` still constrain anything you append at the call site to the modalities supported by the selected model. + +## Best Practices + +1. **Use appropriate source type**: Use `data` for small content or when you need to include content inline. Use `url` for large files or when the content is already hosted. + +2. **Include metadata**: Provide relevant metadata (like `mimeType` or `detail`) to help the model process the content correctly. + +3. **Check model support**: Not all models support all modalities. Verify the model you're using supports the content types you want to send. + +4. **Handle errors gracefully**: When a model doesn't support a particular modality, it may throw an error. Handle these cases in your application. + +## Client-Side Multimodal Messages + +When using the `ChatClient` from `@tanstack/ai-client`, you can send multimodal messages directly from your UI using the `sendMessage` method. + +### Basic Usage + +The `sendMessage` method accepts either a simple string or a `MultimodalContent` object: + +```typescript +import { ChatClient, fetchServerSentEvents } from '@tanstack/ai-client' + +const client = new ChatClient({ + connection: fetchServerSentEvents('/api/chat'), +}) + +// Simple text message +await client.sendMessage('Hello!') + +// Multimodal message with image +await client.sendMessage({ + content: [ + { type: 'text', content: 'What is in this image?' }, + { + type: 'image', + source: { type: 'url', value: 'https://example.com/photo.jpg' } + } + ] +}) +``` + +### Custom Message ID + +You can provide a custom ID for the message: + +```typescript +import { ChatClient, fetchServerSentEvents } from '@tanstack/ai-client' + +const client = new ChatClient({ + connection: fetchServerSentEvents('/api/chat'), +}) + +await client.sendMessage({ + content: 'Hello!', + id: 'custom-message-id-123' +}) +``` + +### Per-Message Forwarded Props + +The second parameter allows you to pass additional `forwardedProps` for that specific request. These are shallow-merged with the client's base `forwardedProps` configuration, with per-message values taking priority: + +```typescript +import { ChatClient, fetchServerSentEvents } from '@tanstack/ai-client' + +const client = new ChatClient({ + connection: fetchServerSentEvents('/api/chat'), + forwardedProps: { model: 'gpt-5' }, // Base forwarded props +}) + +// Override model for this specific message +await client.sendMessage('Analyze this complex problem', { + model: 'gpt-5', + temperature: 0.2, +}) +``` + +> **Note:** The legacy `body` constructor option is still supported but deprecated. New code should use `forwardedProps`. Both populate the same wire field. + +### React Example + +Here's how to use multimodal messages in a React component: + +```tsx +import { useChat } from '@tanstack/ai-react' +import { fetchServerSentEvents } from '@tanstack/ai-client' +import { useState } from 'react' + +function ChatWithImages() { + const [imageUrl, setImageUrl] = useState('') + const { sendMessage, messages } = useChat({ + connection: fetchServerSentEvents('/api/chat'), + }) + + const handleSendWithImage = () => { + if (imageUrl) { + sendMessage({ + content: [ + { type: 'text', content: 'What do you see in this image?' }, + { type: 'image', source: { type: 'url', value: imageUrl } } + ] + }) + } + } + + return ( +
+ setImageUrl(e.target.value)} + /> + +
+ ) +} +``` + +### File Upload Example + +Here's how to handle file uploads and send them as multimodal content: + +```tsx +import { useChat } from '@tanstack/ai-react' +import { fetchServerSentEvents } from '@tanstack/ai-client' + +function ChatWithFileUpload() { + const { sendMessage } = useChat({ + connection: fetchServerSentEvents('/api/chat'), + }) + + const handleFileUpload = async (file: File) => { + // Convert file to base64 + const base64 = await new Promise((resolve) => { + const reader = new FileReader() + reader.onload = () => { + const result = reader.result as string + // Remove data URL prefix (e.g., "data:image/png;base64,") + resolve(result.split(',')[1]!) + } + reader.readAsDataURL(file) + }) + + // Determine content type based on file type + const type = file.type.startsWith('image/') + ? 'image' + : file.type.startsWith('audio/') + ? 'audio' + : file.type.startsWith('video/') + ? 'video' + : 'document' + + await sendMessage({ + content: [ + { type: 'text', content: `Please analyze this ${type}` }, + { + type, + source: { type: 'data', value: base64, mimeType: file.type } + } + ] + }) + } + + return ( + { + const file = e.target.files?.[0] + if (file) handleFileUpload(file) + }} + /> + ) +} +``` + diff --git a/mintlify/ai/advanced/otel.md b/mintlify/ai/advanced/otel.md new file mode 100644 index 000000000..810879b52 --- /dev/null +++ b/mintlify/ai/advanced/otel.md @@ -0,0 +1,241 @@ +--- +title: OpenTelemetry +id: otel +order: 3 +description: "Emit vendor-neutral OpenTelemetry traces and metrics from every TanStack AI chat() call, following the OTel GenAI semantic conventions." +keywords: + - tanstack ai + - opentelemetry + - otel + - observability + - tracing + - metrics + - gen_ai + - semantic conventions +--- + +The `otelMiddleware` factory wires TanStack AI into your existing OpenTelemetry setup. Every `chat()` call produces a root span, one child span per agent-loop iteration, and one grandchild span per tool call — all with [GenAI semantic-convention attributes](https://opentelemetry.io/docs/specs/semconv/gen-ai/). It also records GenAI token and duration histograms when a `Meter` is provided. + +## Setup + +Install `@opentelemetry/api` — it's an optional peer dependency of `@tanstack/ai`: + +```bash +pnpm add @opentelemetry/api +``` + +Wire up your OTel SDK however you already do (e.g. `@opentelemetry/sdk-node`). Then pass a `Tracer` (and optionally a `Meter`) into the middleware. The OTel middleware lives on its own subpath — importing it never affects users who don't need OTel: + +```ts +import { chat } from '@tanstack/ai' +import { otelMiddleware } from '@tanstack/ai/middlewares/otel' +import { openaiText } from '@tanstack/ai-openai' +import { trace, metrics } from '@opentelemetry/api' + +const otel = otelMiddleware({ + tracer: trace.getTracer('my-app'), + meter: metrics.getMeter('my-app'), +}) + +const result = await chat({ + adapter: openaiText('gpt-5.5'), + messages: [{ role: 'user', content: 'hi' }], + middleware: [otel], + stream: false, +}) +``` + +## What gets emitted + +### Spans + +```text +chat gpt-5.5 (root, kind: INTERNAL) +├── chat gpt-5.5 #0 (iteration, kind: CLIENT) +│ ├── execute_tool get_weather +│ └── execute_tool get_time +└── chat gpt-5.5 #1 (iteration, kind: CLIENT) +``` + +Iteration spans are numbered (`#0`, `#1`, ...) so distinct iterations of the same chat are easy to pick apart in trace viewers. + +### Attribute reference + +| Level | Attribute | Value | +| --- | --- | --- | +| root / iteration | `gen_ai.system` | `openai`, `anthropic`, ... | +| iteration | `gen_ai.operation.name` | `chat` | +| root / iteration | `gen_ai.request.model` | requested model | +| iteration | `gen_ai.response.model` | actual model | +| iteration | `gen_ai.request.temperature` | from config | +| iteration | `gen_ai.request.top_p` | from config | +| iteration | `gen_ai.request.max_tokens` | from config | +| iteration | `gen_ai.usage.input_tokens` | per iteration | +| iteration | `gen_ai.usage.output_tokens` | per iteration | +| root / iteration | `gen_ai.usage.total_tokens` | provider-reported total | +| root / iteration | `gen_ai.usage.cost` | provider-reported cost, when available | +| root / iteration | `gen_ai.usage.cache_read.input_tokens` | cached prompt tokens, when reported | +| root / iteration | `gen_ai.usage.cache_creation.input_tokens` | cache-write prompt tokens, when reported | +| root / iteration | `gen_ai.usage.reasoning.output_tokens` | reasoning/thinking tokens, when reported | +| root / iteration | `tanstack.ai.usage.duration_seconds` | duration-based billing (e.g. transcription), when reported | +| root / iteration | `tanstack.ai.usage.upstream_cost` | gateway upstream cost (e.g. OpenRouter), when reported | +| root / iteration | `tanstack.ai.usage.upstream_input_cost` | upstream input cost split, when reported | +| root / iteration | `tanstack.ai.usage.upstream_output_cost` | upstream output cost split, when reported | +| iteration | `gen_ai.response.finish_reasons` | `[stop]`, `[tool_calls]`, ... | +| root | `gen_ai.usage.input_tokens` | rolled up | +| root | `gen_ai.usage.output_tokens` | rolled up | +| root | `tanstack.ai.iterations` | iteration count | +| tool | `gen_ai.tool.name` | tool name | +| tool | `gen_ai.tool.call.id` | tool call id | +| tool | `gen_ai.tool.type` | `function` | +| tool | `tanstack.ai.tool.outcome` | `success` / `error` | + +Usage attributes beyond input/output tokens are emitted only when the provider reports them, so spans stay clean otherwise. Cache and reasoning breakdowns use the official GenAI semconv names; `gen_ai.usage.cost` and `gen_ai.usage.total_tokens` are de-facto extensions consumed directly by backends like PostHog — without them, backends re-derive cost from their own price tables and lose cache discounts and gateway markup. Fields with no established convention (duration-based billing, the upstream cost split) are TanStack-namespaced. + +### Metrics + +Two GenAI-standard histograms: + +- `gen_ai.client.operation.duration` (seconds) — recorded **once per `chat()` call**, covering all agent-loop iterations and tool execution. On error or abort the record carries an `error.type` attribute (the thrown error's `name`, or `"cancelled"` for aborts). +- `gen_ai.client.token.usage` (tokens) — recorded **once per iteration** (two records: input and output), tagged with `gen_ai.token.type`. + +Both `gen_ai.response.id` and `gen_ai.response.model` are deliberately excluded from metric attributes to keep cardinality low (per-request custom-model names and request IDs would blow up the series set). + +## Privacy: capturing prompts and completions + +By default, only metadata lands on spans. To record prompt and completion content, set `captureContent: true`. Content is captured as OTel span events following the GenAI convention: + +- `gen_ai.user.message`, `gen_ai.system.message`, `gen_ai.assistant.message`, `gen_ai.tool.message`, `gen_ai.choice` + +Pass a `redact` function to strip PII before anything is recorded: + +```ts +import { otelMiddleware } from '@tanstack/ai/middlewares/otel' +import { trace } from '@opentelemetry/api' + +const tracer = trace.getTracer('my-app') + +otelMiddleware({ + tracer, + captureContent: true, + redact: (text) => text.replace(/\b\d{3}-\d{2}-\d{4}\b/g, '[SSN]'), +}) +``` + +If `redact` throws, the middleware writes the literal sentinel `"[redaction_failed]"` into the span event and logs a warning — it never falls back to the raw content. This is the load-bearing invariant for users who ship traces to third-party backends: a broken redactor should shut off capture, not leak prompts. + +Accumulated assistant text (the `gen_ai.choice` event) is capped at `maxContentLength` characters (default `100 000`); longer completions are truncated with a trailing `"…"` marker. + +Multimodal content (images, audio, video, documents) is represented as placeholder strings (`[image]`, `[audio]`, ...) to preserve message order without dumping binary data onto spans. Use `onSpanEnd` if you need richer multimodal capture. + +Prompt/system/user message events fire from `onConfig` at the start of every iteration, which means the full conversation history (as the adapter will re-send it) is re-emitted on each iteration span. This mirrors what the provider actually sees on the wire. + +## Extension points + +All four extensions are optional. Each wraps user code in try/catch — a thrown callback becomes a log line, never a broken chat. + +### `spanNameFormatter(info)` + +Override default span names. `info.kind` is `'chat' | 'iteration' | 'tool'`. + +```ts +import { otelMiddleware } from '@tanstack/ai/middlewares/otel' +import { trace } from '@opentelemetry/api' + +const tracer = trace.getTracer('my-app') + +otelMiddleware({ + tracer, + spanNameFormatter: (info) => + info.kind === 'tool' ? `tool:${info.toolName}` : `chat:${info.ctx.model}`, +}) +``` + +### `attributeEnricher(info)` + +Add custom attributes to every span. Fires once per span. + +```ts +import { otelMiddleware } from '@tanstack/ai/middlewares/otel' +import { trace } from '@opentelemetry/api' +import { getCurrentTenant } from './context' + +const tracer = trace.getTracer('my-app') + +otelMiddleware({ + tracer, + attributeEnricher: () => ({ + 'tenant.id': getCurrentTenant(), + }), +}) +``` + +### `onBeforeSpanStart(info, options)` + +Mutate `SpanOptions` immediately before `tracer.startSpan(...)`. Useful for adding links, custom start times, or extra default attributes. + +### `onSpanEnd(info, span)` + +Fires just before every `span.end()`. Common uses: record custom events, emit per-tool metrics via your own `Meter`. + +```ts +import { otelMiddleware } from '@tanstack/ai/middlewares/otel' +import { trace, metrics } from '@opentelemetry/api' + +const tracer = trace.getTracer('my-app') +const meter = metrics.getMeter('my-app') + +const toolDuration = meter.createHistogram('tool.duration') +otelMiddleware({ + tracer, + onSpanEnd: (info, span) => { + if (info.kind === 'tool') { + // span is still recording; read timestamps from your own store if needed + toolDuration.record(1, { 'tool.name': info.toolName }) + } + }, +}) +``` + +## Beyond chat: media activities + +`otelMiddleware` is not chat-only. The media activities — `generateImage`, `generateVideo`, `generateAudio`, `generateSpeech`, and `generateTranscription` — accept the **same** `otelMiddleware` value on their `middleware` option. Each is a single request → response (or submit → poll for video), so the middleware emits one span per call instead of the chat span tree: + +```ts +import { generateImage } from '@tanstack/ai' +import { otelMiddleware } from '@tanstack/ai/middlewares/otel' +import { openaiImage } from '@tanstack/ai-openai' +import { trace, metrics } from '@opentelemetry/api' + +const otel = otelMiddleware({ + tracer: trace.getTracer('my-app'), + meter: metrics.getMeter('my-app'), +}) + +const result = await generateImage({ + adapter: openaiImage('gpt-image-2'), + prompt: 'A serene mountain landscape at sunset', + middleware: [otel], +}) +``` + +The same `otel` value can be passed to `chat()` and to any media activity — its shared lifecycle hooks (`onStart` / `onUsage` / `onFinish` / `onAbort` / `onError`) are authored against the activity-agnostic `GenerationMiddlewareContext`, so the one instance works everywhere. + +Each media call produces one `CLIENT` span tagged with the activity's `gen_ai.operation.name`: + +| Activity | `gen_ai.operation.name` | +| --- | --- | +| `generateImage` | `image_generation` | +| `generateVideo` | `video_generation` | +| `generateAudio` | `audio_generation` | +| `generateSpeech` | `text_to_speech` | +| `generateTranscription` | `transcription` | + +The span carries `gen_ai.system` and `gen_ai.request.model` at start and, on finish, the same `gen_ai.usage.*` / `tanstack.ai.usage.*` attributes documented above — including `tanstack.ai.usage.units_billed` for unit-billed media. When a `Meter` is supplied it records the `gen_ai.client.operation.duration` histogram, tagged per activity. For streaming video the span covers the full create → poll → complete lifecycle; for non-streaming `generateVideo` it covers job submission. If a streaming video consumer abandons the stream before completion, the span is ended via `onAbort` (status `ERROR`, `tanstack.ai.completion.reason = cancelled`) rather than leaked. + +`otelMiddleware` applies the same `spanNameFormatter`, `attributeEnricher`, `onBeforeSpanStart`, and `onSpanEnd` extension points to media spans — the span info is discriminated by `kind`, where media spans report `kind: 'generation'`. For a custom backend, implement the base `GenerationMiddleware` contract directly; its hooks (`onStart` / `onUsage` / `onFinish` / `onAbort` / `onError`) receive the `GenerationMiddlewareContext` and fire for every activity, chat included. The `GenerationMiddleware` types are exported from the package root, while the `otelMiddleware` value lives on the `@tanstack/ai/middlewares/otel` subpath so importing `@tanstack/ai` never requires the optional `@opentelemetry/api` peer. + +## Related + +- [Middleware](/ai/advanced/middleware) — the lifecycle this middleware hooks into +- [Debug Logging](/ai/advanced/debug-logging) — quick console-output diagnostics, complementary to OTel \ No newline at end of file diff --git a/mintlify/ai/advanced/per-model-type-safety.md b/mintlify/ai/advanced/per-model-type-safety.md new file mode 100644 index 000000000..bd9d3c00e --- /dev/null +++ b/mintlify/ai/advanced/per-model-type-safety.md @@ -0,0 +1,82 @@ +--- +title: Per-Model Type Safety +id: per-model-type-safety +order: 5 +description: "TanStack AI narrows modelOptions and content types to the specific model you select, enforcing capabilities at compile time." +keywords: + - tanstack ai + - type safety + - per-model types + - modelOptions + - typescript + - autocomplete + - compile-time +--- + +The AI SDK provides **model-specific type safety** for `modelOptions`. Each model's capabilities determine which model options are allowed, and TypeScript will enforce this at compile time. + +> **Tip:** For structured outputs, most users should prefer the first-class `chat({ outputSchema })` option over the raw provider `text` option shown below — it works across providers and validates the result for you. The raw `text` option is for when you need provider-specific control. + +## How It Works + +Each adapter factory captures the model literal as a type parameter — `openaiText(model)` — so the adapter carries the exact model you selected at the type level. + +The `modelOptions` you pass are then resolved against a per-model map (`ResolveProviderOptions`). Each model's entry declares only the options that model actually supports. A model without a structured-output capability simply has no `text` property in its resolved options type, so TypeScript's excess-property checking rejects `text` for that model — at compile time, with zero runtime cost. + +This is the same mechanism described in [Typed Pre-Configured Options](/ai/advanced/typed-options) (which captures these resolved options in a reusable object) and [Extend Adapter](/ai/advanced/extend-adapter) (which lets you attach the same typed `modelOptions` to custom models). + +## Usage Examples + +### ✅ Correct Usage + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +// ✅ gpt-5 supports structured outputs - `text` is allowed +const validCall = chat({ + adapter: openaiText("gpt-5"), + messages: [], + modelOptions: { + // OK - text is included for gpt-5 + text: { + format: { + type: "json_schema", + name: "my_schema", + schema: { + /* JSON Schema object */ + }, + }, + }, + }, +}); +``` + +### ❌ Incorrect Usage + +```typescript ignore +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +// ❌ gpt-4-turbo does NOT support structured outputs - `text` is rejected +const invalidCall = chat({ + adapter: openaiText("gpt-4-turbo"), + messages: [], + modelOptions: { + text: {}, // ❌ TypeScript error: 'text' does not exist in type + }, +}); +``` + +TypeScript will produce: + +``` +error TS2353: Object literal may only specify known properties, and 'text' does not exist in type ...'. +``` + +## Benefits + +- **Compile-time safety**: Catch incorrect model options before deployment +- **Better IDE experience**: Autocomplete shows only valid options for each model +- **Self-documenting**: Model capabilities are explicit in the type system +- **Zero runtime overhead**: All type checking happens at compile time diff --git a/mintlify/ai/advanced/runtime-adapter-switching.md b/mintlify/ai/advanced/runtime-adapter-switching.md new file mode 100644 index 000000000..cc63a2e2c --- /dev/null +++ b/mintlify/ai/advanced/runtime-adapter-switching.md @@ -0,0 +1,239 @@ +--- +title: Runtime Adapter Switching +id: runtime-adapter-switching +order: 6 +description: "Let users switch between LLM providers at runtime in TanStack AI while keeping full TypeScript type safety for each adapter's model options." +keywords: + - tanstack ai + - runtime switching + - multi-provider + - adapter factory + - type safety + - dynamic adapter +--- + +# Runtime Adapter Switching with Type Safety + +Learn how to build interfaces where users can switch between LLM providers at runtime while maintaining full TypeScript type safety. + +## The Simple Approach + +With TanStack AI, the model is passed directly to the adapter factory function. This gives you full type safety and autocomplete at the point of definition: + +```typescript +import { chat, toServerSentEventsResponse } from '@tanstack/ai' +import { anthropicText } from '@tanstack/ai-anthropic' +import { openaiText } from '@tanstack/ai-openai' + +type Provider = 'openai' | 'anthropic' + +// Define adapters with their models - autocomplete works here! +const adapters = { + anthropic: () => anthropicText('claude-sonnet-4-6'), // ✅ Autocomplete! + openai: () => openaiText('gpt-5.5'), // ✅ Autocomplete! +} + +async function handleRequest(request: Request) { + // In your request handler: + const body = await request.json() + const provider: Provider = body.forwardedProps?.provider || 'openai' + + const stream = chat({ + adapter: adapters[provider](), + messages: body.messages, + }) +} +``` + +## Why This Works + +Each adapter factory function accepts a model name as its first argument and returns a fully typed adapter: + +```typescript +import { openaiText, OpenAITextAdapter } from '@tanstack/ai-openai' + +// These are equivalent: +const adapter1 = openaiText('gpt-5.5') +const adapter2 = new OpenAITextAdapter({ apiKey: process.env.OPENAI_API_KEY! }, 'gpt-5.5') + +// The model is stored on the adapter +console.log(adapter1.model) // 'gpt-5.5' +``` + +When you pass an adapter to `chat()`, it uses the model from `adapter.model`. This means: + +- **Full autocomplete** - When typing the model name, TypeScript knows valid options +- **Type validation** - Invalid model names cause compile errors +- **Clean code** - No separate `model` parameter needed + +## Full Example + +Here's a complete example showing a multi-provider chat API: + +```typescript ignore +import { createFileRoute } from '@tanstack/react-router' +import { chat, maxIterations, toServerSentEventsResponse } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { anthropicText } from '@tanstack/ai-anthropic' +import { geminiText } from '@tanstack/ai-gemini' +import { ollamaText } from '@tanstack/ai-ollama' + +type Provider = 'openai' | 'anthropic' | 'gemini' | 'ollama' + +// Define adapters with their models +const adapters = { + anthropic: () => anthropicText('claude-sonnet-4-6'), + gemini: () => geminiText('gemini-3-flash-preview'), + ollama: () => ollamaText('mistral:7b'), + openai: () => openaiText('gpt-5.5'), +} + +export const Route = createFileRoute('/api/chat')({ + server: { + handlers: { + POST: async ({ request }) => { + const abortController = new AbortController() + const body = await request.json() + // `forwardedProps` is the AG-UI field set by `useChat({ forwardedProps })`. + // The legacy `body.data.provider` access still works (mirrored on the + // wire for backward compatibility) but `forwardedProps` is preferred. + const provider: Provider = body.forwardedProps?.provider || 'openai' + + const stream = chat({ + adapter: adapters[provider](), + tools: [...], + systemPrompts: [...], + messages: body.messages, + abortController, + }) + + return toServerSentEventsResponse(stream, { abortController }) + }, + }, + }, +}) +``` + +## Using with Image Adapters + +The same pattern works for image generation: + +```typescript +import { generateImage } from '@tanstack/ai' +import { openaiImage } from '@tanstack/ai-openai' +import { geminiImage } from '@tanstack/ai-gemini' + +type ImageProvider = 'openai' | 'gemini' + +const imageAdapters: Record ReturnType> = { + openai: () => openaiImage('gpt-image-1'), + gemini: () => geminiImage('gemini-3.1-flash-image-preview'), +} + +export async function POST(request: Request) { + const body = await request.json() + const provider: ImageProvider = body.provider ?? 'openai' + + const result = await generateImage({ + adapter: imageAdapters[provider](), + prompt: 'A beautiful sunset over mountains', + size: '1024x1024', + }) + + return Response.json(result) +} +``` + +## Using with Summarize Adapters + +And for summarization: + +```typescript +import { summarize } from '@tanstack/ai' +import { openaiSummarize } from '@tanstack/ai-openai' +import { anthropicSummarize } from '@tanstack/ai-anthropic' + +type SummarizeProvider = 'openai' | 'anthropic' + +const summarizeAdapters: Record ReturnType> = { + openai: () => openaiSummarize('gpt-5.4-mini'), + anthropic: () => anthropicSummarize('claude-sonnet-4-6'), +} + +export async function POST(request: Request) { + const body = await request.json() + const provider: SummarizeProvider = body.provider ?? 'openai' + const longDocument: string = body.text + + const result = await summarize({ + adapter: summarizeAdapters[provider](), + text: longDocument, + maxLength: 100, + style: 'concise', + }) + + return Response.json(result) +} +``` + +## Migration from Switch Statements + +If you have existing code using switch statements, here's how to migrate: + +### Before + +```typescript ignore +let adapter +let model + +switch (provider) { + case 'anthropic': + adapter = anthropicText() + model = 'claude-sonnet-4-6' + break + case 'openai': + default: + adapter = openaiText() + model = 'gpt-5.5' + break +} + +const stream = chat({ + adapter: adapter as any, + model: model as any, + messages, +}) +``` + +### After + +```typescript +import { chat, toServerSentEventsResponse } from '@tanstack/ai' +import { anthropicText } from '@tanstack/ai-anthropic' +import { openaiText } from '@tanstack/ai-openai' + +type AfterProvider = 'openai' | 'anthropic' + +const adapters = { + anthropic: () => anthropicText('claude-sonnet-4-6'), + openai: () => openaiText('gpt-5.5'), +} + +export async function POST(request: Request) { + const body = await request.json() + const provider: AfterProvider = body.forwardedProps?.provider ?? 'openai' + + const stream = chat({ + adapter: adapters[provider](), + messages: body.messages, + }) + + return toServerSentEventsResponse(stream) +} +``` + +The key changes: + +1. Replace the switch statement with an object of factory functions +2. Each factory function creates an adapter with the model included +3. No more `as any` casts - full type safety! diff --git a/mintlify/ai/advanced/runtime-context.md b/mintlify/ai/advanced/runtime-context.md new file mode 100644 index 000000000..18c703fb7 --- /dev/null +++ b/mintlify/ai/advanced/runtime-context.md @@ -0,0 +1,319 @@ +--- +title: Runtime Context +id: runtime-context +order: 2 +description: "Pass typed runtime dependencies to TanStack AI tools and middleware without serializing them to the model or AG-UI protocol context." +keywords: + - tanstack ai + - runtime context + - typed context + - tools context + - middleware context + - ag-ui context +--- + +Runtime context is application state you pass to tool implementations and middleware. Use it for request-scoped or client-local dependencies such as authenticated users, database clients, tenancy, feature flags, audit loggers, or browser services. + +Runtime context is not prompt context and is not the AG-UI `RunAgentInput.context` field. It is never sent to the model automatically. + +## How Type Safety Works + +Runtime context is checked from the point of view of the code that consumes it. Tools and middleware declare the context shape they need, and `chat()`, `ChatClient`, and framework hooks check that the `context` value you pass satisfies those requirements. + +The source of truth is: + +- `toolDefinition(...).server(...)` for server tools. +- `toolDefinition(...).client(...)` for client tools. +- `ChatMiddleware` for middleware. + +This means the context value is the implementation detail you provide at runtime, while tools and middleware are the contract. TanStack AI infers the required context from every typed tool and middleware in the call, merges those requirements, and checks your `context` option against the result. + +```typescript +import { chat, toServerSentEventsResponse, toolDefinition, type ChatMiddleware } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +type UserContext = { + userId: string; +}; + +type TenantContext = { + tenantId: string; +}; + +const currentUserTool = toolDefinition({ + name: "current_user", + description: "Read the current user", +}).server((_input, ctx) => { + return { userId: ctx.context.userId }; +}); + +const tenantMiddleware: ChatMiddleware = { + name: "tenant", + onStart(ctx) { + console.log(ctx.context.tenantId); + }, +}; + +export async function POST(request: Request) { + const { messages } = await request.json(); + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages, + tools: [currentUserTool], + middleware: [tenantMiddleware], + context: { + userId: "user_123", + tenantId: "tenant_456", + }, + }); + return toServerSentEventsResponse(stream); +} +``` + +In this example, the tool requires `UserContext` and the middleware requires `TenantContext`, so the `context` value must satisfy both. If you remove `tenantId`, TypeScript reports an error because `tenantMiddleware` declared that it needs it. + +This is intentional. The `context` object alone should not decide what tools and middleware are allowed to read. The consumers define their requirements, and the call site proves that it supplied them. Untyped tools and middleware still work; they receive `unknown` context and do not force a `context` option. + +This inference also works when reusable tools or middleware are declared outside the `chat()` call and passed in as arrays. A consumer can opt into optional runtime context by declaring `TContext | undefined`; then the `context` option can be omitted when all typed consumers accept `undefined`. If a context value is provided, it still has to satisfy every typed consumer. + +The same rule applies on the client: + +```typescript +import { clientTools } from "@tanstack/ai-client"; +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; +import { toolDefinition } from "@tanstack/ai"; + +type ClientRuntimeContext = { + currentTabId: string; +}; + +const inspectClientContext = toolDefinition({ + name: "inspect_client_context", + description: "Inspect local browser context", +}).client((_input, ctx) => { + return { + tabId: ctx.context.currentTabId, + mode: ctx.context.mode, + }; +}); + +useChat({ + connection: fetchServerSentEvents("/api/chat"), + tools: clientTools(inspectClientContext), + context: { + currentTabId: "settings", + mode: "debug", + }, +}); +``` + +Because the client tool declares `ClientRuntimeContext & { mode: "debug" }`, `useChat()` requires a `context` value with both `currentTabId` and the literal `mode: "debug"`. + +## Server Runtime Context + +Define the context type once, use it in server tools and middleware, then pass the matching `context` value to `chat()`. + +```typescript +import { + chat, + toServerSentEventsResponse, + toolDefinition, + type ChatMiddleware, +} from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { z } from "zod"; +import { requireUser, db } from "./auth"; + +type AppContext = { + userId: string; + tenantId: string; + db: { + notes: { + findMany(args: { userId: string; tenantId: string }): Promise>; + }; + }; +}; + +const listNotes = toolDefinition({ + name: "list_notes", + description: "List notes for the current user", + inputSchema: z.object({}), + outputSchema: z.array(z.object({ title: z.string() })), +}).server(async (_input, ctx) => { + return ctx.context.db.notes.findMany({ + userId: ctx.context.userId, + tenantId: ctx.context.tenantId, + }); +}); + +const auditMiddleware: ChatMiddleware = { + name: "audit", + onStart(ctx) { + console.log("chat started", { + requestId: ctx.requestId, + userId: ctx.context.userId, + tenantId: ctx.context.tenantId, + }); + }, +}; + +export async function POST(request: Request) { + const { messages } = await request.json(); + const user = await requireUser(request); + + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages, + tools: [listNotes], + middleware: [auditMiddleware], + context: { + userId: user.id, + tenantId: user.tenantId, + db, + }, + }); + + return toServerSentEventsResponse(stream); +} +``` + +When any tool or middleware in a `chat()` call declares a concrete context type, TypeScript checks the `context` value against that type. Existing untyped tools and middleware continue to work; their `ctx.context` type remains `unknown`. + +## Client Runtime Context + +Client runtime context is local to `ChatClient` and framework hooks. It is passed to client tool implementations and is not serialized to the server. + +```typescript +import { createChatClientOptions, clientTools } from "@tanstack/ai-client"; +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; +import { toolDefinition } from "@tanstack/ai"; + +type ClientContext = { + currentTabId: string; + toast(message: string): void; +}; + +const notifyUser = toolDefinition({ + name: "notify_user", + description: "Show a notification in the current browser tab", +}).client((_input, ctx) => { + ctx.context.toast(`Updated tab ${ctx.context.currentTabId}`); + return { ok: true }; +}); + +const chatOptions = createChatClientOptions({ + connection: fetchServerSentEvents("/api/chat"), + tools: clientTools(notifyUser), + context: { + currentTabId: "settings", + toast: (message) => window.alert(message), + }, +}); + +const chat = useChat(chatOptions); +``` + +Use client context for local dependencies only. Do not put values there expecting the server to receive them. + +## Client-to-Server Handoff + +To send serializable client data to the server, use `forwardedProps`, validate it in your route, and explicitly map it into the server runtime context. + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; +import { clientTools } from "@tanstack/ai-client"; +import { toolDefinition } from "@tanstack/ai"; + +type ClientContext = { + currentTabId: string; + toast(message: string): void; +}; + +const notifyUser = toolDefinition({ + name: "notify_user", + description: "Show a notification in the current browser tab", +}).client((_input, ctx) => { + ctx.context.toast(`Updated tab ${ctx.context.currentTabId}`); + return { ok: true }; +}); + +// Client +useChat({ + connection: fetchServerSentEvents("/api/chat"), + tools: clientTools(notifyUser), + forwardedProps: { + tenantId: "tenant_456", + }, + context: { + currentTabId: "settings", + toast: (message) => window.alert(message), + }, +}); +``` + +```typescript +// Server +import { + chat, + chatParamsFromRequest, + toServerSentEventsResponse, +} from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { requireUser } from "./auth"; +import { serverTools } from "./tools"; + +type AppContext = { + userId: string; + tenantId: string; +}; + +export async function POST(request: Request) { + const params = await chatParamsFromRequest(request); + const user = await requireUser(request); + + const tenantId = + typeof params.forwardedProps.tenantId === "string" + ? params.forwardedProps.tenantId + : user.defaultTenantId; + + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: params.messages, + tools: serverTools, + context: { + userId: user.id, + tenantId, + } satisfies AppContext, + }); + + return toServerSentEventsResponse(stream); +} +``` + +Treat `forwardedProps` as client-controlled input. Validate and allowlist every field before using it to build server runtime context. + +## AG-UI Context + +AG-UI also defines `RunAgentInput.context`, usually as protocol-level context entries for interoperable agents. TanStack AI surfaces that field through `chatParamsFromRequest`, but it is separate from `chat({ context })`. + +TanStack AI does not automatically copy AG-UI `params.aguiContext` into runtime context. If you want to use AG-UI context values, validate and map them yourself. `params.context` is a deprecated alias of `params.aguiContext` kept for backward compatibility. + +```typescript +import { chat, chatParamsFromRequest, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { buildRuntimeContextFrom } from "./context"; +import { serverTools } from "./tools"; + +export async function POST(request: Request) { + const params = await chatParamsFromRequest(request); + + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: params.messages, + tools: serverTools, + context: buildRuntimeContextFrom(params.aguiContext), + }); + + return toServerSentEventsResponse(stream); +} +``` diff --git a/mintlify/ai/advanced/tree-shaking.md b/mintlify/ai/advanced/tree-shaking.md new file mode 100644 index 000000000..49328e199 --- /dev/null +++ b/mintlify/ai/advanced/tree-shaking.md @@ -0,0 +1,300 @@ +--- +title: Tree-Shaking +id: tree-shaking +order: 7 +description: "TanStack AI's tree-shakeable architecture — import only the activities and adapters you use for minimal bundle size across chat, image, and speech." +keywords: + - tanstack ai + - tree-shaking + - bundle size + - modular imports + - performance + - tree-shakeable +--- + +# Tree-Shaking & Bundle Optimization + +TanStack AI is designed from the ground up for maximum tree-shakeability. The entire system—from activity functions to adapters—uses a functional, modular architecture that ensures you only bundle the code you actually use. + +## Design Philosophy + +Instead of a monolithic API that includes everything, TanStack AI provides: + +- **Individual activity functions** - Import only the activities you need (`chat`, `summarize`, etc.) +- **Individual adapter functions** - Import only the adapters you need (`openaiText`, `openaiSummarize`, etc.) +- **Functional API design** - Pure functions that can be easily eliminated by bundlers +- **Separate modules** - Each activity and adapter lives in its own module + +This design means that if you only use `chat` with OpenAI, you won't bundle code for summarization, image generation, or other providers. + +## Activity Functions + +Each AI activity is exported as a separate function from `@tanstack/ai`: + +```ts +// Import only the activities you need +import { chat } from '@tanstack/ai' // Chat/text generation +import { summarize } from '@tanstack/ai' // Summarization +import { generateImage } from '@tanstack/ai' // Image generation +import { generateSpeech } from '@tanstack/ai' // Text-to-speech +import { generateTranscription } from '@tanstack/ai' // Audio transcription +import { generateVideo } from '@tanstack/ai' // Video generation +``` + +### Example: Chat Only + +If you only need chat functionality: + +```ts +// Only chat code is bundled +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages: [{ role: 'user', content: 'Hello!' }], +}) +``` + +Your bundle will **not** include: +- Summarization logic +- Image generation logic +- Other activity implementations + +## Adapter Functions + +Each provider package exports individual adapter functions for each activity type: + +### OpenAI + +```ts +import { + openaiText, // Chat/text generation + openaiSummarize, // Summarization + openaiImage, // Image generation + openaiSpeech, // Text-to-speech + openaiTranscription, // Audio transcription + openaiVideo, // Video generation +} from '@tanstack/ai-openai' +``` + +### Anthropic + +```ts +import { + anthropicText, // Chat/text generation + anthropicSummarize, // Summarization +} from '@tanstack/ai-anthropic' +``` + +### Gemini + +```ts +import { + geminiText, // Chat/text generation + geminiSummarize, // Summarization + geminiImage, // Image generation + geminiSpeech, // Text-to-speech (experimental) +} from '@tanstack/ai-gemini' +``` + +### Ollama + +```ts +import { + ollamaText, // Chat/text generation + ollamaSummarize, // Summarization +} from '@tanstack/ai-ollama' +``` + +## Complete Example + +Here's how the tree-shakeable design works in practice: + +```ts +// Only import what you need +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +// Chat generation - returns AsyncIterable +const chatResult = chat({ + adapter: openaiText('gpt-5.5'), + messages: [{ role: 'user', content: 'Hello!' }], +}) + +for await (const chunk of chatResult) { + console.log(chunk) +} +``` + +**What gets bundled:** +- ✅ `chat` function and its dependencies +- ✅ `openaiText` adapter and its dependencies +- ✅ Chat-specific streaming and tool handling logic + +**What doesn't get bundled:** +- ❌ `summarize` function +- ❌ `generateImage` function +- ❌ Other adapter implementations (Anthropic, Gemini, etc.) +- ❌ Other activity implementations + +## Using Multiple Activities + +If you need multiple activities, import only what you use: + +```ts +import { chat, summarize } from '@tanstack/ai' +import { + openaiText, + openaiSummarize +} from '@tanstack/ai-openai' + +// Each activity is independent +const chatResult = chat({ + adapter: openaiText('gpt-5.5'), + messages: [{ role: 'user', content: 'Hello!' }], +}) + +const summarizeResult = await summarize({ + adapter: openaiSummarize('gpt-5.4-mini'), + text: 'Long text to summarize...', +}) +``` + +Each activity is in its own module, so bundlers can eliminate unused ones. + +## Type Safety + +The tree-shakeable design doesn't sacrifice type safety. Each adapter provides full type safety for its supported models: + +```ts ignore +import { openaiText, type OpenAIChatModel } from '@tanstack/ai-openai' + +const adapter = openaiText('gpt-5.5') + +// TypeScript knows the exact models supported +const model: OpenAIChatModel = 'gpt-5.5' // ✓ Valid +const model2: OpenAIChatModel = 'invalid' // ✗ Type error +``` + +## Create Options Functions + +The `create___Options` functions are also tree-shakeable: + +```ts +import { + createChatOptions, + createImageOptions +} from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +// Only import what you need +const chatOptions = createChatOptions({ + adapter: openaiText('gpt-5.5'), +}) +``` + +## Bundle Size Benefits + +The functional, modular design provides significant bundle size benefits: + +### Importing Everything (Less Efficient) + +```ts +// ❌ Importing more than needed +import * as ai from '@tanstack/ai' +import * as openai from '@tanstack/ai-openai' + +// This bundles all exports from both packages +``` + +### Importing Only What You Need (Recommended) + +```ts +// ✅ Only what you use gets bundled +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +// You only get: +// - Chat activity implementation +// - OpenAI text adapter +// - Chat-specific dependencies +``` + +### Real-World Impact + +For a typical chat application, importing a single activity and one adapter pulls in substantially less code than bundling every activity and every provider adapter. Because each activity and adapter lives in its own side-effect-free module, your bundler drops everything you don't reference — so the more providers and activities the library supports, the larger the difference between a focused import and a namespace import. + +## How It Works + +The tree-shakeability is achieved through: + +1. **ES Module exports** - Each function is a named export, not a default export +2. **Separate modules** - Each activity and adapter lives in its own file +3. **No side effects** - Functions are pure and don't have module-level side effects +4. **Functional composition** - Functions compose together, allowing dead code elimination +5. **Type-only imports** - Type imports are stripped at build time + +Modern bundlers (Vite, Webpack, Rollup, esbuild) can easily eliminate unused code because: + +- Functions are statically analyzable +- No dynamic imports of unused code +- No module-level side effects +- Clear dependency graphs + +## Best Practices + +1. **Import only what you need** - Don't import entire namespaces +2. **Use specific adapter functions** - Import `openaiText` not `openai` +3. **Separate activities by route** - Different API routes can use different activities +4. **Lazy load when possible** - Use dynamic imports for code-split routes +5. **Keep mobile chat bundles client-only** - React Native and Expo chat screens + should import `useChat` and chat connection adapters, not provider SDKs, + server response helpers, React DOM UI, devtools UI, or other framework + packages. See [Quick Start: React Native](/ai/getting-started/quick-start-react-native) + for the server-only provider boundary and mobile transport setup. + +```ts +// ✅ Good - Only imports chat +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +// ❌ Bad - Imports everything +import * as ai from '@tanstack/ai' +import * as openai from '@tanstack/ai-openai' +``` + +## Adapter Types + +Each adapter type implements a specific interface: + +- `ChatAdapter` - Provides `chatStream()` method for streaming chat responses +- `SummarizeAdapter` - Provides `summarize()` method for text summarization +- `ImageAdapter` - Provides `generateImage()` method for image generation +- `TTSAdapter` - Provides `generateSpeech()` method for text-to-speech +- `TranscriptionAdapter` - Provides `generateTranscription()` method for audio transcription +- `VideoAdapter` - Provides `generateVideo()` method for video generation + +All adapters have a `kind` property that indicates their type: + +```ts +import { openaiText, openaiSummarize } from '@tanstack/ai-openai' + +const chatAdapter = openaiText('gpt-5.5') +console.log(chatAdapter.kind) // 'text' + +const summarizeAdapter = openaiSummarize('gpt-5.4-mini') +console.log(summarizeAdapter.kind) // 'summarize' +``` + +## Summary + +TanStack AI's tree-shakeable design means: + +- ✅ **Smaller bundles** - Only include code you actually use +- ✅ **Faster load times** - Less JavaScript to download and parse +- ✅ **Better performance** - Less code means faster execution +- ✅ **Type safety** - Full TypeScript support without runtime overhead +- ✅ **Flexibility** - Mix and match activities and adapters as needed + +The functional, modular architecture ensures that modern bundlers can eliminate unused code effectively, resulting in optimal bundle sizes for your application. diff --git a/mintlify/ai/advanced/typed-options.md b/mintlify/ai/advanced/typed-options.md new file mode 100644 index 000000000..b9e85fd83 --- /dev/null +++ b/mintlify/ai/advanced/typed-options.md @@ -0,0 +1,163 @@ +--- +title: Typed Pre-Configured Options +id: typed-options +order: 11 +description: "Define typed, reusable option objects for chat, summarize, image, video, audio, speech, and transcription with createChatOptions and friends — share configuration across routes without losing per-model type safety." +keywords: + - tanstack ai + - createChatOptions + - createSummarizeOptions + - createImageOptions + - createSpeechOptions + - createTranscriptionOptions + - createAudioOptions + - createVideoOptions + - typed options + - shared configuration +--- + +You have a `chat()` (or `generateImage()`, `generateSpeech()`, …) configuration you want to reuse — across multiple routes, between a server function and its caller, or simply factored out of a handler for clarity. By the end of this guide, you'll have a single typed options object that infers the adapter's model, modalities, and provider options, and that you can spread into any call site without losing type safety. + +## The pattern + +Every activity in `@tanstack/ai` ships a paired `createXxxOptions` helper that takes the exact same options object as the activity itself and returns it unchanged — at runtime it's the identity function. The point is **type inference**: the returned object carries the adapter's full type, so when you spread it into the activity, TypeScript still narrows `modelOptions`, content modalities, and `outputSchema` to the adapter you chose. + +```typescript +import { chat, createChatOptions } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +const chatOptions = createChatOptions({ + adapter: openaiText('gpt-5.5'), + // modelOptions, systemPrompts, tools — all type-checked against the + // adapter+model pair above. Sampling params (temperature, top_p, + // max_output_tokens, …) live inside modelOptions, under each provider's + // native key. + modelOptions: { + temperature: 0.3, + reasoning: { effort: 'medium' }, + }, +}) + +// Later, anywhere in your codebase: +const stream = chat({ ...chatOptions, messages: [{ role: 'user', content: 'Hello' }] }) +``` + +Without the helper you'd have to either inline the configuration at every call site, or hand-write the full chat options type with its adapter/model generics resolved manually — `createChatOptions` does that for you. + +## When to reach for it + +- **Sharing a configuration across multiple routes** — define once, spread into each handler. +- **Passing options through a layer** (a server function, a wrapper, a test fixture) without erasing the adapter's model-specific types. +- **Branching on a runtime value while keeping types intact** — build different options objects and choose between them, instead of weaving conditionals into a single `chat({...})` call. +- **Co-locating tools, system prompts, and middleware** with the adapter they target. + +If you only call an activity once at one site, you don't need this helper. Inline the options. + +## Available helpers + +Each helper mirrors the activity it pairs with. Same options, same return type. + +| Helper | Activity | Adapter | +|---|---|---| +| `createChatOptions` | `chat()` | text adapter (e.g. `openaiText`, `anthropicText`) | +| `createSummarizeOptions` | `summarize()` | summarize adapter (e.g. `openaiSummarize`) | +| `createImageOptions` | `generateImage()` | image adapter (e.g. `openaiImage`, `falImage`) | +| `createAudioOptions` | `generateAudio()` | audio adapter (e.g. `falAudio`, `geminiAudio`) | +| `createVideoOptions` | `generateVideo()` / `getVideoJobStatus()` | video adapter (e.g. `falVideo`, `openaiVideo`) | +| `createSpeechOptions` | `generateSpeech()` | speech adapter (e.g. `openaiSpeech`, `elevenlabsSpeech`) | +| `createTranscriptionOptions` | `generateTranscription()` | transcription adapter (e.g. `openaiTranscription`, `falTranscription`) | + +All helpers are exported from `@tanstack/ai`. + +## Example: shared chat configuration across routes + +Suppose you have several routes that all hit the same model with the same provider options and tool set. Factor the configuration out once: + +```typescript +// lib/ai/chat-options.ts +import { createChatOptions, toolDefinition } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { z } from 'zod' +import { db } from './db' + +const lookupOrderDef = toolDefinition({ + name: 'lookupOrder', + description: 'Look up a customer order by ID', + inputSchema: z.object({ orderId: z.string() }), +}) + +const lookupOrder = lookupOrderDef.server(async ({ orderId }) => { + return db.orders.findUnique({ where: { id: orderId } }) +}) + +export const supportChatOptions = createChatOptions({ + adapter: openaiText('gpt-5.5'), + systemPrompts: ['You are a customer-support assistant for Acme Corp.'], + tools: [lookupOrder], + modelOptions: { + reasoning: { effort: 'medium' }, + }, +}) +``` + +```typescript ignore +// routes/api/support/chat.ts +import { chat, toServerSentEventsResponse } from '@tanstack/ai' +import { supportChatOptions } from '@/lib/ai/chat-options' + +export async function POST(request: Request) { + const { messages } = await request.json() + const stream = chat({ ...supportChatOptions, messages }) + return toServerSentEventsResponse(stream) +} +``` + +```typescript ignore +// routes/api/support/draft-reply.ts — same adapter+tools, different schema +import { chat } from '@tanstack/ai' +import { supportChatOptions } from '@/lib/ai/chat-options' +import { z } from 'zod' + +export async function POST(request: Request) { + const { ticket } = await request.json() + const draft = await chat({ + ...supportChatOptions, + messages: [{ role: 'user', content: `Draft a reply to: ${ticket}` }], + outputSchema: z.object({ subject: z.string(), body: z.string() }), + stream: false, + }) + return Response.json(draft) +} +``` + +Both routes share the adapter, system prompt, tools, and reasoning settings; each adds what it needs. Override or omit any field at the call site — the spread wins on the right. + +## Example: typed pre-configured image generation + +```typescript +import { createImageOptions, generateImage } from '@tanstack/ai' +import { openaiImage } from '@tanstack/ai-openai' + +const heroImageOptions = createImageOptions({ + adapter: openaiImage('gpt-image-1'), + prompt: 'A glass sphere refracting a sunset over a calm sea', + size: '1536x1024', + numberOfImages: 1, +}) + +const result = await generateImage(heroImageOptions) +``` + +The same pattern works for `createVideoOptions`, `createSpeechOptions`, `createTranscriptionOptions`, `createAudioOptions`, and `createSummarizeOptions` — the adapter is captured in the typed options object and every downstream call is narrowed to it. + +## What the helper does NOT do + +- **No runtime behavior.** `createChatOptions(opts)` is `opts`. There is no validation, freezing, cloning, or memoization. If you mutate the returned object after creation, the next call sees the mutation. Treat the result as immutable by convention. +- **No partial typing.** The helper expects the full options shape it'll be spread into. If you need to build options up incrementally, type the intermediate state yourself (a `Partial<>` of the full chat options shape) and only call the helper at the boundary where the shape is complete. +- **No request execution.** The helper does not call the model. Only the activity function (`chat`, `generateImage`, …) makes the request. + +## Related + +- [Per-Model Type Safety](/ai/advanced/per-model-type-safety) — how the adapter+model pair drives `modelOptions` inference. +- [Tree-Shaking](/ai/advanced/tree-shaking) — why each adapter is exported separately, and how the typed-options pattern keeps your bundle small. +- [Extend Adapter](/ai/advanced/extend-adapter) — when you need to add custom models to an adapter without losing the same typed-options ergonomics. diff --git a/mintlify/ai/api/ai-angular.md b/mintlify/ai/api/ai-angular.md new file mode 100644 index 000000000..d1f6ff1d4 --- /dev/null +++ b/mintlify/ai/api/ai-angular.md @@ -0,0 +1,581 @@ +--- +title: "@tanstack/ai-angular" +id: ai-angular +order: 6 +description: "API reference for @tanstack/ai-angular — Angular signal-based injectables including injectChat for streaming chat with full type safety." +keywords: + - tanstack ai + - "@tanstack/ai-angular" + - angular + - signals + - injectChat + - injectables + - api reference +--- + +Angular signal-based bindings for TanStack AI, providing convenient Angular bindings for the headless client. + +> **Injection context requirement:** Every `inject*` function in this package calls Angular's `inject()` internally. They **must** be called within an Angular injection context — a component or directive class field initializer, the constructor, or inside `runInInjectionContext`. Calling them outside an injection context will throw a runtime error. + +## Installation + +```bash +npm install @tanstack/ai-angular +``` + +## `injectChat(options?)` + +Main injectable for managing chat state in Angular with full type safety. + +```typescript +import { Component } from "@angular/core"; +import { injectChat, fetchServerSentEvents } from "@tanstack/ai-angular"; + +@Component({ + selector: "app-chat", + standalone: true, + template: `...`, +}) +export class ChatComponent { + // injectChat is called in a field initializer — valid injection context. + chat = injectChat({ + connection: fetchServerSentEvents("/api/chat"), + }); +} +``` + +### Options + +Extends `ChatClientOptions` from `@tanstack/ai-client` (minus internal state callbacks): + +- `connection` - Connection adapter (required, or use `fetcher`) +- `fetcher?` - Direct async function for one-shot generation (alternative to `connection`) +- `tools?` - Array of client tool implementations (with `.client()` method) +- `initialMessages?` - Initial messages array +- `id?` - Unique identifier for this chat instance +- `threadId?` - Thread ID for AG-UI run correlation. Persists across sends; auto-generated if omitted +- `forwardedProps?` - Arbitrary client-controlled JSON forwarded to the server in the AG-UI `RunAgentInput.forwardedProps` field. Reactive — accepts a plain value, an Angular `Signal`, or a zero-arg getter; changes sync automatically via `effect` +- `body?` - **Deprecated.** Use `forwardedProps` instead. Still works for backward compatibility; values are merged into `forwardedProps` on the wire. Reactive (same forms as `forwardedProps`) +- `context?` - Typed client-local runtime context passed to client tool implementations. Reactive (same forms). This value is not serialized to the server +- `live?` - Enable live subscription mode (auto-subscribes/unsubscribes). Reactive (same forms) +- `outputSchema?` - Standard-schema-compatible schema (Zod, Valibot, ArkType, or JSON Schema). When provided, adds typed `partial` and `final` signals to the return value +- `persistence?` - Persistence configuration +- `devtools?` - Display options for TanStack AI Devtools +- `onResponse?` - Callback when response is received +- `onChunk?` - Callback when stream chunk is received +- `onFinish?` - Callback when response finishes +- `onError?` - Callback when error occurs +- `onCustomEvent?` - Callback for custom stream events +- `streamProcessor?` - Stream processing configuration + +**Reactive options** (`body`, `forwardedProps`, `context`, `live`) accept a `ReactiveOption`, which is one of: + +```typescript +import type { Signal } from "@angular/core"; + +type ReactiveOption = T | Signal | (() => T); +``` + +A plain value becomes a constant; a `Signal` is read directly; a zero-arg getter is wrapped in `computed` so any signals read inside it are tracked. + +**Note:** Client tools are automatically executed — no `onToolCall` callback needed! + +### Returns + +```typescript +import type { Signal } from "@angular/core"; +import type { + UIMessage, + MultimodalContent, + DeepPartial, +} from "@tanstack/ai-angular"; +import type { ModelMessage, InferSchemaType } from "@tanstack/ai/client"; +import type { ChatClientState, ConnectionStatus } from "@tanstack/ai-client"; +type TSchema = any; + +interface InjectChatResult { + messages: Signal; + sendMessage: (content: string | MultimodalContent) => Promise; + append: (message: ModelMessage | UIMessage) => Promise; + addToolResult: (result: { + toolCallId: string; + tool: string; + output: any; + state?: "output-available" | "output-error"; + errorText?: string; + }) => Promise; + addToolApprovalResponse: (response: { + id: string; + approved: boolean; + }) => Promise; + reload: () => Promise; + stop: () => void; + clear: () => void; + setMessages: (messages: UIMessage[]) => void; + isLoading: Signal; + error: Signal; + status: Signal; + isSubscribed: Signal; + connectionStatus: Signal; + sessionGenerating: Signal; + // Only present when outputSchema is supplied: + partial: Signal>>; + final: Signal | null>; +} +``` + +**Note:** All reactive state (`messages`, `isLoading`, `error`, `status`, `isSubscribed`, `connectionStatus`, `sessionGenerating`) is exposed as read-only Angular `Signal`s. Read them by calling them as functions (e.g., `chat.messages()`, `chat.isLoading()`). Cleanup is automatic via `DestroyRef.onDestroy`. + +## Connection Adapters + +Re-exported from `@tanstack/ai-client` for convenience: + +```typescript +import { + fetchServerSentEvents, + fetchHttpStream, + xhrServerSentEvents, + xhrHttpStream, + stream, + rpcStream, + type ConnectionAdapter, +} from "@tanstack/ai-angular"; +``` + +## Example: Basic Chat + +```typescript +import { Component } from "@angular/core"; +import { CommonModule } from "@angular/common"; +import { injectChat, fetchServerSentEvents } from "@tanstack/ai-angular"; + +@Component({ + selector: "app-chat", + standalone: true, + imports: [CommonModule], + template: ` +
    + @for (message of chat.messages(); track message.id) { +
  • + {{ message.role }}: + @for (part of message.parts; track $index) { + @if (part.type === 'thinking') { + Thinking: {{ part.content }} + } @else if (part.type === 'text') { + {{ part.content }} + } + } +
  • + } +
+ + + @if (chat.isLoading()) { +

Thinking...

+ } + `, +}) +export class ChatComponent { + chat = injectChat({ + connection: fetchServerSentEvents("/api/chat"), + }); +} +``` + +## Example: Tool Approval + +```typescript +import { Component } from "@angular/core"; +import { CommonModule } from "@angular/common"; +import { injectChat, fetchServerSentEvents } from "@tanstack/ai-angular"; + +@Component({ + selector: "app-approval-chat", + standalone: true, + imports: [CommonModule], + template: ` + @for (message of chat.messages(); track message.id) { + @for (part of message.parts; track $index) { + @if ( + part.type === 'tool-call' && + part.state === 'approval-requested' && + part.approval + ) { +
+

Approve: {{ part.name }}

+ + +
+ } + } + } + `, +}) +export class ApprovalChatComponent { + chat = injectChat({ + connection: fetchServerSentEvents("/api/chat"), + }); +} +``` + +## Example: Client Tools with Type Safety + +```typescript +import { Component } from "@angular/core"; +import { CommonModule } from "@angular/common"; +import { injectChat, fetchServerSentEvents } from "@tanstack/ai-angular"; +import { + clientTools, + createChatClientOptions, + type InferChatMessages, +} from "@tanstack/ai-client"; +import { toolDefinition } from "@tanstack/ai"; +import { z } from "zod"; + +const updateUIDef = toolDefinition({ + name: "updateUI", + description: "Show a notification in the UI", + inputSchema: z.object({ message: z.string(), type: z.string() }), +}); + +const saveToStorageDef = toolDefinition({ + name: "saveToStorage", + description: "Save a value to localStorage", + inputSchema: z.object({ key: z.string(), value: z.string() }), +}); + +@Component({ + selector: "app-typed-chat", + standalone: true, + imports: [CommonModule], + template: ` + @for (message of chat.messages(); track message.id) { + @for (part of message.parts; track $index) { + @if (part.type === 'tool-call' && part.name === 'updateUI') { +
Tool executed: {{ part.name }}
+ } + } + } + `, +}) +export class TypedChatComponent { + // Create client implementations + private updateUI = updateUIDef.client((input) => { + // input is fully typed! + return { success: true }; + }); + + private saveToStorage = saveToStorageDef.client((input) => { + localStorage.setItem(input.key, input.value); + return { saved: true }; + }); + + // Create typed tools array (no 'as const' needed!) + private tools = clientTools(this.updateUI, this.saveToStorage); + + chat = injectChat({ + connection: fetchServerSentEvents("/api/chat"), + tools: this.tools, // Automatic execution, full type safety + }); +} +``` + +## Example: Reactive Options with Signals + +```typescript +import { Component, signal } from "@angular/core"; +import { injectChat, fetchServerSentEvents } from "@tanstack/ai-angular"; + +@Component({ + selector: "app-reactive-chat", + standalone: true, + template: ` + + @for (message of chat.messages(); track message.id) { +

{{ message.role }}: {{ message.parts[0]?.content }}

+ } + `, +}) +export class ReactiveChatComponent { + language = signal("en"); + + // forwardedProps is reactive — the signal is read on every request + chat = injectChat({ + connection: fetchServerSentEvents("/api/chat"), + forwardedProps: () => ({ language: this.language() }), + }); + + toggleLanguage() { + this.language.set(this.language() === "en" ? "fr" : "en"); + } +} +``` + +## Example: Structured Output + +```typescript +import { Component } from "@angular/core"; +import { injectChat, fetchServerSentEvents } from "@tanstack/ai-angular"; +import { z } from "zod"; + +const recipeSchema = z.object({ + title: z.string(), + ingredients: z.array(z.string()), + steps: z.array(z.string()), +}); + +@Component({ + selector: "app-recipe-chat", + standalone: true, + template: ` + + @if (chat.partial().title) { +

{{ chat.partial().title }}

+ } + @if (chat.final()) { +
    + @for (step of chat.final()!.steps; track $index) { +
  • {{ step }}
  • + } +
+ } + `, +}) +export class RecipeChatComponent { + chat = injectChat({ + connection: fetchServerSentEvents("/api/chat"), + outputSchema: recipeSchema, + }); +} +``` + +## Generation Injectables + +Angular injectables for one-shot generation tasks (images, audio, speech, transcription, summarization, video). All share the same pattern: provide a `connection` or `fetcher`, call `generate()`, and read reactive signals. + +### `injectGeneration(options)` + +Base injectable for custom generation types. All specialized injectables below are built on this. + +```typescript +import { Component } from "@angular/core"; +import { injectGeneration } from "@tanstack/ai-angular"; +import { fetchServerSentEvents } from "@tanstack/ai-client"; + +@Component({ selector: "app-custom", standalone: true, template: `...` }) +export class CustomGenerationComponent { + gen = injectGeneration({ + connection: fetchServerSentEvents("/api/generate/custom"), + }); + + // Call gen.generate(input), read gen.result(), gen.isLoading(), etc. +} +``` + +**Options:** `connection?`, `fetcher?`, `id?`, `body?` (reactive), `devtools?`, `onResult?`, `onError?`, `onProgress?`, `onChunk?` + +**Returns:** `generate`, `result`, `isLoading`, `error`, `status`, `stop`, `reset` — all reactive state is a read-only `Signal`. + +### `injectGenerateImage(options)` + +Image generation injectable. `generate()` accepts `ImageGenerateInput`, result is `ImageGenerationResult`. + +```typescript +import { Component } from "@angular/core"; +import { injectGenerateImage } from "@tanstack/ai-angular"; +import { fetchServerSentEvents } from "@tanstack/ai-client"; + +@Component({ + selector: "app-image", + standalone: true, + template: ` + + @if (gen.result()) { + Generated image + } + `, +}) +export class ImageComponent { + gen = injectGenerateImage({ + connection: fetchServerSentEvents("/api/generate/image"), + }); +} +``` + +### `injectGenerateAudio(options)` + +Audio generation injectable (music, sound effects). `generate()` accepts `AudioGenerateInput`, result is `AudioGenerationResult`. + +```typescript +import { Component } from "@angular/core"; +import { injectGenerateAudio } from "@tanstack/ai-angular"; +import { fetchServerSentEvents } from "@tanstack/ai-client"; + +@Component({ + selector: "app-audio", + standalone: true, + template: ` + + @if (gen.result()) { + + } + `, +}) +export class AudioComponent { + gen = injectGenerateAudio({ + connection: fetchServerSentEvents("/api/generate/audio"), + }); +} +``` + +### `injectGenerateSpeech(options)` + +Text-to-speech injectable. `generate()` accepts `SpeechGenerateInput`, result is `TTSResult`. + +### `injectTranscription(options)` + +Audio transcription injectable. `generate()` accepts `TranscriptionGenerateInput`, result is `TranscriptionResult`. + +### `injectSummarize(options)` + +Text summarization injectable. `generate()` accepts `SummarizeGenerateInput`, result is `SummarizationResult`. + +### `injectGenerateVideo(options)` + +Video generation injectable with job polling. Returns additional `jobId` and `videoStatus` signals. Accepts extra `onJobCreated?` and `onStatusUpdate?` callbacks. + +```typescript +import { Component } from "@angular/core"; +import { injectGenerateVideo } from "@tanstack/ai-angular"; +import { fetchServerSentEvents } from "@tanstack/ai-client"; + +@Component({ + selector: "app-video", + standalone: true, + template: ` + + @if (gen.videoStatus()) { +

Status: {{ gen.videoStatus()!.status }}

+ } + @if (gen.result()) { + + } + `, +}) +export class VideoComponent { + gen = injectGenerateVideo({ + connection: fetchServerSentEvents("/api/generate/video"), + onJobCreated: (jobId) => console.log("Job created:", jobId), + }); +} +``` + +**Additional returns (video only):** +- `jobId: Signal` — The polling job ID, once the server creates it +- `videoStatus: Signal` — Real-time status updates from the polling loop + +All generation injectables automatically clean up via `DestroyRef.onDestroy`. + +## Injection Context + +Angular's DI system requires that `inject()` is called during component construction. Every `inject*` function in this package calls `inject()` internally. Valid call sites: + +```typescript +import { inject, runInInjectionContext, Injector } from "@angular/core"; +import { injectChat, fetchServerSentEvents } from "@tanstack/ai-angular"; + +const injector = inject(Injector); + +// Field initializer (recommended) +export class MyComponent { + chat = injectChat({ connection: fetchServerSentEvents("/api/chat") }); +} + +// Constructor +export class MyComponentAlt { + chat: ReturnType; + constructor() { + this.chat = injectChat({ connection: fetchServerSentEvents("/api/chat") }); + } +} + +// Inside runInInjectionContext +const chat = runInInjectionContext(injector, () => + injectChat({ connection: fetchServerSentEvents("/api/chat") }), +); +``` + +## `createChatClientOptions(options)` + +Helper to create typed chat options (re-exported from `@tanstack/ai-client`). + +```typescript +import { + clientTools, + createChatClientOptions, + type InferChatMessages, +} from "@tanstack/ai-client"; +import { fetchServerSentEvents } from "@tanstack/ai-angular"; +import { tool1, tool2 } from "./tools"; + +// Create typed tools array (no 'as const' needed!) +const tools = clientTools(tool1, tool2); + +const chatOptions = createChatClientOptions({ + connection: fetchServerSentEvents("/api/chat"), + tools, +}); + +type Messages = InferChatMessages; +``` + +## Types + +Re-exported from `@tanstack/ai-angular` (sourced from `@tanstack/ai-client`): + +- `UIMessage` - Message type with tool type parameter +- `InjectChatOptions` - Chat injectable options +- `InjectChatResult` - Chat injectable return type +- `ReactiveOption` - Union of `T | Signal | (() => T)` for reactive option fields +- `DeepPartial` - Recursive partial; used to type the in-flight `partial` value +- `ChatRequestBody` - Request body type +- `MultimodalContent` - Multimodal content type for `sendMessage` +- `ConnectionAdapter` - Connection adapter interface +- `InferChatMessages` - Extract message type from options +- `GenerationClientState` - Generation lifecycle state +- `ImageGenerateInput` - Image generation input type +- `AudioGenerateInput` - Audio generation input type +- `SpeechGenerateInput` - Speech generation input type +- `TranscriptionGenerateInput` - Transcription input type +- `SummarizeGenerateInput` - Summarization input type +- `VideoGenerateInput` - Video generation input type +- `VideoGenerateResult` - Video generation result type +- `VideoStatusInfo` - Video job status info + +Tool authoring types — import directly from `@tanstack/ai` (not re-exported by `@tanstack/ai-angular`): + +- `toolDefinition()` - Create isomorphic tool definition +- `ToolDefinitionInstance` - Tool definition type +- `ClientTool` - Client tool type +- `ServerTool` - Server tool type + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about the isomorphic tool system +- [Client Tools](/ai/tools/client-tools) - Learn about client-side tools diff --git a/mintlify/ai/api/ai-client.md b/mintlify/ai/api/ai-client.md new file mode 100644 index 000000000..950c7d9f6 --- /dev/null +++ b/mintlify/ai/api/ai-client.md @@ -0,0 +1,520 @@ +--- +title: "@tanstack/ai-client" +slug: /api/ai-client +order: 2 +description: "API reference for @tanstack/ai-client — the framework-agnostic headless client for managing chat state and streaming transports." +keywords: + - tanstack ai + - "@tanstack/ai-client" + - headless client + - ChatClient + - chat state + - connection adapters + - api reference +--- + +Framework-agnostic headless client for managing chat state and streaming. + +## Installation + +```bash +npm install @tanstack/ai-client +``` + +## `ChatClient` + +The main client class for managing chat state. + +```typescript +import { + ChatClient, + clientTools, + fetchServerSentEvents, + type UIMessage, +} from "@tanstack/ai-client"; +import { myClientTool } from "./tools"; + +const client = new ChatClient({ + connection: fetchServerSentEvents("/api/chat"), + initialMessages: [], + tools: clientTools(myClientTool), + onMessagesChange: (messages: UIMessage[]) => { + console.log("Messages updated:", messages); + }, +}); +``` + +### Constructor Options + +- `connection` - Connection adapter for streaming +- `initialMessages?` - Initial messages array +- `id?` - Unique identifier for this chat instance +- `threadId?` - Thread ID for AG-UI run correlation. Persists across sends; auto-generated if omitted +- `forwardedProps?` - Arbitrary client-controlled JSON forwarded to the server in the AG-UI `RunAgentInput.forwardedProps` field +- `body?` - **Deprecated.** Use `forwardedProps` instead. Still works — values are merged into `forwardedProps` on the wire and mirrored under the legacy `data` field for backward compatibility +- `context?` - Typed client-local runtime context passed to client tool implementations. This value is not serialized to the server +- `tools?` - Registered `.client()` tool implementations. The client automatically executes matching tools when the model calls them +- `onResponse?` - Callback when response is received +- `onChunk?` - Callback when stream chunk is received +- `onFinish?` - Callback when response finishes +- `onError?` - Callback when error occurs +- `onMessagesChange?` - Callback when messages change +- `onLoadingChange?` - Callback when loading state changes +- `onErrorChange?` - Callback when error state changes +- `streamProcessor?` - Stream processing configuration + +### Methods + +#### `sendMessage(content: string)` + +Sends a user message and gets a response. + +```typescript +import { client } from "./client"; + +await client.sendMessage("Hello!"); +``` + +#### `append(message: ModelMessage | UIMessage)` + +Appends a message to the conversation. + +```typescript +import { client } from "./client"; + +await client.append({ + role: "user", + content: "Additional context", +}); +``` + +#### `reload()` + +Reloads the last assistant message. + +```typescript +import { client } from "./client"; + +await client.reload(); +``` + +#### `stop()` + +Stops the current response generation. + +```typescript +import { client } from "./client"; + +client.stop(); +``` + +#### `clear()` + +Clears all messages. + +```typescript +import { client } from "./client"; + +client.clear(); +``` + +#### `setMessagesManually(messages: UIMessage[])` + +Manually sets the messages array. + +```typescript +import { client } from "./client"; +import type { UIMessage } from "@tanstack/ai-client"; + +const newMessages: UIMessage[] = []; +client.setMessagesManually([...newMessages]); +``` + +#### `addToolResult(result)` + +Adds the result of a client-side tool execution. + +```typescript +import { client } from "./client"; + +await client.addToolResult({ + toolCallId: "call_123", + tool: "toolName", + output: { result: "..." }, + state: "output-available", +}); +``` + +#### `addToolApprovalResponse(response)` + +Responds to a tool approval request. + +```typescript +import { client } from "./client"; + +await client.addToolApprovalResponse({ + id: "approval_123", + approved: true, +}); +``` + +### Properties + +- `messages: UIMessage[]` - Current messages +- `isLoading: boolean` - Whether a response is being generated +- `error: Error | undefined` - Current error, if any + +## Connection Adapters + +For a complete transport walkthrough, see +[Connection Adapters](/ai/chat/connection-adapters). For React Native and Expo, +see [Quick Start: React Native](/ai/getting-started/quick-start-react-native). + +### `fetchServerSentEvents(url, options?)` + +Creates an SSE connection adapter. + +```typescript +import { fetchServerSentEvents } from "@tanstack/ai-client"; + +const adapter = fetchServerSentEvents("/api/chat", { + headers: { + Authorization: "Bearer token", + }, +}); +``` + +### `fetchHttpStream(url, options?)` + +Creates a newline-delimited JSON HTTP stream connection adapter. Pair it with +`toHttpResponse()` on the server. + +```typescript +import { fetchHttpStream } from "@tanstack/ai-client"; + +const adapter = fetchHttpStream("/api/chat"); +``` + +`fetchHttpStream()` requires a runtime with streaming `fetch`, +`Response.body.getReader()`, and `TextDecoder`. If the runtime cannot expose an +incremental response body, it throws `UnsupportedResponseStreamError`; use the +XHR adapters in React Native or Expo. + +### `xhrHttpStream(url, options?)` + +Creates an `XMLHttpRequest`-backed newline-delimited JSON stream adapter. This +is the recommended default for React Native and Expo chat screens. Pair it with +`toHttpResponse()` on the server. + +```typescript +import { xhrHttpStream } from "@tanstack/ai-client"; + +const adapter = xhrHttpStream("http://192.168.1.10:8787/chat/http", { + headers: { Authorization: "Bearer token" }, + withCredentials: true, +}); +``` + +### `xhrServerSentEvents(url, options?)` + +Creates an `XMLHttpRequest`-backed SSE adapter for runtimes where XHR progress +events are more reliable than streaming `fetch`. Pair it with +`toServerSentEventsResponse()` on the server. + +```typescript +import { xhrServerSentEvents } from "@tanstack/ai-client"; + +const adapter = xhrServerSentEvents("http://192.168.1.10:8787/chat/sse"); +``` + +### Adapter options + +Fetch adapters accept: + +- `headers?: Record | Headers` +- `credentials?: RequestCredentials` +- `signal?: AbortSignal` +- `body?: Record` +- `fetchClient?: typeof globalThis.fetch` + +XHR adapters accept: + +- `headers?: Record | Headers` +- `withCredentials?: boolean` +- `signal?: AbortSignal` +- `body?: Record` +- `xhrFactory?: () => XMLHttpRequest` + +`body` is merged into the AG-UI `forwardedProps` payload. Values from +`forwardedProps` on the client and per-message `sendMessage(..., data)` calls +override static adapter `body` values. + +### Stream errors + +- `UnsupportedResponseStreamError` - thrown by fetch-based adapters when + `Response.body`, `Response.body.getReader()`, or `TextDecoder` is missing. +- `StreamTruncatedError` - thrown when an SSE or NDJSON stream ends with + unterminated trailing data, usually because the server, proxy, or network cut + the connection mid-line. + +### `stream(connectFn)` + +Creates a custom connection adapter. + +```typescript ignore +import { stream } from "@tanstack/ai-client"; + +const adapter = stream(async (messages, data, signal) => { + // `data` here carries the merged forwardedProps. The fetch-based + // adapters serialize it as the AG-UI `RunAgentInput.forwardedProps` + // field on the wire (with a backward-compat `data` mirror). + const response = await fetch("/api/chat", { + method: "POST", + body: JSON.stringify({ messages, forwardedProps: data }), + signal, + }); + return processStream(response); +}); +``` + +## Helper Functions + +### `clientTools(...tools)` + +Creates a typed array of client tools with proper type inference. This eliminates the need for `as const` when defining tool arrays and enables proper discriminated union type narrowing. + +```typescript +import { + clientTools, + createChatClientOptions, + fetchServerSentEvents, + type UIMessage, +} from "@tanstack/ai-client"; +import { toolDefinition } from "@tanstack/ai"; +import { z } from "zod"; + +const messages: UIMessage[] = []; + +const myTool1 = toolDefinition({ + name: "myTool1", + description: "First tool", + inputSchema: z.object({ query: z.string() }), + outputSchema: z.object({ result: z.string() }), +}); + +const myTool2 = toolDefinition({ + name: "myTool2", + description: "Second tool", + inputSchema: z.object({ query: z.string() }), + outputSchema: z.object({ result: z.string() }), +}); + +// Create client implementations +const tool1Client = myTool1.client((input) => { + // Implementation + return { result: input.query }; +}); + +const tool2Client = myTool2.client((input) => { + // Implementation + return { result: input.query }; +}); + +// Create typed tools array (no 'as const' needed!) +const tools = clientTools(tool1Client, tool2Client); + +// Now when you use these tools in chat options: +const chatOptions = createChatClientOptions({ + connection: fetchServerSentEvents("/api/chat"), + tools, // Fully typed with literal tool names +}); + +// In your component: +messages.forEach((message) => { + message.parts.forEach((part) => { + if (part.type === "tool-call" && part.name === "myTool1") { + // ✅ TypeScript knows part.name is literally "myTool1" + // ✅ part.input is typed from myTool1's input schema + // ✅ part.output is typed from myTool1's output schema + } + }); +}); +``` + +### `createChatClientOptions(options)` + +Helper function to create typed chat client options with proper type inference. + +```typescript +import { + createChatClientOptions, + clientTools, + fetchServerSentEvents, + type InferChatMessages, +} from "@tanstack/ai-client"; +import { tool1, tool2 } from "./tools"; + +const tools = clientTools(tool1, tool2); + +const chatOptions = createChatClientOptions({ + connection: fetchServerSentEvents("/api/chat"), + tools, +}); + +// Use InferChatMessages to extract message types +type ChatMessages = InferChatMessages; +``` + +`createChatClientOptions` also preserves typed client runtime context: + +```typescript +import { + createChatClientOptions, + clientTools, + fetchServerSentEvents, +} from "@tanstack/ai-client"; +import { toolDefinition } from "@tanstack/ai"; +import { z } from "zod"; + +type ClientContext = { + activeProjectId: string; +}; + +const projectTool = toolDefinition({ + name: "projectAction", + description: "Run a project action", + inputSchema: z.object({ action: z.string() }), + outputSchema: z.object({ ok: z.boolean() }), +}); + +const tool = projectTool.client((input, ctx: { context: ClientContext }) => { + console.log(ctx.context.activeProjectId, input.action); + return { ok: true }; +}); + +const chatOptions = createChatClientOptions({ + connection: fetchServerSentEvents("/api/chat"), + tools: clientTools(tool), + context: { + activeProjectId: "project_123", + }, +}); +``` + +Client runtime context is local to the client instance. Use `forwardedProps` for explicit client-to-server handoff of serializable values, then validate and map those values into server `chat({ context })`. + +## Types + +### `UIMessage` + +```typescript ignore +interface UIMessage { + id: string; + role: "user" | "assistant"; + parts: MessagePart[]; + createdAt?: Date; +} +``` + +### `MessagePart` + +```typescript ignore +type MessagePart = TextPart | ThinkingPart | ToolCallPart | ToolResultPart; +``` + +### `TextPart` + +```typescript +interface TextPart { + type: "text"; + content: string; +} +``` + +### `ThinkingPart` + +```typescript +interface ThinkingPart { + type: "thinking"; + content: string; +} +``` + +Thinking parts represent the model's internal reasoning process. They are typically displayed in a collapsible format and automatically collapse when the response text appears. Thinking parts are UI-only and are not sent back to the model in subsequent requests. + +**Note:** Thinking parts are only available when using models that support reasoning/thinking (e.g., Anthropic Claude with thinking enabled, OpenAI GPT-5 with reasoning enabled). + +### `ToolCallPart` + +```typescript ignore +interface ToolCallPart { + type: "tool-call"; + id: string; + name: string; + arguments: string; // JSON string (may be incomplete during streaming) + input?: any; // Parsed tool input (typed from tool's inputSchema) + state: ToolCallState; + approval?: ApprovalRequest; + output?: any; // Tool execution output (typed from tool's outputSchema) +} +``` + +When using typed tools with `clientTools()` and `createChatClientOptions()`, the `input` and `output` fields are automatically typed based on your tool's Zod schemas, and `name` becomes a discriminated union enabling type narrowing. + +### `ToolResultPart` + +```typescript ignore +interface ToolResultPart { + type: "tool-result"; + toolCallId: string; + content: string; + state: ToolResultState; + error?: string; +} +``` + +### `ToolCallState` + +```typescript ignore +type ToolCallState = + | "awaiting-input" + | "input-streaming" + | "input-complete" + | "approval-requested" + | "approval-responded" + | "complete"; +``` + +### `ToolResultState` + +```typescript ignore +type ToolResultState = + | "streaming" + | "complete" + | "error"; +``` + +## Stream Processing + +Configure stream processing with chunk strategies: + +```typescript +import { + ChatClient, + ImmediateStrategy, + fetchServerSentEvents, +} from "@tanstack/ai-client"; + +const client = new ChatClient({ + connection: fetchServerSentEvents("/api/chat"), + streamProcessor: { + chunkStrategy: new ImmediateStrategy(), // Emit every chunk + }, +}); +``` + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Connection Adapters](/ai/chat/connection-adapters) - Learn about adapters +- [@tanstack/ai-react API](/ai/api/ai-react) - React hooks wrapper diff --git a/mintlify/ai/api/ai-preact.md b/mintlify/ai/api/ai-preact.md new file mode 100644 index 000000000..062d40b4f --- /dev/null +++ b/mintlify/ai/api/ai-preact.md @@ -0,0 +1,357 @@ +--- +title: "@tanstack/ai-preact" +slug: /api/ai-preact +order: 5 +description: "API reference for @tanstack/ai-preact — Preact hooks including useChat for streaming chat with full type safety in Preact apps." +keywords: + - tanstack ai + - "@tanstack/ai-preact" + - preact + - useChat + - preact hooks + - api reference +--- + +Preact hooks for TanStack AI, providing convenient Preact bindings for the headless client. + +## Installation + +```bash +npm install @tanstack/ai-preact +``` + +## `useChat(options?)` + +Main hook for managing chat state in Preact with full type safety. + +```tsx +import { useChat, fetchServerSentEvents } from "@tanstack/ai-preact"; +import { + clientTools, + createChatClientOptions, + type InferChatMessages +} from "@tanstack/ai-client"; +import { toolDefinition } from "@tanstack/ai"; +import { z } from "zod"; +import { useState } from "preact/hooks"; + +const updateUIDef = toolDefinition({ + name: "updateUI", + description: "Show a notification in the UI", + inputSchema: z.object({ message: z.string() }), +}); + +function ChatComponent() { + const [, setNotification] = useState(null); + // Create client tool implementations + const updateUI = updateUIDef.client((input) => { + setNotification(input.message); + return { success: true }; + }); + + // Create typed tools array (no 'as const' needed!) + const tools = clientTools(updateUI); + + const chatOptions = createChatClientOptions({ + connection: fetchServerSentEvents("/api/chat"), + tools, + }); + + // Fully typed messages! + type ChatMessages = InferChatMessages; + + const { messages, sendMessage, isLoading, error, addToolApprovalResponse } = + useChat(chatOptions); + + return
{/* Chat UI with typed messages */}
; +} +``` + +### Options + +Extends `ChatClientOptions` from `@tanstack/ai-client`: + +- `connection` - Connection adapter (required) +- `tools?` - Array of client tool implementations (with `.client()` method) +- `initialMessages?` - Initial messages array +- `id?` - Unique identifier for this chat instance +- `threadId?` - Thread ID for AG-UI run correlation. Persists across sends; auto-generated if omitted +- `forwardedProps?` - Arbitrary client-controlled JSON forwarded to the server in the AG-UI `RunAgentInput.forwardedProps` field (e.g., `{ provider: 'openai', model: 'gpt-4o' }`) +- `body?` - **Deprecated.** Use `forwardedProps` instead. Still works for backward compatibility; values are merged into `forwardedProps` on the wire +- `context?` - Typed client-local runtime context passed to client tool implementations. This value is not serialized to the server +- `onResponse?` - Callback when response is received +- `onChunk?` - Callback when stream chunk is received +- `onFinish?` - Callback when response finishes +- `onError?` - Callback when error occurs +- `streamProcessor?` - Stream processing configuration + +**Note:** Client tools are now automatically executed - no `onToolCall` callback needed! + +### Returns + +```typescript +import type { UIMessage } from "@tanstack/ai-preact"; +import type { ModelMessage } from "@tanstack/ai/client"; + +interface UseChatReturn { + messages: UIMessage[]; + sendMessage: (content: string) => Promise; + append: (message: ModelMessage | UIMessage) => Promise; + addToolResult: (result: { + toolCallId: string; + tool: string; + output: any; + state?: "output-available" | "output-error"; + errorText?: string; + }) => Promise; + addToolApprovalResponse: (response: { + id: string; + approved: boolean; + }) => Promise; + reload: () => Promise; + stop: () => void; + isLoading: boolean; + error: Error | undefined; + setMessages: (messages: UIMessage[]) => void; + clear: () => void; +} +``` + +## Connection Adapters + +Re-exported from `@tanstack/ai-client` for convenience: + +```typescript +import { + fetchServerSentEvents, + fetchHttpStream, + stream, + type ConnectionAdapter, +} from "@tanstack/ai-preact"; +``` + +## Example: Basic Chat + +```tsx +import { useState } from "preact/hooks"; +import { useChat, fetchServerSentEvents } from "@tanstack/ai-preact"; + +export function Chat() { + const [input, setInput] = useState(""); + + const { messages, sendMessage, isLoading } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + }); + + const handleSubmit = (e: Event) => { + e.preventDefault(); + if (input.trim() && !isLoading) { + sendMessage(input); + setInput(""); + } + }; + + return ( +
+
+ {messages.map((message) => ( +
+ {message.role}: + {message.parts.map((part, idx) => { + if (part.type === "thinking") { + return ( +
+ 💭 Thinking: {part.content} +
+ ); + } + if (part.type === "text") { + return {part.content}; + } + return null; + })} +
+ ))} +
+
+ setInput(e.currentTarget.value)} + disabled={isLoading} + /> + +
+
+ ); +} +``` + +## Example: Tool Approval + +```tsx +import { useChat, fetchServerSentEvents } from "@tanstack/ai-preact"; + +export function ChatWithApproval() { + const { messages, sendMessage, addToolApprovalResponse } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + }); + + return ( +
+ {messages.map((message) => + message.parts.map((part) => { + if ( + part.type === "tool-call" && + part.state === "approval-requested" && + part.approval + ) { + return ( +
+

Approve: {part.name}

+ + +
+ ); + } + return null; + }) + )} +
+ ); +} +``` + +## Example: Client Tools with Type Safety + +```tsx +import { useChat, fetchServerSentEvents } from "@tanstack/ai-preact"; +import { + clientTools, + createChatClientOptions, + type InferChatMessages +} from "@tanstack/ai-client"; +import { toolDefinition } from "@tanstack/ai"; +import { z } from "zod"; +import { useState } from "preact/hooks"; + +const updateUIDef = toolDefinition({ + name: "updateUI", + description: "Show a notification in the UI", + inputSchema: z.object({ message: z.string(), type: z.string() }), +}); + +const saveToStorageDef = toolDefinition({ + name: "saveToStorage", + description: "Save a value to localStorage", + inputSchema: z.object({ key: z.string(), value: z.string() }), +}); + +export function ChatWithClientTools() { + const [notification, setNotification] = useState<{ message: string; type: string } | null>(null); + + // Create client implementations + const updateUI = updateUIDef.client((input) => { + // ✅ input is fully typed! + setNotification({ message: input.message, type: input.type }); + return { success: true }; + }); + + const saveToStorage = saveToStorageDef.client((input) => { + localStorage.setItem(input.key, input.value); + return { saved: true }; + }); + + // Create typed tools array (no 'as const' needed!) + const tools = clientTools(updateUI, saveToStorage); + + const { messages, sendMessage } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + tools, // ✅ Automatic execution, full type safety + }); + + return ( +
+ {messages.map((message) => + message.parts.map((part) => { + if (part.type === "tool-call" && part.name === "updateUI") { + // ✅ part.input and part.output are fully typed! + return
Tool executed: {part.name}
; + } + return null; + }) + )} +
+ ); +} +``` + +## `createChatClientOptions(options)` + +Helper to create typed chat options (re-exported from `@tanstack/ai-client`). + +```typescript +import { + clientTools, + createChatClientOptions, + type InferChatMessages +} from "@tanstack/ai-client"; +import { fetchServerSentEvents } from "@tanstack/ai-preact"; +import { tool1, tool2 } from "./tools"; + +// Create typed tools array (no 'as const' needed!) +const tools = clientTools(tool1, tool2); + +const chatOptions = createChatClientOptions({ + connection: fetchServerSentEvents("/api/chat"), + tools, +}); + +type Messages = InferChatMessages; +``` + +## Types + +Re-exported from `@tanstack/ai-client`: + +- `UIMessage` - Message type with tool type parameter +- `MessagePart` - Message part with tool type parameter +- `TextPart` - Text content part +- `ThinkingPart` - Thinking content part +- `ToolCallPart` - Tool call part (discriminated union) +- `ToolResultPart` - Tool result part +- `ChatClientOptions` - Chat client options with typed client runtime context +- `ConnectionAdapter` - Connection adapter interface +- `InferChatMessages` - Extract message type from options + +Re-exported from `@tanstack/ai`: + +- `toolDefinition()` - Create isomorphic tool definition +- `ToolDefinitionInstance` - Tool definition type +- `ClientTool` - Client tool type +- `ServerTool` - Server tool type + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about the isomorphic tool system +- [Client Tools](/ai/tools/client-tools) - Learn about client-side tools diff --git a/mintlify/ai/api/ai-react.md b/mintlify/ai/api/ai-react.md new file mode 100644 index 000000000..a095e4ae3 --- /dev/null +++ b/mintlify/ai/api/ai-react.md @@ -0,0 +1,394 @@ +--- +title: "@tanstack/ai-react" +slug: /api/ai-react +order: 3 +description: "API reference for @tanstack/ai-react — React hooks including useChat for streaming chat with full type safety in React apps." +keywords: + - tanstack ai + - "@tanstack/ai-react" + - react + - useChat + - react hooks + - api reference +--- + +React hooks for TanStack AI, providing convenient React bindings for the headless client. +For React Native, the documented support surface is narrow: `useChat` with chat +connection adapters. React DOM-specific UI packages and TanStack AI devtools UI +are not part of the React Native support surface. + +For a complete native journey, see +[Quick Start: React Native](/ai/getting-started/quick-start-react-native). + +## Installation + +```bash +npm install @tanstack/ai-react +``` + +## `useChat(options?)` + +Main hook for managing chat state in React with full type safety. + +```tsx +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; +import { + clientTools, + createChatClientOptions, + type InferChatMessages +} from "@tanstack/ai-client"; +import { toolDefinition } from "@tanstack/ai"; +import { z } from "zod"; +import { useState } from "react"; + +const updateUIDef = toolDefinition({ + name: "updateUI", + description: "Update the UI with a notification", + inputSchema: z.object({ + message: z.string(), + }), + outputSchema: z.object({ success: z.boolean() }), +}); + +function ChatComponent() { + const [notification, setNotification] = useState(null); + + // Create client tool implementations + const updateUI = updateUIDef.client((input) => { + setNotification(input.message); + return { success: true }; + }); + + // Create typed tools array (no 'as const' needed!) + const tools = clientTools(updateUI); + + const chatOptions = createChatClientOptions({ + connection: fetchServerSentEvents("/api/chat"), + tools, + }); + + // Fully typed messages! + type ChatMessages = InferChatMessages; + + const { messages, sendMessage, isLoading, error, addToolApprovalResponse } = + useChat(chatOptions); + + return
{/* Chat UI with typed messages */}
; +} +``` + +### Options + +Extends `ChatClientOptions` from `@tanstack/ai-client`: + +- `connection` - Connection adapter (required) +- `tools?` - Array of client tool implementations (with `.client()` method) +- `initialMessages?` - Initial messages array +- `id?` - Unique identifier for this chat instance +- `threadId?` - Thread ID for AG-UI run correlation. Persists across sends; auto-generated if omitted +- `forwardedProps?` - Arbitrary client-controlled JSON forwarded to the server in the AG-UI `RunAgentInput.forwardedProps` field (e.g., `{ provider: 'openai', model: 'gpt-4o' }`) +- `body?` - **Deprecated.** Use `forwardedProps` instead. Still works for backward compatibility; values are merged into `forwardedProps` on the wire +- `context?` - Typed client-local runtime context passed to client tool implementations. This value is not serialized to the server +- `onResponse?` - Callback when response is received +- `onChunk?` - Callback when stream chunk is received +- `onFinish?` - Callback when response finishes +- `onError?` - Callback when error occurs +- `streamProcessor?` - Stream processing configuration + +**Note:** Client tools are now automatically executed - no `onToolCall` callback needed! + +### Returns + +```typescript +import type { UIMessage } from "@tanstack/ai-react"; +import type { ModelMessage } from "@tanstack/ai"; + +interface UseChatReturn { + messages: UIMessage[]; + sendMessage: (content: string) => Promise; + append: (message: ModelMessage | UIMessage) => Promise; + addToolResult: (result: { + toolCallId: string; + tool: string; + output: any; + state?: "output-available" | "output-error"; + errorText?: string; + }) => Promise; + addToolApprovalResponse: (response: { + id: string; + approved: boolean; + }) => Promise; + reload: () => Promise; + stop: () => void; + isLoading: boolean; + error: Error | undefined; + setMessages: (messages: UIMessage[]) => void; + clear: () => void; +} +``` + +## Connection Adapters + +Re-exported from `@tanstack/ai-client` for convenience: + +```typescript +import { + fetchServerSentEvents, + fetchHttpStream, + xhrServerSentEvents, + xhrHttpStream, + stream, + type ConnectionAdapter, + type FetchConnectionOptions, + type XhrConnectionOptions, +} from "@tanstack/ai-react"; +``` + +For React Native or Expo chat screens, use an absolute server URL and prefer +`xhrHttpStream()` with a server route that returns `toHttpResponse()`. Use +`xhrServerSentEvents()` with `toServerSentEventsResponse()` when you want SSE. +Use `fetchHttpStream()` only when the runtime supports streaming `fetch`, +`Response.body.getReader()`, and `TextDecoder`; otherwise it throws +`UnsupportedResponseStreamError`. + +XHR adapter options include `headers`, `withCredentials`, `signal`, `body`, and +`xhrFactory`. Fetch adapter options include `headers`, `credentials`, `signal`, +`body`, and `fetchClient`. Both option objects may be provided directly or as a +function that resolves per request. + +For error narrowing, import `UnsupportedResponseStreamError` and +`StreamTruncatedError` from `@tanstack/ai-client`. + +## Example: Basic Chat + +```tsx +import { useState } from "react"; +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; + +export function Chat() { + const [input, setInput] = useState(""); + + const { messages, sendMessage, isLoading } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + }); + + const handleSubmit = (e: React.FormEvent) => { + e.preventDefault(); + if (input.trim() && !isLoading) { + sendMessage(input); + setInput(""); + } + }; + + return ( +
+
+ {messages.map((message) => ( +
+ {message.role}: + {message.parts.map((part, idx) => { + if (part.type === "thinking") { + return ( +
+ 💭 Thinking: {part.content} +
+ ); + } + if (part.type === "text") { + return {part.content}; + } + return null; + })} +
+ ))} +
+
+ setInput(e.target.value)} + disabled={isLoading} + /> + +
+
+ ); +} +``` + +## Example: Tool Approval + +```tsx +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; + +export function ChatWithApproval() { + const { messages, sendMessage, addToolApprovalResponse } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + }); + + return ( +
+ {messages.map((message) => + message.parts.map((part) => { + if ( + part.type === "tool-call" && + part.state === "approval-requested" && + part.approval + ) { + return ( +
+

Approve: {part.name}

+ + +
+ ); + } + return null; + }) + )} +
+ ); +} +``` + +## Example: Client Tools with Type Safety + +```tsx +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; +import { + clientTools, + createChatClientOptions, + type InferChatMessages +} from "@tanstack/ai-client"; +import { toolDefinition } from "@tanstack/ai"; +import { z } from "zod"; +import { useState } from "react"; + +const updateUIDef = toolDefinition({ + name: "updateUI", + description: "Update the UI with a notification", + inputSchema: z.object({ + message: z.string(), + type: z.string(), + }), + outputSchema: z.object({ success: z.boolean() }), +}); + +const saveToStorageDef = toolDefinition({ + name: "saveToStorage", + description: "Save a value to storage", + inputSchema: z.object({ + key: z.string(), + value: z.string(), + }), + outputSchema: z.object({ saved: z.boolean() }), +}); + +export function ChatWithClientTools() { + const [notification, setNotification] = useState<{ message: string; type: string } | null>(null); + + // Create client implementations + const updateUI = updateUIDef.client((input) => { + // ✅ input is fully typed! + setNotification({ message: input.message, type: input.type }); + return { success: true }; + }); + + const saveToStorage = saveToStorageDef.client((input) => { + localStorage.setItem(input.key, input.value); + return { saved: true }; + }); + + // Create typed tools array (no 'as const' needed!) + const tools = clientTools(updateUI, saveToStorage); + + const { messages, sendMessage } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + tools, // ✅ Automatic execution, full type safety + }); + + return ( +
+ {messages.map((message) => + message.parts.map((part) => { + if (part.type === "tool-call" && part.name === "updateUI") { + // ✅ part.input and part.output are fully typed! + return
Tool executed: {part.name}
; + } + return null; + }) + )} +
+ ); +} +``` + +## `createChatClientOptions(options)` + +Helper to create typed chat options (re-exported from `@tanstack/ai-client`). + +```typescript +import { + clientTools, + createChatClientOptions, + fetchServerSentEvents, + type InferChatMessages +} from "@tanstack/ai-client"; +import { tool1, tool2 } from "./tools"; + +// Create typed tools array (no 'as const' needed!) +const tools = clientTools(tool1, tool2); + +const chatOptions = createChatClientOptions({ + connection: fetchServerSentEvents("/api/chat"), + tools, +}); + +type Messages = InferChatMessages; +``` + +## Types + +Re-exported from `@tanstack/ai-client`: + +- `UIMessage` - Message type with tool type parameter +- `MessagePart` - Message part with tool type parameter +- `TextPart` - Text content part +- `ThinkingPart` - Thinking content part +- `ToolCallPart` - Tool call part (discriminated union) +- `ToolResultPart` - Tool result part +- `ChatClientOptions` - Chat client options with typed client runtime context +- `ConnectionAdapter` - Connection adapter interface +- `InferChatMessages` - Extract message type from options + +Re-exported from `@tanstack/ai`: + +- `toolDefinition()` - Create isomorphic tool definition +- `ToolDefinitionInstance` - Tool definition type +- `ClientTool` - Client tool type +- `ServerTool` - Server tool type + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about the isomorphic tool system +- [Client Tools](/ai/tools/client-tools) - Learn about client-side tools diff --git a/mintlify/ai/api/ai-solid.md b/mintlify/ai/api/ai-solid.md new file mode 100644 index 000000000..251c4448e --- /dev/null +++ b/mintlify/ai/api/ai-solid.md @@ -0,0 +1,375 @@ +--- +title: "@tanstack/ai-solid" +slug: /api/ai-solid +order: 4 +description: "API reference for @tanstack/ai-solid — SolidJS primitives including useChat for streaming chat with full type safety." +keywords: + - tanstack ai + - "@tanstack/ai-solid" + - solidjs + - solid + - useChat + - solid primitives + - api reference +--- + +SolidJS primitives for TanStack AI, providing convenient SolidJS bindings for the headless client. + +## Installation + +```bash +npm install @tanstack/ai-solid +``` + +## `useChat(options?)` + +Main primitive for managing chat state in SolidJS with full type safety. + +```tsx +import { useChat, fetchServerSentEvents } from "@tanstack/ai-solid"; +import { + clientTools, + createChatClientOptions, + type InferChatMessages +} from "@tanstack/ai-client"; +import { toolDefinition } from "@tanstack/ai"; +import { z } from "zod"; +import { createSignal } from "solid-js"; + +const updateUIDef = toolDefinition({ + name: "updateUI", + description: "Show a notification in the UI", + inputSchema: z.object({ message: z.string() }), +}); + +function ChatComponent() { + const [, setNotification] = createSignal(null); + // Create client tool implementations + const updateUI = updateUIDef.client((input) => { + setNotification(input.message); + return { success: true }; + }); + + // Create typed tools array (no 'as const' needed!) + const tools = clientTools(updateUI); + + const chatOptions = createChatClientOptions({ + connection: fetchServerSentEvents("/api/chat"), + tools, + }); + + // Fully typed messages! + type ChatMessages = InferChatMessages; + + const { messages, sendMessage, isLoading, error, addToolApprovalResponse } = + useChat(chatOptions); + + return
{/* Chat UI with typed messages */}
; +} +``` + +### Options + +Extends `ChatClientOptions` from `@tanstack/ai-client`: + +- `connection` - Connection adapter (required) +- `tools?` - Array of client tool implementations (with `.client()` method) +- `initialMessages?` - Initial messages array +- `id?` - Unique identifier for this chat instance +- `threadId?` - Thread ID for AG-UI run correlation. Persists across sends; auto-generated if omitted +- `forwardedProps?` - Arbitrary client-controlled JSON forwarded to the server in the AG-UI `RunAgentInput.forwardedProps` field (e.g., `{ provider: 'openai', model: 'gpt-4o' }`) +- `body?` - **Deprecated.** Use `forwardedProps` instead. Still works for backward compatibility; values are merged into `forwardedProps` on the wire +- `context?` - Typed client-local runtime context passed to client tool implementations. This value is not serialized to the server +- `onResponse?` - Callback when response is received +- `onChunk?` - Callback when stream chunk is received +- `onFinish?` - Callback when response finishes +- `onError?` - Callback when error occurs +- `streamProcessor?` - Stream processing configuration + +**Note:** Client tools are now automatically executed - no `onToolCall` callback needed! + +### Returns + +```typescript +import type { Accessor } from "solid-js"; +import type { UIMessage } from "@tanstack/ai-solid"; +import type { ModelMessage } from "@tanstack/ai/client"; + +interface UseChatReturn { + messages: Accessor; + sendMessage: (content: string) => Promise; + append: (message: ModelMessage | UIMessage) => Promise; + addToolResult: (result: { + toolCallId: string; + tool: string; + output: any; + state?: "output-available" | "output-error"; + errorText?: string; + }) => Promise; + addToolApprovalResponse: (response: { + id: string; + approved: boolean; + }) => Promise; + reload: () => Promise; + stop: () => void; + isLoading: Accessor; + error: Accessor; + setMessages: (messages: UIMessage[]) => void; + clear: () => void; +} +``` + +**Note:** Unlike React, `messages`, `isLoading`, and `error` are SolidJS `Accessor` functions, so you need to call them to get their values (e.g., `messages()` instead of just `messages`). + +## Connection Adapters + +Re-exported from `@tanstack/ai-client` for convenience: + +```typescript +import { + fetchServerSentEvents, + fetchHttpStream, + stream, + type ConnectionAdapter, +} from "@tanstack/ai-solid"; +``` + +## Example: Basic Chat + +```tsx +import { createSignal, For } from "solid-js"; +import { useChat, fetchServerSentEvents } from "@tanstack/ai-solid"; + +export function Chat() { + const [input, setInput] = createSignal(""); + + const { messages, sendMessage, isLoading } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + }); + + const handleSubmit = (e: Event) => { + e.preventDefault(); + if (input().trim() && !isLoading()) { + sendMessage(input()); + setInput(""); + } + }; + + return ( +
+
+ + {(message) => ( +
+ {message.role}: + + {(part) => { + if (part.type === "thinking") { + return ( +
+ 💭 Thinking: {part.content} +
+ ); + } + if (part.type === "text") { + return {part.content}; + } + return null; + }} +
+
+ )} +
+
+
+ setInput(e.currentTarget.value)} + disabled={isLoading()} + /> + +
+
+ ); +} +``` + +## Example: Tool Approval + +```tsx +import { For, Show } from "solid-js"; +import { useChat, fetchServerSentEvents } from "@tanstack/ai-solid"; + +export function ChatWithApproval() { + const { messages, sendMessage, addToolApprovalResponse } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + }); + + return ( +
+ + {(message) => ( + + {(part) => { + if ( + part.type === "tool-call" && + part.state === "approval-requested" && + part.approval + ) { + return ( +
+

Approve: {part.name}

+ + +
+ ); + } + return null; + }} +
+ )} +
+
+ ); +} +``` + +## Example: Client Tools with Type Safety + +```tsx +import { useChat, fetchServerSentEvents } from "@tanstack/ai-solid"; +import { + clientTools, + createChatClientOptions, + type InferChatMessages +} from "@tanstack/ai-client"; +import { toolDefinition } from "@tanstack/ai"; +import { z } from "zod"; +import { createSignal, For } from "solid-js"; + +const updateUIDef = toolDefinition({ + name: "updateUI", + description: "Show a notification in the UI", + inputSchema: z.object({ message: z.string(), type: z.string() }), +}); + +const saveToStorageDef = toolDefinition({ + name: "saveToStorage", + description: "Save a value to localStorage", + inputSchema: z.object({ key: z.string(), value: z.string() }), +}); + +export function ChatWithClientTools() { + const [notification, setNotification] = createSignal<{ message: string; type: string } | null>(null); + + // Create client implementations + const updateUI = updateUIDef.client((input) => { + // ✅ input is fully typed! + setNotification({ message: input.message, type: input.type }); + return { success: true }; + }); + + const saveToStorage = saveToStorageDef.client((input) => { + localStorage.setItem(input.key, input.value); + return { saved: true }; + }); + + // Create typed tools array (no 'as const' needed!) + const tools = clientTools(updateUI, saveToStorage); + + const { messages, sendMessage } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + tools, // ✅ Automatic execution, full type safety + }); + + return ( +
+ + {(message) => ( + + {(part) => { + if (part.type === "tool-call" && part.name === "updateUI") { + // ✅ part.input and part.output are fully typed! + return
Tool executed: {part.name}
; + } + return null; + }} +
+ )} +
+
+ ); +} +``` + +## `createChatClientOptions(options)` + +Helper to create typed chat options (re-exported from `@tanstack/ai-client`). + +```typescript +import { + clientTools, + createChatClientOptions, + type InferChatMessages +} from "@tanstack/ai-client"; +import { fetchServerSentEvents } from "@tanstack/ai-solid"; +import { tool1, tool2 } from "./tools"; + +// Create typed tools array (no 'as const' needed!) +const tools = clientTools(tool1, tool2); + +const chatOptions = createChatClientOptions({ + connection: fetchServerSentEvents("/api/chat"), + tools, +}); + +type Messages = InferChatMessages; +``` + +## Types + +Re-exported from `@tanstack/ai-client`: + +- `UIMessage` - Message type with tool type parameter +- `MessagePart` - Message part with tool type parameter +- `TextPart` - Text content part +- `ThinkingPart` - Thinking content part +- `ToolCallPart` - Tool call part (discriminated union) +- `ToolResultPart` - Tool result part +- `ChatClientOptions` - Chat client options with typed client runtime context +- `ConnectionAdapter` - Connection adapter interface +- `InferChatMessages` - Extract message type from options +- `ChatRequestBody` - Request body type + +Re-exported from `@tanstack/ai`: + +- `toolDefinition()` - Create isomorphic tool definition +- `ToolDefinitionInstance` - Tool definition type +- `ClientTool` - Client tool type +- `ServerTool` - Server tool type + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about the isomorphic tool system +- [Client Tools](/ai/tools/client-tools) - Learn about client-side tools diff --git a/mintlify/ai/api/ai-svelte.md b/mintlify/ai/api/ai-svelte.md new file mode 100644 index 000000000..ab5cd145b --- /dev/null +++ b/mintlify/ai/api/ai-svelte.md @@ -0,0 +1,386 @@ +--- +title: "@tanstack/ai-svelte" +id: ai-svelte +order: 6 +description: "API reference for @tanstack/ai-svelte — Svelte 5 reactive factory functions for streaming chat built on runes." +keywords: + - tanstack ai + - "@tanstack/ai-svelte" + - svelte + - svelte 5 + - createChat + - runes + - api reference +--- + +Svelte 5 bindings for TanStack AI, providing reactive factory functions for the headless client using Svelte runes. + +## Installation + +```bash +npm install @tanstack/ai-svelte +``` + +## `createChat(options)` + +Factory function for managing chat state in Svelte 5 with full type safety. + +```typescript +import { createChat, fetchServerSentEvents } from "@tanstack/ai-svelte"; +import { + clientTools, + createChatClientOptions, + type InferChatMessages, +} from "@tanstack/ai-client"; +import { toolDefinition } from "@tanstack/ai"; +import { z } from "zod"; + +const updateUIDef = toolDefinition({ + name: "updateUI", + description: "Update the UI with a notification", + inputSchema: z.object({ + message: z.string(), + }), + outputSchema: z.object({ success: z.boolean() }), +}); + +// In + +
+
+ {#each chat.messages as message (message.id)} +
+ {message.role}: + {#each message.parts as part, idx} + {#if part.type === "thinking"} +
+ Thinking: {part.content} +
+ {:else if part.type === "text"} + {part.content} + {/if} + {/each} +
+ {/each} +
+
+ + +
+
+``` + +## Example: Tool Approval + +```svelte + + +
+ {#each chat.messages as message (message.id)} + {#each message.parts as part} + {#if part.type === "tool-call" && part.state === "approval-requested" && part.approval} +
+

Approve: {part.name}

+ + +
+ {/if} + {/each} + {/each} +
+``` + +## Example: Client Tools with Type Safety + +```svelte + + +
+ {#each chat.messages as message (message.id)} + {#each message.parts as part} + {#if part.type === "tool-call" && part.name === "updateUI"} +
Tool executed: {part.name}
+ {/if} + {/each} + {/each} +
+``` + +## Generation Functions + +Factory functions for one-shot generation tasks (images, speech, transcription, summarization, video). All share the same pattern: provide a `connection` or `fetcher`, call `generate()`, and read reactive state. + +### `createGeneration(options)` + +Base factory for custom generation types. All specialized functions below are built on this. + +```typescript +import { createGeneration, fetchServerSentEvents } from "@tanstack/ai-svelte"; + +const gen = createGeneration({ + connection: fetchServerSentEvents("/api/generate/custom"), +}); + +// gen.generate({ prompt: 'Hello' }) +// gen.result, gen.isLoading, gen.error, gen.status +``` + +**Options:** `connection?`, `fetcher?`, `id?`, `body?`, `onResult?`, `onError?`, `onProgress?`, `onChunk?` + +**Returns:** `generate`, `result`, `isLoading`, `error`, `status`, `stop`, `reset`, `updateBody` -- all state properties are reactive getters. + +### `createGenerateImage(options)` + +Image generation factory. `generate()` accepts `ImageGenerateInput`, result is `ImageGenerationResult`. + +### `createGenerateSpeech(options)` + +Text-to-speech factory. `generate()` accepts `SpeechGenerateInput`, result is `TTSResult`. + +### `createTranscription(options)` + +Audio transcription factory. `generate()` accepts `TranscriptionGenerateInput`, result is `TranscriptionResult`. + +### `createSummarize(options)` + +Text summarization factory. `generate()` accepts `SummarizeGenerateInput`, result is `SummarizationResult`. + +### `createGenerateVideo(options)` + +Video generation factory with job polling. Returns additional `jobId` and `videoStatus` reactive getters. Accepts extra `onJobCreated?` and `onStatusUpdate?` callbacks. + +No generation function includes automatic cleanup. Call `.stop()` manually when done. + +## `createChatClientOptions(options)` + +Helper to create typed chat options (re-exported from `@tanstack/ai-client`). + +```typescript +import { + clientTools, + createChatClientOptions, + fetchServerSentEvents, + type InferChatMessages, +} from "@tanstack/ai-client"; +import { tool1, tool2 } from "./tools"; + +// Create typed tools array (no 'as const' needed!) +const tools = clientTools(tool1, tool2); + +const chatOptions = createChatClientOptions({ + connection: fetchServerSentEvents("/api/chat"), + tools, +}); + +type Messages = InferChatMessages; +``` + +## Types + +Re-exported from `@tanstack/ai-client`: + +- `UIMessage` - Message type with tool type parameter +- `MessagePart` - Message part with tool type parameter +- `TextPart` - Text content part +- `ThinkingPart` - Thinking content part +- `ToolCallPart` - Tool call part (discriminated union) +- `ToolResultPart` - Tool result part +- `ChatClientOptions` - Chat client options with typed client runtime context +- `ConnectionAdapter` - Connection adapter interface +- `InferChatMessages` - Extract message type from options +- `ChatRequestBody` - Request body type +- `GenerationClientState` - Generation lifecycle state +- `ImageGenerateInput` - Image generation input type +- `SpeechGenerateInput` - Speech generation input type +- `TranscriptionGenerateInput` - Transcription input type +- `SummarizeGenerateInput` - Summarization input type +- `VideoGenerateInput` - Video generation input type +- `VideoGenerateResult` - Video generation result type +- `VideoStatusInfo` - Video job status info + +Re-exported from `@tanstack/ai`: + +- `toolDefinition()` - Create isomorphic tool definition +- `ToolDefinitionInstance` - Tool definition type +- `ClientTool` - Client tool type +- `ServerTool` - Server tool type + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about the isomorphic tool system +- [Client Tools](/ai/tools/client-tools) - Learn about client-side tools diff --git a/mintlify/ai/api/ai-vue.md b/mintlify/ai/api/ai-vue.md new file mode 100644 index 000000000..68a3a72c2 --- /dev/null +++ b/mintlify/ai/api/ai-vue.md @@ -0,0 +1,403 @@ +--- +title: "@tanstack/ai-vue" +id: ai-vue +order: 5 +description: "API reference for @tanstack/ai-vue — Vue 3 composables including useChat for streaming chat with full type safety." +keywords: + - tanstack ai + - "@tanstack/ai-vue" + - vue + - vue 3 + - useChat + - composables + - api reference +--- + +Vue composables for TanStack AI, providing convenient Vue 3 bindings for the headless client. + +## Installation + +```bash +npm install @tanstack/ai-vue +``` + +## `useChat(options?)` + +Main composable for managing chat state in Vue with full type safety. + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-vue"; +import { + clientTools, + createChatClientOptions, + type InferChatMessages, +} from "@tanstack/ai-client"; +import { toolDefinition } from "@tanstack/ai"; +import { z } from "zod"; +import { ref } from "vue"; + +const updateUIDef = toolDefinition({ + name: "updateUI", + description: "Show a notification in the UI", + inputSchema: z.object({ message: z.string() }), +}); + +const notification = ref(null); + +// In + + +``` + +## Example: Tool Approval + +```vue + + + +``` + +## Example: Client Tools with Type Safety + +```vue + + + +``` + +## Generation Composables + +Vue composables for one-shot generation tasks (images, speech, transcription, summarization, video). All share the same pattern: provide a `connection` or `fetcher`, call `generate()`, and read reactive state. + +### `useGeneration(options)` + +Base composable for custom generation types. All specialized composables below are built on this. + +```typescript +import { useGeneration } from "@tanstack/ai-vue"; +import { fetchServerSentEvents } from "@tanstack/ai-client"; + +const { generate, result, isLoading, error, status, stop, reset } = + useGeneration({ + connection: fetchServerSentEvents("/api/generate/custom"), + }); +``` + +**Options:** `connection?`, `fetcher?`, `id?`, `body?`, `onResult?`, `onError?`, `onProgress?`, `onChunk?` + +**Returns:** `generate`, `result`, `isLoading`, `error`, `status`, `stop`, `reset` -- all reactive state is `DeepReadonly>`. + +### `useGenerateImage(options)` + +Image generation composable. `generate()` accepts `ImageGenerateInput`, result is `ImageGenerationResult`. + +### `useGenerateSpeech(options)` + +Text-to-speech composable. `generate()` accepts `SpeechGenerateInput`, result is `TTSResult`. + +### `useTranscription(options)` + +Audio transcription composable. `generate()` accepts `TranscriptionGenerateInput`, result is `TranscriptionResult`. + +### `useSummarize(options)` + +Text summarization composable. `generate()` accepts `SummarizeGenerateInput`, result is `SummarizationResult`. + +### `useGenerateVideo(options)` + +Video generation composable with job polling. Returns additional `jobId` and `videoStatus` refs. Accepts extra `onJobCreated?` and `onStatusUpdate?` callbacks. + +All generation composables automatically clean up via `onScopeDispose`. + +## `createChatClientOptions(options)` + +Helper to create typed chat options (re-exported from `@tanstack/ai-client`). + +```typescript +import { + clientTools, + createChatClientOptions, + type InferChatMessages, +} from "@tanstack/ai-client"; +import { fetchServerSentEvents } from "@tanstack/ai-vue"; +import { tool1, tool2 } from "./tools"; + +// Create typed tools array (no 'as const' needed!) +const tools = clientTools(tool1, tool2); + +const chatOptions = createChatClientOptions({ + connection: fetchServerSentEvents("/api/chat"), + tools, +}); + +type Messages = InferChatMessages; +``` + +## Types + +Re-exported from `@tanstack/ai-client`: + +- `UIMessage` - Message type with tool type parameter +- `MessagePart` - Message part with tool type parameter +- `TextPart` - Text content part +- `ThinkingPart` - Thinking content part +- `ToolCallPart` - Tool call part (discriminated union) +- `ToolResultPart` - Tool result part +- `ChatClientOptions` - Chat client options with typed client runtime context +- `ConnectionAdapter` - Connection adapter interface +- `InferChatMessages` - Extract message type from options +- `ChatRequestBody` - Request body type +- `GenerationClientState` - Generation lifecycle state +- `ImageGenerateInput` - Image generation input type +- `SpeechGenerateInput` - Speech generation input type +- `TranscriptionGenerateInput` - Transcription input type +- `SummarizeGenerateInput` - Summarization input type +- `VideoGenerateInput` - Video generation input type +- `VideoGenerateResult` - Video generation result type +- `VideoStatusInfo` - Video job status info + +Re-exported from `@tanstack/ai`: + +- `toolDefinition()` - Create isomorphic tool definition +- `ToolDefinitionInstance` - Tool definition type +- `ClientTool` - Client tool type +- `ServerTool` - Server tool type + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about the isomorphic tool system +- [Client Tools](/ai/tools/client-tools) - Learn about client-side tools diff --git a/mintlify/ai/api/ai.md b/mintlify/ai/api/ai.md new file mode 100644 index 000000000..65a3dad53 --- /dev/null +++ b/mintlify/ai/api/ai.md @@ -0,0 +1,551 @@ +--- +title: "@tanstack/ai" +id: tanstack-ai-api +order: 1 +description: "API reference for @tanstack/ai — the core TanStack AI library providing chat(), generateImage(), toolDefinition(), and streaming utilities." +keywords: + - tanstack ai + - "@tanstack/ai" + - api reference + - chat + - toolDefinition + - generateImage + - core library +--- + +The core AI library for TanStack AI. + +## Installation + +```bash +npm install @tanstack/ai +``` + +## `chat(options)` + +Creates a streaming chat response. + +```typescript +import { chat, maxIterations } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { myTool } from "./tools"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Hello!" }], + tools: [myTool], + systemPrompts: ["You are a helpful assistant"], + agentLoopStrategy: maxIterations(20), +}); +``` + +### Parameters + +- `adapter` - An AI adapter instance with model (e.g., `openaiText('gpt-5.2')`, `anthropicText('claude-sonnet-4-5')`) +- `messages` - Array of chat messages. Accepts mixed `UIMessage | ModelMessage` arrays — internal conversion handles AG-UI fan-out dedup, drops `reasoning`/`activity`, and collapses `developer` → `system` +- `tools?` - Array of tools for function calling +- `context?` - Typed runtime context passed to server tools and middleware. If a tool or middleware declares a concrete context type, `chat()` requires a compatible value here +- `systemPrompts?` - System prompts to prepend to messages +- `agentLoopStrategy?` - Strategy for agent loops (default: `maxIterations(5)`) +- `abortController?` - AbortController for cancellation +- `modelOptions?` - Provider-native model options. This is where sampling parameters live — `temperature`, `top_p`/`topP`, and the provider's token-limit key (`max_output_tokens`, `max_tokens`, `maxOutputTokens`, …) — under each provider's canonical name, rather than as generic root-level props. See [Moving Sampling Options into modelOptions](/ai/migration/sampling-options-to-model-options). (Renamed from `providerOptions`.) +- `threadId?` - AG-UI thread identifier propagated into `RUN_STARTED` events for run correlation +- `runId?` - AG-UI run identifier (auto-generated if omitted) +- `parentRunId?` - AG-UI parent run identifier for nested runs + +### Returns + +An async iterable of `StreamChunk`. + +## `summarize(options)` + +Creates a text summarization. + +```typescript +import { summarize } from "@tanstack/ai"; +import { openaiSummarize } from "@tanstack/ai-openai"; + +const result = await summarize({ + adapter: openaiSummarize("gpt-5.2"), + text: "Long text to summarize...", + maxLength: 100, + style: "concise", +}); +``` + +### Parameters + +- `adapter` - An AI adapter instance with model +- `text` - Text to summarize +- `maxLength?` - Maximum length of summary +- `style?` - Summary style ("concise" | "detailed") +- `modelOptions?` - Model-specific options + +### Returns + +A `SummarizationResult` with the summary text. + +## `toolDefinition(config)` + +Creates an isomorphic tool definition that can be instantiated for server or client execution. + +```typescript +import { chat, toolDefinition } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { z } from "zod"; + +const myToolDef = toolDefinition({ + name: "my_tool", + description: "Tool description", + inputSchema: z.object({ + param: z.string(), + }), + outputSchema: z.object({ + result: z.string(), + }), + needsApproval: false, // Optional +}); + +// Or create client implementation +const myClientTool = myToolDef.client(async ({ param }) => { + // Client-side implementation + return { result: "..." }; +}); + +// Use directly in chat() (server-side, no execute) +chat({ + adapter: openaiText("gpt-5.2"), + tools: [myToolDef], + messages: [{ role: "user", content: "..." }], +}); + +// Or create server implementation +const myServerTool = myToolDef.server(async ({ param }) => { + // Server-side implementation + return { result: "..." }; +}); + +// Use directly in chat() (server-side, no execute) +chat({ + adapter: openaiText("gpt-5.2"), + tools: [myServerTool], + messages: [{ role: "user", content: "..." }], +}); +``` + +Tools can declare typed runtime context for request-scoped dependencies: + +```typescript +import { chat, toolDefinition, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { session, db } from "./app"; + +type AppContext = { + userId: string; + db: { users: { findName(id: string): Promise } }; +}; + +const currentUser = toolDefinition({ + name: "current_user", + description: "Get the current user", +}).server(async (_input: unknown, ctx) => { + return { name: await ctx.context.db.users.findName(ctx.context.userId) }; +}); + +export async function POST(request: Request) { + const { messages } = await request.json(); + const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages, + tools: [currentUser], + context: { userId: session.user.id, db }, + }); + return toServerSentEventsResponse(stream); +} +``` + +### Parameters + +- `name` - Tool name (must be unique) +- `description` - Tool description for the model +- `inputSchema` - Zod schema for input validation +- `outputSchema?` - Zod schema for output validation +- `needsApproval?` - Whether tool requires user approval +- `metadata?` - Additional metadata + +### Returns + +A `ToolDefinition` object with `.server()` and `.client()` methods for creating concrete implementations. + +## `toServerSentEventsStream(stream, abortController?)` + +Converts a stream to a ReadableStream in Server-Sent Events format. + +```typescript +import { chat, toServerSentEventsStream } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Hello!" }], +}); +const readableStream = toServerSentEventsStream(stream); +``` + +### Parameters + +- `stream` - Async iterable of `StreamChunk` +- `abortController?` - Optional AbortController to abort when stream is cancelled + +### Returns + +A `ReadableStream` in Server-Sent Events format. Each chunk is: +- Prefixed with `"data: "` +- Followed by `"\n\n"` +- Stream ends with `"data: [DONE]\n\n"` + +## `toServerSentEventsResponse(stream, init?)` + +Converts a stream to an HTTP Response with proper SSE headers. + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +async function POST() { + const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Hello!" }], + }); + return toServerSentEventsResponse(stream); +} +``` + +### Parameters + +- `stream` - Async iterable of `StreamChunk` +- `init?` - Optional ResponseInit options (including `abortController`) + +### Returns + +A `Response` object suitable for HTTP endpoints with SSE headers (`Content-Type: text/event-stream`, `Cache-Control: no-cache`, `Connection: keep-alive`). + +## `chatParamsFromRequest(req)` + +Reads an HTTP `Request`, parses its JSON body, and validates it against AG-UI `RunAgentInputSchema`. Returns parsed chat parameters ready to spread into `chat()`. On a malformed body, **throws a 400 `Response`** that frameworks like TanStack Start, SolidStart, Remix, and React Router 7 return to the client automatically. + +```typescript +import { chat, chatParamsFromRequest, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { serverTools } from "./tools"; + +export async function POST(req: Request) { + const params = await chatParamsFromRequest(req); + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: params.messages, + tools: serverTools, + }); + return toServerSentEventsResponse(stream); +} +``` + +### Parameters + +- `req` - An incoming `Request` whose JSON body conforms to AG-UI `RunAgentInput` + +### Returns + +A promise resolving to `{ messages, threadId, runId, parentRunId?, tools, forwardedProps, state, aguiContext, context }`. + +The returned `aguiContext` is the AG-UI protocol `RunAgentInput.context` field. It is not the same as TanStack AI runtime `chat({ context })`; validate and map it explicitly if you want those values available to tools or middleware. + +The returned `context` field is a deprecated alias of `aguiContext` kept for backward compatibility. Prefer `aguiContext` in new code. + +> **Framework note.** Next.js Route Handlers, SvelteKit, Hono, and raw Node do not auto-handle thrown `Response` objects. In those, wrap with try/catch or use `chatParamsFromRequestBody(await req.json())` directly. + +## `chatParamsFromRequestBody(body)` + +Lower-level variant of `chatParamsFromRequest` that validates an already-parsed body. Rejects with an `AGUIError` on malformed input. Use this when you need explicit error handling control. + +```typescript +import { chatParamsFromRequestBody } from "@tanstack/ai"; + +async function handler(req: Request): Promise { + const body = await req.json(); + try { + const params = await chatParamsFromRequestBody(body); + // ... + return new Response("ok"); + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + return new Response(message, { status: 400 }); + } +} +``` + +## `mergeAgentTools(serverTools, clientTools)` + +Merges a server-side tool registry with the AG-UI client-declared tools received in the request payload. Server tools win on name collision; client-only tools become no-execute stubs that the runtime dispatches via `ClientToolRequest` events. + +```typescript +import { chat, chatParamsFromRequest, mergeAgentTools } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { serverTools } from "./tools"; + +async function handler(req: Request) { + const params = await chatParamsFromRequest(req); + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: params.messages, + tools: mergeAgentTools(serverTools, params.tools), + }); +} +``` + +### Parameters + +- `serverTools` - The server's `toolDefinition().server(...)` registry, keyed by tool name +- `clientTools` - The `tools` array from `chatParamsFromRequest`'s return value + +### Returns + +A merged tool record suitable for `chat({ tools })`. + +## `maxIterations(count)` + +Creates an agent loop strategy that limits iterations. + +```typescript +import { chat, maxIterations } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Hello!" }], + agentLoopStrategy: maxIterations(20), +}); +``` + +### Parameters + +- `count` - Maximum number of tool execution iterations + +### Returns + +An `AgentLoopStrategy` object. + +## Types + +### `ModelMessage` + +```typescript +interface ModelMessage { + role: "user" | "assistant" | "system" | "tool"; + content: string; + toolCallId?: string; +} +``` + +### `StreamChunk` + +```typescript ignore +type StreamChunk = + | ContentStreamChunk + | ThinkingStreamChunk + | ToolCallStreamChunk + | ToolResultStreamChunk + | DoneStreamChunk + | ErrorStreamChunk; + +interface ThinkingStreamChunk { + type: "thinking"; + id: string; + model: string; + timestamp: number; + delta?: string; // Incremental thinking token + content: string; // Accumulated thinking content +} +``` + +Stream chunks represent different types of data in the stream: + +- **Content chunks** - Text content being generated +- **Thinking chunks** - Model's reasoning process (when supported by the model) +- **Tool call chunks** - When the model calls a tool +- **Tool result chunks** - Results from tool execution +- **Done chunks** - Stream completion +- **Error chunks** - Stream errors + +### `Tool` + +```typescript +import type { SchemaInput, ToolExecutionContext } from "@tanstack/ai"; + +interface Tool { + name: string; + description: string; + inputSchema?: SchemaInput; + outputSchema?: SchemaInput; + execute?: ( + args: any, + context?: ToolExecutionContext + ) => Promise | any; + needsApproval?: boolean; + lazy?: boolean; + metadata?: Record; +} +``` + +### `ToolExecutionContext` + +```typescript ignore +type ToolExecutionContext = { + toolCallId?: string; + emitCustomEvent: (eventName: string, value: Record) => void; +} & (unknown extends TContext ? { context?: TContext } : { context: TContext }); +``` + +`context` is the runtime value from `chat({ context })` for server tools, or from `ChatClient` / framework hook options for client tools. It is required when a tool declares a concrete `TContext` and optional for untyped tools where the context type is `unknown`. + +### `ChatMiddleware` + +```typescript +import type { + StreamChunk, + ChatMiddlewarePhase, + ToolCallHookContext, + BeforeToolCallDecision, + AfterToolCallInfo, + FinishInfo, + AbortInfo, + ErrorInfo, +} from "@tanstack/ai"; + +interface ChatMiddlewareContext { + requestId: string; + streamId: string; + threadId: string; + phase: ChatMiddlewarePhase; + iteration: number; + context: TContext; + abort(reason?: string): void; + defer(promise: Promise): void; +} + +interface ChatMiddleware { + name?: string; + onStart?: (ctx: ChatMiddlewareContext) => void | Promise; + onChunk?: ( + ctx: ChatMiddlewareContext, + chunk: StreamChunk + ) => void | StreamChunk | StreamChunk[] | null | Promise; + onBeforeToolCall?: ( + ctx: ChatMiddlewareContext, + hookCtx: ToolCallHookContext + ) => BeforeToolCallDecision | Promise; + onAfterToolCall?: ( + ctx: ChatMiddlewareContext, + info: AfterToolCallInfo + ) => void | Promise; + onFinish?: ( + ctx: ChatMiddlewareContext, + info: FinishInfo + ) => void | Promise; + onAbort?: ( + ctx: ChatMiddlewareContext, + info: AbortInfo + ) => void | Promise; + onError?: ( + ctx: ChatMiddlewareContext, + info: ErrorInfo + ) => void | Promise; +} +``` + +See [Runtime Context](/ai/advanced/runtime-context) for the recommended context patterns. + +## Usage Examples + +```typescript +import { chat, summarize, generateImage, toolDefinition } from "@tanstack/ai"; +import { + openaiText, + openaiSummarize, + openaiImage, +} from "@tanstack/ai-openai"; +import { z } from "zod"; + +// --- Streaming chat +const stream = chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Hello!" }], +}); + +// --- Structured response with tools +const weatherTool = toolDefinition({ + name: "getWeather", + description: "Get the current weather for a city", + inputSchema: z.object({ + city: z.string(), + }), +}).server(async ({ city }) => { + // Implementation that fetches weather info + return JSON.stringify({ temperature: 72, condition: "Sunny" }); +}); + +async function examples() { + // --- One-shot chat response (stream: false) + const response = await chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "What's the capital of France?" }], + stream: false, // Returns a Promise instead of AsyncIterable + }); + + // --- Structured response with outputSchema + const parsed = await chat({ + adapter: openaiText("gpt-5.2"), + messages: [{ role: "user", content: "Summarize this text in JSON with keys 'summary' and 'keywords': ... " }], + outputSchema: z.object({ + summary: z.string(), + keywords: z.array(z.string()), + }), + }); + + const toolResult = await chat({ + adapter: openaiText("gpt-5.2"), + messages: [ + { role: "user", content: "What's the weather in Paris?" } + ], + tools: [weatherTool], + outputSchema: z.object({ + answer: z.string(), + weather: z.object({ + temperature: z.number(), + condition: z.string(), + }), + }), + }); + + // --- Summarization + const summary = await summarize({ + adapter: openaiSummarize("gpt-5.2"), + text: "Long text to summarize...", + maxLength: 100, + }); + + // --- Image generation + const image = await generateImage({ + adapter: openaiImage("dall-e-3"), + prompt: "A futuristic city skyline at sunset", + numberOfImages: 1, + size: "1024x1024", + }); +} +``` + +## Next Steps + +- [Getting Started](/ai/getting-started/quick-start) - Learn the basics +- [Tools Guide](/ai/tools/tools) - Learn about tools +- [Adapters](/ai/adapters/openai) - Explore adapter options diff --git a/mintlify/ai/architecture/approval-flow-processing.md b/mintlify/ai/architecture/approval-flow-processing.md new file mode 100644 index 000000000..04632bbe6 --- /dev/null +++ b/mintlify/ai/architecture/approval-flow-processing.md @@ -0,0 +1,448 @@ +--- +title: Approval Flow Processing Architecture +id: approval-flow-processing +description: "Internal architecture of TanStack AI's tool approval system — state machine, streaming protocol, concurrency control, and chained approval mechanics." +keywords: + - tanstack ai + - approval flow + - tool approval + - architecture + - state machine + - streaming protocol + - internals + - concurrency +--- + +# Approval Flow Processing Architecture + +> Internal architecture reference for the tool approval system in TanStack AI. +> Covers the full lifecycle from stream event to continuation, with emphasis on +> concurrency control and the chained approval mechanism. + +--- + +## Table of Contents + +1. [Overview](#overview) +2. [Component Responsibilities](#component-responsibilities) +3. [Type System](#type-system) +4. [State Machine](#state-machine) +5. [Single Approval Lifecycle](#single-approval-lifecycle) +6. [Chained Approvals and Continuation Control](#chained-approvals-and-continuation-control) +7. [Stream Event Protocol](#stream-event-protocol) +8. [Key Source Files](#key-source-files) + +--- + +## Overview + +The approval flow allows tools marked with `needsApproval: true` to pause +execution until the user explicitly approves or denies the action. This +creates a human-in-the-loop checkpoint for sensitive operations (sending +emails, making purchases, deleting data). + +The flow spans three layers: + +```mermaid +flowchart TD + A[Server - TextEngine] + B[StreamProcessor] + C[ChatClient] + + A -- "AG-UI stream · SSE / HTTP" --> B + B -- "events" --> C +``` + +| Layer | Responsibility | +|-------|----------------| +| **Server (TextEngine)** | `chat()` detects `needsApproval` → emits CUSTOM `approval-requested` instead of executing the tool | +| **StreamProcessor** | Receives CUSTOM chunk → `updateToolCallApproval()` → fires `onApprovalRequest` callback | +| **ChatClient** | Exposes `addToolApprovalResponse()` → updates message state → triggers `checkForContinuation()` → sends new stream | + +Framework hooks (`useChat` in React, Solid, Vue, Svelte) delegate to +`ChatClient`, which owns all concurrency and continuation logic. + +--- + +## Component Responsibilities + +### TextEngine (server) + +- Runs the agent loop: calls the LLM adapter, accumulates tool calls, executes + tools, and re-invokes the adapter with results. +- When a tool has `needsApproval: true`, the engine emits a + `CUSTOM { name: "approval-requested" }` event instead of executing the tool. +- The stream ends with `RUN_FINISHED { finishReason: "tool_calls" }` so the + client knows tools are pending. + +### StreamProcessor (`packages/ai/src/activities/chat/stream/processor.ts`) + +- Single source of truth for `UIMessage[]` state. +- On `approval-requested` custom event: + 1. Calls `updateToolCallApproval()` to set the tool-call part's state to + `approval-requested` and attach approval metadata. + 2. Fires `onApprovalRequest` so the ChatClient can emit devtools events. +- On `addToolApprovalResponse(approvalId, approved)`: + 1. Calls `updateToolCallApprovalResponse()` to set state to + `approval-responded` and record `approval.approved`. +- Provides `areAllToolsComplete()` which the ChatClient uses to decide whether + to auto-continue the conversation. + +### ChatClient (`packages/ai-client/src/chat-client.ts`) + +- Owns the streaming lifecycle (`streamResponse`), post-stream action queue, + and continuation control flags. +- Exposes `addToolApprovalResponse()` as the public API for responding to + approval requests. +- Manages two critical flags for continuation: + - `continuationPending` — prevents concurrent `streamResponse` calls. + - `continuationSkipped` — detects when a queued continuation check was + suppressed by `continuationPending` and needs re-evaluation. + +### Framework Hooks (React `useChat`, Solid `useChat`, etc.) + +- Wrap `ChatClient` methods in framework-specific reactive primitives. +- Expose `addToolApprovalResponse` directly to the component. +- No approval-specific logic beyond delegation. + +--- + +## Type System + +### ToolCallState + +```typescript group=approval-flow-processing +type ToolCallState = + | 'awaiting-input' // TOOL_CALL_START received, no arguments yet + | 'input-streaming' // Partial arguments being received + | 'input-complete' // All arguments received (TOOL_CALL_END) + | 'approval-requested' // Waiting for user approval + | 'approval-responded' // User has approved or denied +``` + +### ToolCallPart (approval-relevant fields) + +```typescript group=approval-flow-processing +interface ToolCallPart { + type: 'tool-call' + id: string // Unique tool call ID + name: string // Tool name + arguments: string // JSON string of arguments + state: ToolCallState + + approval?: { + id: string // Unique approval ID (NOT the toolCallId) + needsApproval: boolean // Always true when present + approved?: boolean // undefined until user responds + } + + output?: any // Set after execution (client tools) +} +``` + +### Key distinction: approval ID vs tool call ID + +The `approval.id` is a separate identifier generated per approval request. +All user-facing APIs (`addToolApprovalResponse`) use the **approval ID**, +not the tool call ID. This allows the system to correlate approval responses +even when multiple tools share similar call IDs across different messages. + +--- + +## State Machine + +```mermaid +flowchart TD + S1[awaiting-input] + S2[input-streaming] + S3[input-complete] + S4[approval-requested] + S5([execute directly]) + S6[approval-responded] + S7([execute tool]) + S8([cancelled]) + + S1 -- "TOOL_CALL_START" --> S2 + S2 -- "TOOL_CALL_ARGS" --> S3 + S3 -- "TOOL_CALL_END · needsApproval: true" --> S4 + S3 -- "TOOL_CALL_END · needsApproval: false" --> S5 + S4 -- "addToolApprovalResponse()" --> S6 + S6 -- "approved: true" --> S7 + S6 -- "approved: false" --> S8 +``` + +### Terminal states for `areAllToolsComplete()` + +A tool call is considered complete (and eligible for auto-continuation) when +any of the following is true: + +1. `state === 'approval-responded'` — user approved or denied +2. `output !== undefined && !approval` — client tool finished (no approval flow) +3. A corresponding `tool-result` part exists — server tool finished + +--- + +## Single Approval Lifecycle + +Step-by-step flow for a single tool requiring approval: + +### 1. Server emits approval request during stream + +``` +TOOL_CALL_START { toolCallId: "tc-1", toolName: "send_email" } +TOOL_CALL_ARGS { toolCallId: "tc-1", delta: '{"to":"..."}' } +TOOL_CALL_END { toolCallId: "tc-1" } +CUSTOM { name: "approval-requested", value: { + toolCallId: "tc-1", + toolName: "send_email", + input: { to: "..." }, + approval: { id: "appr-1", needsApproval: true } + }} +RUN_FINISHED { finishReason: "tool_calls" } +``` + +### 2. StreamProcessor processes the CUSTOM chunk + +``` +handleCustomEvent(): + 1. updateToolCallApproval(messages, messageId, "tc-1", "appr-1") + → Sets part.state = "approval-requested" + → Sets part.approval = { id: "appr-1", needsApproval: true } + 2. emitMessagesChange() + 3. fires onApprovalRequest({ toolCallId, toolName, input, approvalId }) +``` + +### 3. Stream ends, ChatClient processes + +``` +streamResponse() finally block: + 1. setIsLoading(false) + 2. drainPostStreamActions() → (nothing queued) + 3. streamCompletedSuccessfully check: + lastPart is tool-call (not tool-result) → no auto-continue + → Returns to caller (sendMessage resolves) +``` + +The conversation is now paused. The UI renders the approval prompt. + +### 4. User approves + +``` +addToolApprovalResponse({ id: "appr-1", approved: true }): + 1. processor.addToolApprovalResponse("appr-1", true) + → updateToolCallApprovalResponse(): + part.approval.approved = true + part.state = "approval-responded" + 2. isLoading is false → call checkForContinuation() directly +``` + +### 5. Continuation + +``` +checkForContinuation(): + 1. continuationPending = false, isLoading = false → proceed + 2. shouldAutoSend() → areAllToolsComplete(): + part.state === "approval-responded" → true + 3. continuationPending = true + 4. streamResponse() → new stream to server with approval in messages + 5. Server sees approval, executes tool, returns result + LLM response + 6. continuationPending = false +``` + +--- + +## Chained Approvals and Continuation Control + +The most complex scenario: a continuation stream produces **another** tool call +that also needs approval, and the user responds to it while the stream is still +active. + +### The Problem + +``` +Timeline: +───────────────────────────────────────────────────────────── + +1. User approves tool A + └─ checkForContinuation() sets continuationPending = true + └─ streamResponse() starts (stream 2) + +2. Stream 2 produces tool B needing approval + └─ approval-requested chunk processed + └─ UI shows approval prompt for tool B + +3. User approves tool B WHILE stream 2 is still active + └─ addToolApprovalResponse(): + └─ processor state updated (approval-responded) + └─ isLoading is true → queues checkForContinuation + +4. Stream 2 ends + └─ streamResponse() finally block: + └─ setIsLoading(false) + └─ drainPostStreamActions(): + └─ Runs queued checkForContinuation() + └─ BUT continuationPending is STILL TRUE (from step 1) + └─ *** EARLY RETURN — approval swallowed *** + └─ Returns to step 1's checkForContinuation() + └─ continuationPending = false + +5. Nobody re-checks → tool B's approval is lost +``` + +### The Solution: `continuationSkipped` Flag + +Two flags work together to handle this: + +- **`continuationPending`** — prevents concurrent `streamResponse()` calls. + Set to `true` when entering `checkForContinuation`'s streaming path, cleared + in the `finally` block. + +- **`continuationSkipped`** — set to `true` whenever `checkForContinuation()` + returns early due to `continuationPending` or `isLoading` being true. + Checked after `continuationPending` is cleared to trigger a re-evaluation. + +```typescript +class ChatClient { + private continuationPending = false + private continuationSkipped = false + private isLoading = false + + private shouldAutoSend(): boolean { return false } + private async streamResponse(): Promise {} + + private async checkForContinuation(): Promise { + if (this.continuationPending || this.isLoading) { + this.continuationSkipped = true // ← Mark that a check was suppressed + return + } + + if (this.shouldAutoSend()) { + this.continuationPending = true + this.continuationSkipped = false // ← Reset before entering stream + try { + await this.streamResponse() + } finally { + this.continuationPending = false + } + // If a check was skipped during the stream, re-evaluate now + if (this.continuationSkipped) { + this.continuationSkipped = false + await this.checkForContinuation() // ← Recurse safely + } + } + } +} +``` + +### Why the recursion is safe + +The recursion terminates because: + +1. **`continuationSkipped` is only set when a real check was suppressed.** After + the final stream (e.g., a text-only response), no new approvals arrive, so + `continuationSkipped` stays `false` and the recursion stops. + +2. **`shouldAutoSend()` returns `false` when tools are still pending approval.** + If a new approval arrives that hasn't been responded to yet, `areAllToolsComplete()` + returns `false` and the method exits without streaming. + +3. **Each recursion level sets `continuationPending = true`**, preventing any + concurrent checks from entering the streaming path. + +### Corrected Timeline + +``` +Timeline (with fix): +───────────────────────────────────────────────────────────── + +1. User approves tool A + └─ checkForContinuation() [OUTER] + └─ continuationPending = true, continuationSkipped = false + └─ streamResponse() starts (stream 2) + +2. Stream 2 produces tool B, user approves during stream + └─ Queues checkForContinuation as post-stream action + +3. Stream 2 ends + └─ drainPostStreamActions(): + └─ checkForContinuation(): continuationPending is true + └─ continuationSkipped = true ← MARKED + └─ returns early + └─ Back in OUTER: continuationPending = false + +4. OUTER checks continuationSkipped → true + └─ continuationSkipped = false + └─ Recurses into checkForContinuation() [INNER] + └─ shouldAutoSend() → true (tool B is approval-responded) + └─ continuationPending = true + └─ streamResponse() → stream 3 (final text response) + └─ continuationPending = false + └─ continuationSkipped is false → no further recursion + +5. Done. All three streams completed correctly. +``` + +--- + +## Stream Event Protocol + +### Approval-related AG-UI events + +These are `CUSTOM` events emitted by the TextEngine, not by adapters directly. + +#### `approval-requested` + +Emitted when a tool with `needsApproval: true` has its arguments finalized. + +```typescript ignore +{ + type: 'CUSTOM', + name: 'approval-requested', + value: { + toolCallId: string, // ID of the tool call + toolName: string, // Name of the tool + input: any, // Parsed arguments + approval: { + id: string, // Unique approval ID + needsApproval: true + } + } +} +``` + +**Processor handling:** `handleCustomEvent()` → `updateToolCallApproval()` → +`onApprovalRequest` callback. + +#### Relation to other tool events + +A complete approval tool call in the stream looks like: + +``` +TOOL_CALL_START → creates tool-call part (state: awaiting-input) +TOOL_CALL_ARGS* → accumulates arguments (state: input-streaming) +TOOL_CALL_END → finalizes arguments (state: input-complete) +CUSTOM → approval-requested (state: approval-requested) +RUN_FINISHED → finishReason: "tool_calls" +``` + +After the stream ends and the user responds, the ChatClient: +1. Updates the tool-call part (state: `approval-responded`) +2. Sends a new stream request with the full conversation (including approval) +3. The server sees the approval and either executes or cancels the tool + +--- + +## Key Source Files + +| File | Role | +|------|------| +| `packages/ai/src/types.ts` | `ToolCallState`, `ToolCallPart`, tool approval types | +| `packages/ai/src/activities/chat/stream/processor.ts` | `handleCustomEvent()` (approval-requested), `areAllToolsComplete()`, `addToolApprovalResponse()` | +| `packages/ai/src/activities/chat/stream/message-updaters.ts` | `updateToolCallApproval()`, `updateToolCallApprovalResponse()` | +| `packages/ai-client/src/chat-client.ts` | `addToolApprovalResponse()`, `checkForContinuation()`, continuation flags | +| `packages/ai-react/src/use-chat.ts` | React hook: exposes `addToolApprovalResponse` | +| `packages/ai-solid/src/use-chat.ts` | Solid hook: exposes `addToolApprovalResponse` | +| `packages/ai-vue/src/use-chat.ts` | Vue composable: exposes `addToolApprovalResponse` | +| `packages/ai-svelte/src/create-chat.svelte.ts` | Svelte: exposes `addToolApprovalResponse` | +| `packages/ai-client/tests/chat-client.test.ts` | Chained approval test (`describe('chained tool approvals')`) | +| `packages/ai/docs/chat-architecture.md` | Internal stream processing architecture | diff --git a/mintlify/ai/chat/agentic-cycle.md b/mintlify/ai/chat/agentic-cycle.md new file mode 100644 index 000000000..95de0303b --- /dev/null +++ b/mintlify/ai/chat/agentic-cycle.md @@ -0,0 +1,205 @@ +--- +title: Agentic Cycle +id: agentic-cycle +order: 1 +description: "The agentic cycle in TanStack AI — how the LLM loops through tool calls, results, and reasoning until it produces a final answer." +keywords: + - tanstack ai + - agentic cycle + - agent loop + - tool calling + - multi-step reasoning + - ai agents +--- + +The agentic cycle is the pattern where the LLM repeatedly calls tools, receives results, and continues reasoning until it can provide a final answer. This enables complex multi-step operations. + +> **Tip:** Code Mode can reduce agent loop iterations by letting the LLM write a program that calls multiple tools in a single execution. See [Code Mode](/ai/code-mode/code-mode). + +```mermaid +graph TD + A[User sends message] --> B[LLM analyzes request] + B --> C{Does task need tools?} + C -->|No| D[Generate text response] + C -->|Yes| E[Call appropriate tool] + E --> F{Where does
tool execute?} + F -->|Server| G[Execute on server] + F -->|Client| H[Execute on client] + G --> I[Tool returns result] + H --> I + I --> J[Add result to conversation] + J --> K[LLM analyzes result] + K --> L{Task complete?} + L -->|No| E + L -->|Yes| D + D --> M[Stream response to user] + M --> N[Done] + + style E fill:#e1f5ff + style G fill:#ffe1e1 + style H fill:#ffe1e1 + style L fill:#fff4e1 +``` + +### Detailed Agentic Flow + +```mermaid +sequenceDiagram + participant User + participant Client + participant Server + participant LLM + participant Tools + + User->>Client: "What's the weather in SF and LA?" + Client->>Server: Send message + Server->>LLM: Message + tool definitions + + Note over LLM: Cycle 1: Call first tool + + LLM->>Server: tool_call: get_weather(SF) + Server->>Tools: Execute get_weather + Tools-->>Server: {temp: 65, conditions: "sunny"} + Server->>LLM: tool_result + + Note over LLM: Cycle 2: Call second tool + + LLM->>Server: tool_call: get_weather(LA) + Server->>Tools: Execute get_weather + Tools-->>Server: {temp: 75, conditions: "clear"} + Server->>LLM: tool_result + + Note over LLM: Cycle 3: Generate answer + + LLM-->>Server: content: "SF is 65°F..." + Server-->>Client: Stream response + Client->>User: Display answer +``` + +### Multi-Step Example + +Here's a real-world example of the agentic cycle: + +**User**: "Find me flights to Paris under $500 and book the cheapest one" + +**Cycle 1**: LLM calls `searchFlights({destination: "Paris", maxPrice: 500})` +- Tool returns: `[{id: "F1", price: 450}, {id: "F2", price: 480}]` + +**Cycle 2**: LLM analyzes results and calls `bookFlight({flightId: "F1"})` +- Tool requires approval (sensitive operation) — see [Tool Approval](/ai/tools/tool-approval) +- User approves +- Tool returns: `{bookingId: "B123", confirmed: true}` + +**Cycle 3**: LLM generates final response +- "I found 2 flights under $500. I've booked the cheapest one (Flight F1) for $450. Your booking ID is B123." + +### Code Example: Agentic Weather Assistant + +```typescript +import { chat, toolDefinition, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { z } from "zod"; + +// Tool definitions +const getWeatherDef = toolDefinition({ + name: "get_weather", + description: "Get current weather for a city", + inputSchema: z.object({ + city: z.string(), + }), +}); + +const getClothingAdviceDef = toolDefinition({ + name: "get_clothing_advice", + description: "Get clothing recommendations based on weather", + inputSchema: z.object({ + temperature: z.number(), + conditions: z.string(), + }), +}); + +// Server implementations +const getWeather = getWeatherDef.server(async ({ city }) => { + const response = await fetch(`https://api.weather.com/v1/${city}`); + return await response.json(); +}); + +const getClothingAdvice = getClothingAdviceDef.server(async ({ temperature, conditions }) => { + // Business logic for clothing recommendations + if (temperature < 50) { + return { recommendation: "Wear a warm jacket" }; + } + return { recommendation: "Light clothing is fine" }; +}); + +// Server route +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages, + tools: [getWeather, getClothingAdvice], + }); + + return toServerSentEventsResponse(stream); +} +``` + +**User**: "What should I wear in San Francisco today?" + +**Agentic Cycle**: +1. LLM calls `get_weather({city: "San Francisco"})` → Returns `{temp: 62, conditions: "cloudy"}` +2. LLM calls `get_clothing_advice({temperature: 62, conditions: "cloudy"})` → Returns `{recommendation: "Light jacket recommended"}` +3. LLM generates: "The weather in San Francisco is 62°F and cloudy. I recommend wearing a light jacket." + +The loop continues only while the model's finish reason is `tool_calls` (with pending tool calls) **and** the agent loop strategy permits another iteration; it ends as soon as the model returns a normal `stop` finish reason. + +### Controlling the loop + +By default the loop is bounded by `maxIterations(5)` — after five iterations it stops even if the model would keep calling tools. Override this with the `agentLoopStrategy` option: + +```typescript +import { chat, maxIterations, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { getWeather, getClothingAdvice } from "./tools"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages, + tools: [getWeather, getClothingAdvice], + agentLoopStrategy: maxIterations(3), // default is 5 + }); + return toServerSentEventsResponse(stream); +} +``` + +Other built-in strategies: + +- **`untilFinishReason([...])`** — continue until the model returns one of the given finish reasons (e.g. `untilFinishReason(["stop", "length"])`). +- **`combineStrategies([...])`** — combine multiple strategies with AND logic; the loop continues only while every strategy agrees. + +A strategy is just a function that receives `{ iterationCount, finishReason, messages }` and returns `true` to allow another iteration or `false` to stop, so you can also write your own: + +```typescript +import { chat, combineStrategies, maxIterations, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import type { AgentLoopState } from "@tanstack/ai"; +import { getWeather, getClothingAdvice } from "./tools"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages, + tools: [getWeather, getClothingAdvice], + agentLoopStrategy: combineStrategies([ + maxIterations(10), + ({ messages }: AgentLoopState) => messages.length < 100, + ]), + }); + return toServerSentEventsResponse(stream); +} +``` diff --git a/mintlify/ai/chat/connection-adapters.md b/mintlify/ai/chat/connection-adapters.md new file mode 100644 index 000000000..e44400ace --- /dev/null +++ b/mintlify/ai/chat/connection-adapters.md @@ -0,0 +1,533 @@ +--- +title: Connection Adapters +id: connection-adapters +order: 3 +description: "Connection adapters bridge your client and server in TanStack AI — SSE, HTTP streaming, server functions, RPC, and persistent transports like WebSockets via subscribe/send." +keywords: + - tanstack ai + - connection adapters + - sse + - server-sent events + - http stream + - websocket + - rpc + - server functions + - fetcher + - streaming transport + - fetchServerSentEvents + - subscribe send +--- + +A **connection adapter** is the piece that decides _how_ chunks get from your server to the `ChatClient` (and through it, to your framework's `useChat`). Everything else in TanStack AI — chunk processing, message reassembly, tool calls, UI updates — is transport-agnostic. The adapter is the only thing that touches the network. + +This page covers every supported transport, when to pick which, and how to build a custom one. + +## Pick a Transport + +| You have… | Use | +| --- | --- | +| A normal HTTP server and want the default | [`fetchServerSentEvents`](#server-sent-events-sse) | +| An environment that blocks SSE (some edge runtimes, strict proxies) | [`fetchHttpStream`](#http-streaming-ndjson) | +| React Native or Expo | [`xhrHttpStream`](#react-native-and-expo) by default, [`xhrServerSentEvents`](#react-native-and-expo) for SSE, or [`fetchHttpStream`](#http-streaming-ndjson) only when streaming `fetch` is available | +| Code that **synchronously** returns an `AsyncIterable` (in-process `chat()`, an RSC stream, tests) | [`stream`](#server-functions-and-direct-async-iterables) | +| An **async** call — a TanStack Start server function or any `Promise`-returning function — resolving to a `Response` or an `AsyncIterable` | [`fetcher`](#server-functions-via-fetcher) | +| An RPC framework like Cap'n Web, gRPC-Web, or tRPC | [`rpcStream`](#rpc-streams) | +| A single long-lived WebSocket (or BroadcastChannel, postMessage, shared worker) serving many runs | [Custom `subscribe` / `send` adapter](#persistent-transports-websockets-and-friends) | +| Standard SSE but with custom fetch wrapping (auth refresh, retries) | [`fetchServerSentEvents` with `fetchClient`](#custom-fetch-client) | +| Something else entirely (HTTP/3, Server-Sent Events over a different protocol, etc.) | [Custom `connect` adapter](#custom-request-scoped-adapters) | + +All adapters produce the same `StreamChunk` events ([AG-UI Protocol](/ai/migration/ag-ui-compliance)) — the choice is purely about transport. + +## Server-Sent Events (SSE) + +The default. SSE is well-supported across browsers, transparent through most proxies, and easy to debug. Pair it with `toServerSentEventsResponse()` on the server. + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; + +const { messages, sendMessage } = useChat({ + connection: fetchServerSentEvents("/api/chat"), +}); +``` + +**Dynamic URL and headers.** Pass functions when the value depends on per-request state (current user, fresh token): + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; +import { currentUserId, getToken } from "./auth"; + +const { messages } = useChat({ + connection: fetchServerSentEvents( + () => `/api/chat?user=${currentUserId}`, + () => ({ + headers: { Authorization: `Bearer ${getToken()}` }, + }), + ), +}); +``` + +**Static body.** Anything in `options.body` is merged into the AG-UI `forwardedProps` payload sent to your server. Per-message data passed to `sendMessage` wins over this: + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; + +const { messages } = useChat({ + connection: fetchServerSentEvents("/api/chat", { + body: { provider: "openai", model: "gpt-5.1" }, + }), +}); +``` + +> **Tip:** `body` and `forwardedProps` populate the same wire field. Use `body` for static defaults, the `forwardedProps` constructor option (or per-`sendMessage` `data`) for dynamic values. Runtime values always win. + +## HTTP Streaming (NDJSON) + +For environments that don't speak SSE — some edge runtimes, certain mobile WebViews, or anywhere a proxy strips `text/event-stream` — use raw newline-delimited JSON. The wire format is one JSON `StreamChunk` per line: + +```typescript +import { useChat, fetchHttpStream } from "@tanstack/ai-react"; + +const { messages } = useChat({ + connection: fetchHttpStream("/api/chat"), +}); +``` + +Server-side, write each chunk as `JSON.stringify(chunk) + "\n"` to the response body. Options (`url`, `headers`, `body`, `fetchClient`, dynamic functions) match `fetchServerSentEvents` exactly. + +## React Native and Expo + +You have a native app that needs to call your own backend rather than a +same-origin browser route. Use `useChat` from `@tanstack/ai-react` with an +explicit chat transport and an absolute URL. By the end of this section, the +client adapter and server response helper will be paired correctly for React +Native or Expo. + +```typescript +const baseUrl = + process.env.EXPO_PUBLIC_TANSTACK_AI_BASE_URL ?? + 'http://127.0.0.1:8787' +const httpUrl = `${baseUrl}/chat/http` +const sseUrl = `${baseUrl}/chat/sse` +``` + +Use the URL your runtime can reach. iOS simulators can often use `localhost` or +`127.0.0.1`, Android emulators commonly use `10.0.2.2` to reach the host +machine, and physical devices need a LAN or tunneled URL. + +Prefer `xhrHttpStream()` for Expo and React Native. It pairs with +`toHttpResponse()` and reads newline-delimited JSON through incremental XHR +progress events: + +```typescript +import { useChat, xhrHttpStream } from "@tanstack/ai-react"; + +const baseUrl = process.env.EXPO_PUBLIC_TANSTACK_AI_BASE_URL ?? 'http://127.0.0.1:8787'; +const httpUrl = `${baseUrl}/chat/http`; + +const chat = useChat({ + connection: xhrHttpStream(httpUrl), +}); +``` + +Use `xhrServerSentEvents()` when your server returns `text/event-stream` via +`toServerSentEventsResponse()`: + +```typescript +import { useChat, xhrServerSentEvents } from "@tanstack/ai-react"; + +const baseUrl = process.env.EXPO_PUBLIC_TANSTACK_AI_BASE_URL ?? 'http://127.0.0.1:8787'; +const sseUrl = `${baseUrl}/chat/sse`; + +const chat = useChat({ + connection: xhrServerSentEvents(sseUrl), +}); +``` + +Only use `fetchHttpStream()` if your exact React Native runtime exposes +streaming `fetch` responses, `Response.body.getReader()`, and `TextDecoder`. +The server still returns newline-delimited JSON with `toHttpResponse()`: + +```typescript +import { useChat, fetchHttpStream } from "@tanstack/ai-react"; + +const baseUrl = process.env.EXPO_PUBLIC_TANSTACK_AI_BASE_URL ?? 'http://127.0.0.1:8787'; +const httpUrl = `${baseUrl}/chat/http`; + +const chat = useChat({ + connection: fetchHttpStream(httpUrl), +}); +``` + +If one of those fetch-streaming APIs is missing, `fetchHttpStream()` throws +`UnsupportedResponseStreamError`. A polyfill that buffers the response does not +make fetch streaming compatible; the adapter needs incremental bytes. Switch to +`xhrHttpStream()` or `xhrServerSentEvents()` instead. + +Keep provider SDKs and server helpers on your backend. The React Native bundle +should import hooks and connection adapters, not OpenAI/Anthropic/Gemini SDKs, +React DOM UI, devtools UI, or other framework packages. For a complete mobile +walkthrough, see [Quick Start: React Native](/ai/getting-started/quick-start-react-native). + +## Server Functions and Direct Async Iterables + +When your client can call into your server without going over HTTP — RSC streams, in-process tests, a direct in-process `chat()` call — skip the transport entirely. `stream()` takes a factory that returns an `AsyncIterable` **synchronously** and wires it straight into the client. (A [TanStack Start](https://tanstack.com/start) server function returns a `Promise`, so it needs [`fetcher`](#server-functions-via-fetcher), not `stream()` — see the next section.) + +```typescript +import { useChat, stream } from "@tanstack/ai-react"; +import { chatServerFn } from "./server/chat.server"; + +// `chatServerFn` is an in-process server-side function that synchronously +// returns an AsyncIterable — e.g. the result of +// `chat({ adapter, model, messages })` on the server. +const { messages } = useChat({ + connection: stream((messages, data) => chatServerFn({ messages, ...data })), +}); +``` + +The factory receives the conversation messages plus any per-request `data` you passed to `sendMessage`. Return any async iterable that yields `StreamChunk` objects — a generator, the output of `chat()` on the server, a transformed stream, anything. + +> **Tip:** `stream()` is **request-scoped**. The factory is invoked once per `sendMessage`, the iterable runs to completion, and the connection closes. If you need a single long-lived channel that multiplexes many sends — for example a WebSocket — use [`subscribe` / `send`](#persistent-transports-websockets-and-friends) instead. + +## Server Functions via `fetcher` + +When you call into your server with an **async** function — the universal case for a [TanStack Start](https://tanstack.com/start) server function, which always returns a `Promise` — use the top-level `fetcher` option instead of a connection adapter. `fetcher` is a sibling of `connection` (provide exactly one), and it accepts a plain async function. It mirrors the `fetcher` option on the [generation hooks](/ai/media/generation-hooks). The most common shape is a handler that ends with `toServerSentEventsResponse(...)` and resolves to a `Response`: + +```typescript ignore +// server/chat.server.ts +import { createServerFn } from "@tanstack/react-start"; +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import type { UIMessage } from "@tanstack/ai"; + +export const chatFn = createServerFn({ method: "POST" }) + .inputValidator((data: { messages: Array }) => data) + .handler(({ data }) => + toServerSentEventsResponse( + chat({ adapter: openaiText("gpt-5.1"), messages: data.messages }), + ), + ); +``` + +```typescript +import { useChat } from "@tanstack/ai-react"; +import { chatFn } from "./server/chat.server"; + +const { messages, sendMessage } = useChat({ + fetcher: ({ messages }, { signal }) => chatFn({ data: { messages }, signal }), +}); +``` + +The fetcher receives `{ messages, data, threadId, runId }` plus an `AbortSignal` (triggered by `stop()` or when a send is superseded). Return a `Response` — whose SSE body the chat client parses for you — **or** an `AsyncIterable`, which is yielded directly. If your server function returns the stream itself (instead of wrapping it in a `Response`), the fetcher handles that too. Sync and `Promise`-wrapped returns are both accepted. + +> **Tip:** The choice between `fetcher` and [`stream()`](#server-functions-and-direct-async-iterables) is about **async vs sync**, not `Response`-vs-iterable — both can yield an `AsyncIterable`. `stream()`'s factory must return that iterable **synchronously**, so a server-function call (which returns a `Promise`) won't typecheck there — that's the gap `fetcher` fills ([issue #509](https://github.com/TanStack/ai/issues/509)). Use `stream()` when you can hand back an async iterable synchronously (in-process `chat()`, an RPC client, tests); use `fetcher` for anything you have to `await`. Both normalize to the same request-scoped adapter, so `stop()`/abort, error handling, and tool calls behave identically. + +## RPC Streams + +`rpcStream()` is identical in behavior to `stream()` but reads better at call sites that hand off to an RPC client. Use it when integrating with Cap'n Web, gRPC-Web, tRPC subscriptions, or any RPC framework that already returns an async iterable: + +```typescript +import { useChat, rpcStream } from "@tanstack/ai-react"; +import { api } from "./rpc-client"; + +// `api.chat.stream` is your RPC method; it must return an AsyncIterable. +const { messages } = useChat({ + connection: rpcStream((messages, data) => + api.chat.stream({ messages, ...data }), + ), +}); +``` + +## Persistent Transports (WebSockets and Friends) + +A persistent transport — WebSocket, BroadcastChannel, postMessage between iframes, a shared worker — is fundamentally different from request/response. You open the channel **once**, then send and receive over it for the lifetime of the client. `stream()`/`connect()` can't model this cleanly because they assume one async iterable per request. + +For these cases, implement the `SubscribeConnectionAdapter` interface directly. The shape (full definition in [The Adapter Interface](#the-adapter-interface)): + +```typescript +import type { SubscribeConnectionAdapter } from "@tanstack/ai-react"; + +// subscribe(abortSignal?): AsyncIterable — long-lived +// send(messages, data?, abortSignal?, runContext?): Promise — one per user message +``` + +- `subscribe()` is called **once** by the `ChatClient` and returns a long-lived async iterable of every chunk the channel produces. +- `send()` is called **once per user message** to push a request frame onto the channel. It returns when the frame has been written — chunks arrive separately through `subscribe()`. + +The runtime correlates them: chunks emitted on the subscription queue between `send()` and the next terminal event (`RUN_FINISHED` / `RUN_ERROR`) are attributed to that run. + +### WebSocket example + +```typescript +import { useChat, type SubscribeConnectionAdapter } from "@tanstack/ai-react"; +import type { StreamChunk } from "@tanstack/ai"; + +function websocketConnection(url: string): SubscribeConnectionAdapter { + const ws = new WebSocket(url); + const queue: Array = []; + let pending: ((chunk: StreamChunk | null) => void) | null = null; + let closed = false; + + const ready = new Promise((resolve) => { + ws.addEventListener("open", () => resolve(), { once: true }); + }); + + function deliver(chunk: StreamChunk | null) { + const resolve = pending; + if (resolve) { + pending = null; + resolve(chunk); + } else if (chunk !== null) { + queue.push(chunk); + } + } + + ws.addEventListener("message", (event) => { + const chunk: StreamChunk = JSON.parse(event.data); + deliver(chunk); + }); + ws.addEventListener("close", () => { + closed = true; + deliver(null); + }); + + return { + async *subscribe(abortSignal) { + while (!abortSignal?.aborted && !closed) { + const buffered = queue.shift(); + if (buffered !== undefined) { + yield buffered; + continue; + } + const chunk = await new Promise((resolve) => { + pending = resolve; + abortSignal?.addEventListener("abort", () => resolve(null), { + once: true, + }); + }); + if (chunk === null) return; + yield chunk; + } + }, + + async send(messages, data, _abortSignal, runContext) { + await ready; + ws.send( + JSON.stringify({ + threadId: runContext?.threadId, + runId: runContext?.runId, + messages, + data, + }), + ); + }, + }; +} + +const { messages } = useChat({ + connection: websocketConnection("wss://example.com/chat"), +}); +``` + +> **Tip:** Your server is responsible for emitting `RUN_FINISHED` (or `RUN_ERROR`) at the end of each run. Without it, the client will not know the assistant turn has ended and will wait indefinitely. See [Streaming](/ai/chat/streaming) for the full event lifecycle. + +### When to choose persistent over request-scoped + +Pick `subscribe` / `send` when **any** of these are true: + +- A single connection multiplexes many runs (chat thread keeps the socket open across messages). +- The server pushes chunks outside of a request (presence updates, server-initiated tool calls, broadcast notifications). +- You want to share one connection across multiple tabs (BroadcastChannel) or workers. + +Otherwise, prefer `fetchServerSentEvents` or `stream()` — they're simpler and require no connection lifecycle management. + +## Custom Fetch Client + +If you're keeping SSE or HTTP streaming but need to wrap `fetch` — for auth refresh, retries, logging, or routing through an edge proxy — pass a `fetchClient`: + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; +import { refreshToken } from "./auth"; + +async function authedFetch(input: RequestInfo | URL, init?: RequestInit) { + let response = await fetch(input, init); + if (response.status === 401) { + await refreshToken(); + response = await fetch(input, init); + } + return response; +} + +const { messages } = useChat({ + connection: fetchServerSentEvents("/api/chat", { + fetchClient: authedFetch, + }), +}); +``` + +The `fetchClient` must satisfy the standard `fetch` signature. `fetchHttpStream` accepts the same option. + +## Custom Request-Scoped Adapters + +When none of the built-ins fit but the transport is still request-scoped (one request per user message), implement `ConnectConnectionAdapter` directly. This is the lowest-level escape hatch short of going persistent: + +```typescript +import { useChat, type ConnectConnectionAdapter } from "@tanstack/ai-react"; +import type { StreamChunk } from "@tanstack/ai"; + +const myAdapter: ConnectConnectionAdapter = { + async *connect(messages, data, abortSignal, runContext) { + const response = await fetch("/api/chat", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + threadId: runContext?.threadId, + runId: runContext?.runId, + messages, + ...data, + }), + ...(abortSignal ? { signal: abortSignal } : {}), + }); + + if (!response.ok) throw new Error(`HTTP ${response.status}`); + if (!response.body) throw new Error("Response has no body"); + + // Example: newline-delimited JSON. Replace this loop with whatever + // framing your wire format uses, yielding one `StreamChunk` per event. + const reader = response.body.getReader(); + const decoder = new TextDecoder(); + let buffer = ""; + while (true) { + const { done, value } = await reader.read(); + if (done) break; + buffer += decoder.decode(value, { stream: true }); + const lines = buffer.split("\n"); + buffer = lines.pop() ?? ""; + for (const line of lines) { + if (line.trim()) { + const chunk: StreamChunk = JSON.parse(line); + yield chunk; + } + } + } + }, +}; + +const { messages } = useChat({ connection: myAdapter }); +``` + +`runContext` carries `threadId`, `runId`, `clientTools`, and `forwardedProps`. Include them in your request payload so the server can build an AG-UI-compliant response. If your `connect` stream completes without emitting `RUN_FINISHED`, the runtime synthesizes one for you; if it throws, a `RUN_ERROR` is synthesized. + +## The Adapter Interface + +A `ConnectionAdapter` is a union — provide **either** `connect`, **or** both `subscribe` and `send`. Never both modes. + +```typescript +import type { UIMessage } from "@tanstack/ai-client"; +import type { ModelMessage, StreamChunk } from "@tanstack/ai"; + +export interface RunAgentInputContext { + threadId: string; + runId: string; + parentRunId?: string; + clientTools?: Array<{ name: string; description: string; parameters: unknown }>; + forwardedProps?: Record; +} + +export interface ConnectConnectionAdapter { + connect( + messages: UIMessage[] | ModelMessage[], + data?: Record, + abortSignal?: AbortSignal, + runContext?: RunAgentInputContext, + ): AsyncIterable; +} + +export interface SubscribeConnectionAdapter { + subscribe(abortSignal?: AbortSignal): AsyncIterable; + send( + messages: UIMessage[] | ModelMessage[], + data?: Record, + abortSignal?: AbortSignal, + runContext?: RunAgentInputContext, + ): Promise; +} + +export type ConnectionAdapter = + | ConnectConnectionAdapter + | SubscribeConnectionAdapter; +``` + +Internally, `ChatClient` normalizes both shapes to a single `subscribe`/`send` pair via `normalizeConnectionAdapter()`. If you provide `connect`, it gets wrapped in an async queue; if you provide `subscribe` + `send` natively, they're used as-is. + +## Authentication + +Static headers go in `options.headers`: + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; +import { token } from "./auth"; + +const { messages } = useChat({ + connection: fetchServerSentEvents("/api/chat", { + headers: { Authorization: `Bearer ${token}` }, + }), +}); +``` + +For tokens that change per request (refresh tokens, short-lived JWTs), pass a function — it's called on every send, so the header always reflects the latest token: + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; +import { getToken } from "./auth"; + +const { messages } = useChat({ + connection: fetchServerSentEvents("/api/chat", () => ({ + headers: { Authorization: `Bearer ${getToken()}` }, + })), +}); +``` + +Cookies are sent automatically when `credentials` is `"same-origin"` (default) or `"include"`. + +## Cancellation + +Every adapter — built-in or custom — receives an `AbortSignal`. Built-ins propagate it to `fetch`; custom adapters must honor it themselves. `useChat`'s `stop()` aborts the current run by triggering the signal: + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; + +const { stop } = useChat({ connection: fetchServerSentEvents("/api/chat") }); +stop(); // aborts the active stream +``` + +For `SubscribeConnectionAdapter`, the signal in `subscribe()` ends the entire subscription (component unmount); the signal in `send()` ends just the in-flight send. + +## Error Handling + +Adapters should throw on transport errors (HTTP non-2xx, parse failures, dropped sockets). The `ChatClient` catches the throw, emits a `RUN_ERROR` chunk if none has been emitted yet, and surfaces it via `onError` / the `error` state: + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; + +const { error } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + onError: (err) => console.error("Chat failed:", err), +}); +``` + +Don't swallow `AbortError` — let it propagate so the client knows the abort succeeded. + +## Best Practices + +- **Default to SSE.** It's the most compatible and the easiest to debug. Switch only when something blocks it. +- **Use `stream()` when you can.** If you control both sides and don't need HTTP semantics, server functions are faster to wire up than building a custom adapter. +- **Reach for `subscribe`/`send` only when you need persistence.** WebSockets are powerful but require you to handle reconnection, run correlation, and lifecycle yourself. +- **Always honor `abortSignal`.** It's how the client cleans up on unmount and on `stop()`. +- **Emit `RUN_FINISHED` from the server.** Without it, the client never knows the turn ended. + +## Next Steps + +- [Streaming](/ai/chat/streaming) — the full event lifecycle and `StreamChunk` types +- [AG-UI Client Compliance](/ai/migration/ag-ui-compliance) — the wire protocol your server emits +- [Cloudflare Adapter](/ai/community-adapters/cloudflare) — example of a custom `fetchClient` in production +- [API Reference: `@tanstack/ai-client`](/ai/api/ai-client) — full type signatures diff --git a/mintlify/ai/chat/persistence.md b/mintlify/ai/chat/persistence.md new file mode 100644 index 000000000..985c8aa2d --- /dev/null +++ b/mintlify/ai/chat/persistence.md @@ -0,0 +1,153 @@ +--- +title: Persistence +id: chat-persistence +order: 5 +description: "Persist chat conversations on the client with TanStack AI — hydrate on load, save on change, and clear on reset using a simple getItem/setItem/removeItem adapter." +keywords: + - tanstack ai + - persistence + - chat history + - localStorage + - indexeddb + - offline + - hydration +--- + +By default a `ChatClient` (and every framework `useChat`/`createChat` wrapper) keeps messages in memory only — reload the page or navigate away and the conversation is gone. The optional **persistence adapter** wires the client to a storage backend so conversations survive reloads, with no manual `initialMessages` + `onFinish` boilerplate. + +This is especially useful for SPAs, Electron apps, and offline-first setups where the client is the source of truth and there's no server managing conversation state. + +## The adapter interface + +A persistence adapter is any object with three methods — the same `getItem`/`setItem`/`removeItem` shape used elsewhere in TanStack AI. Each method may be synchronous or return a `Promise`: + +```typescript +import type { UIMessage } from "@tanstack/ai-client"; + +interface ChatClientPersistence { + getItem: ( + id: string, + ) => + | Array + | null + | undefined + | Promise | null | undefined>; + setItem: (id: string, messages: Array) => void | Promise; + removeItem: (id: string) => void | Promise; +} +``` + +The `id` passed to each method is the client's `id` option. Provide a stable `id` per conversation so the right history is loaded back: + +```typescript +import { ChatClient } from "@tanstack/ai-client"; +import { adapter, myPersistenceAdapter } from "./chat-setup"; + +const client = new ChatClient({ + id: "conversation-123", + connection: adapter, + persistence: myPersistenceAdapter, +}); +``` + +## What the client does for you + +When a `persistence` adapter is provided, `ChatClient`: + +- **Hydrates on construction** — calls `getItem(id)`. If it returns an array, those messages populate the client (overriding `initialMessages`). Async adapters hydrate as soon as the promise resolves, unless you've already started a new conversation in the meantime. +- **Saves on every change** — calls `setItem(id, messages)` whenever the message list changes (new user message, streamed assistant content, tool calls/results, approval responses). Writes are queued so they never overlap or land out of order. +- **Clears on `clear()`** — calls `removeItem(id)` and discards any in-flight stream so a cleared conversation doesn't get repopulated by late chunks. + +When `persistence` is omitted, nothing changes — the client behaves exactly as before. The option is fully backwards compatible. + +Persistence is **best-effort**: if an adapter method throws or rejects, the error is swallowed so storage problems never break the chat. Handle and surface errors inside your adapter if you need to react to them. + +## Framework usage + +Every framework wrapper accepts the same `persistence` option and forwards it to the underlying `ChatClient`: + +```tsx +// React / Preact +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; +import { myPersistenceAdapter } from "./persistence"; + +const chat = useChat({ + id: "conversation-123", + connection: fetchServerSentEvents("/api/chat"), + persistence: myPersistenceAdapter, +}); +``` + +```ts +// Solid / Vue — same option +import { useChat, fetchServerSentEvents } from "@tanstack/ai-solid"; +import { myPersistenceAdapter } from "./persistence"; + +const chat = useChat({ + id: "conversation-123", + connection: fetchServerSentEvents("/api/chat"), + persistence: myPersistenceAdapter, +}); +``` + +```ts ignore +// Svelte +const chat = createChat({ + id: "conversation-123", + connection: fetchServerSentEvents("/api/chat"), + persistence: myPersistenceAdapter, +}); +``` + +## Example: `localStorage` + +A synchronous adapter backed by `localStorage`. Note that `UIMessage.createdAt` is a `Date`, which `JSON.stringify` turns into a string — revive it on read if you depend on it: + +```typescript +import type { ChatClientPersistence, UIMessage } from "@tanstack/ai-client"; + +const localStoragePersistence: ChatClientPersistence = { + getItem: (id) => { + const raw = window.localStorage.getItem(id); + if (!raw) return null; + const stored: Array = JSON.parse(raw); + return stored.map((message) => ({ + ...message, + createdAt: + typeof message.createdAt === "string" + ? new Date(message.createdAt) + : message.createdAt, + })); + }, + setItem: (id, messages) => { + window.localStorage.setItem(id, JSON.stringify(messages)); + }, + removeItem: (id) => { + window.localStorage.removeItem(id); + }, +}; +``` + +## Example: IndexedDB (async) + +For larger histories or structured queries, back the adapter with an async store such as IndexedDB. The client awaits async methods automatically: + +```typescript +import type { ChatClientPersistence } from "@tanstack/ai-client"; +import { db } from "./db"; + +const indexedDbPersistence: ChatClientPersistence = { + getItem: async (id) => { + const record = await db.conversations.get(id); + return record?.messages; + }, + setItem: async (id, messages) => { + await db.conversations.put({ id, messages, updatedAt: Date.now() }); + }, + removeItem: async (id) => { + await db.conversations.delete(id); + }, +}; +``` + +Any backend works — IndexedDB, SQLite (Electron/Tauri), a remote database, or an in-memory `Map` for tests — as long as it implements the three methods. diff --git a/mintlify/ai/chat/streaming.md b/mintlify/ai/chat/streaming.md new file mode 100644 index 000000000..bcfc3118d --- /dev/null +++ b/mintlify/ai/chat/streaming.md @@ -0,0 +1,217 @@ +--- +title: Streaming +id: streaming-responses +order: 2 +description: "Stream AI responses in real time with TanStack AI — async iterable chunks, chunk strategies, and partial JSON for responsive chat UIs." +keywords: + - tanstack ai + - streaming + - streaming responses + - real-time ai + - async iterable + - chunks + - partial json +--- + +TanStack AI supports streaming responses for real-time chat experiences. Streaming allows you to display responses as they're generated, rather than waiting for the complete response. + +## How Streaming Works + +When you use `chat()`, it returns an async iterable stream of chunks: + +```typescript +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: [{ role: "user", content: "Hello!" }], +}); + +// Stream contains chunks as they arrive +for await (const chunk of stream) { + console.log(chunk); // Process each chunk +} +``` + +## Server-Side Streaming + +Convert the stream to an HTTP response using `toServerSentEventsResponse`: + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages, + }); + + // Convert to HTTP response with proper headers + return toServerSentEventsResponse(stream); +} +``` + +## Client-Side Streaming + +The `useChat` hook automatically handles streaming: + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; + +const { messages, sendMessage, isLoading } = useChat({ + connection: fetchServerSentEvents("/api/chat"), +}); + +// Messages update in real-time as chunks arrive +messages.forEach((message) => { + // Message content updates incrementally +}); +``` + +## Stream Events (AG-UI Protocol) + +TanStack AI implements the [AG-UI Protocol](https://docs.ag-ui.com/introduction) for streaming. Stream events contain different types of data: + +### AG-UI Events + +- **RUN_STARTED** - Emitted when a run begins +- **TEXT_MESSAGE_START/CONTENT/END** - Text content streaming lifecycle +- **TOOL_CALL_START/ARGS/END** - Tool invocation lifecycle +- **STEP_STARTED/STEP_FINISHED** - Thinking/reasoning steps +- **CUSTOM** - Namespaced extension events (sandbox file changes, Code Mode progress, structured-output completion, and your own `emitCustomEvent` calls) — see the [Custom Events Reference](/ai/protocol/custom-events) for the full typed taxonomy and how to narrow `chunk.value` with a plain `if` +- **RUN_FINISHED** - Run completion with finish reason and usage +- **RUN_ERROR** - Error occurred during the run + +> **Tip:** Some models expose their internal reasoning as thinking content that streams before the response. See [Thinking & Reasoning](/ai/chat/thinking-content). + +### Thinking Chunks + +Adapters emit reasoning as both the canonical `REASONING_MESSAGE_*` events and the older `STEP_STARTED` / `STEP_FINISHED` events. Rather than parsing those raw events yourself, read the reconciled `ThinkingPart` from `message.parts` — the stream processor merges both event families into a single part for you: + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; + +const { messages } = useChat({ + connection: fetchServerSentEvents("/api/chat"), +}); + +for (const message of messages) { + for (const part of message.parts) { + if (part.type === "thinking") { + console.log("Thinking:", part.content); // Accumulated thinking content + } + } +} +``` + +Thinking content is automatically converted to `ThinkingPart` in `UIMessage` objects. It is UI-only and excluded from messages sent back to the model. See [Thinking & Reasoning](/ai/chat/thinking-content) for the full rendering pattern. + +## Connection Adapters + +TanStack AI provides connection adapters for different streaming protocols: + +### Server-Sent Events (SSE) + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; + +const { messages } = useChat({ + connection: fetchServerSentEvents("/api/chat"), +}); +``` + +### HTTP Stream + +```typescript +import { useChat, fetchHttpStream } from "@tanstack/ai-react"; + +const { messages } = useChat({ + connection: fetchHttpStream("/api/chat"), +}); +``` + +### Custom Stream + +For a fully custom request, use the `fetcher` transport. The fetcher receives the request input plus an `AbortSignal`, and returns a `Response` (whose SSE body the client parses) or an `AsyncIterable`. It may return that value synchronously, as a `Promise`, or as an `async function*`: + +```typescript +import { useChat } from "@tanstack/ai-react"; + +const { messages } = useChat({ + fetcher: ({ messages, data }, { signal }) => + fetch("/api/chat", { + method: "POST", + body: JSON.stringify({ messages, ...data }), + signal, + }), +}); +``` + +> **Note:** The lower-level `stream()` connection adapter takes a factory that must return an `AsyncIterable` **synchronously** (e.g. a generator) — it does not accept an `async (...) => {...}` function that returns a `Promise`. Prefer the `fetcher` transport above unless you specifically need the connection adapter. + +## Monitoring Stream Progress + +You can monitor stream progress with callbacks: + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; + +const { messages } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + onChunk: (chunk) => { + console.log("Received chunk:", chunk); + }, + onFinish: (message) => { + console.log("Stream finished:", message); + }, +}); +``` + +## Cancelling Streams + +Cancel ongoing streams: + +```typescript +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; + +const { stop } = useChat({ + connection: fetchServerSentEvents("/api/chat"), +}); + +// Cancel the current stream +stop(); +``` + +Calling `stop()` aborts the underlying fetch; the resulting `AbortError` is expected and normal. This differs from a connection being cut mid-line: a truncated stream throws a `StreamTruncatedError` and moves the client into its `error` state. See [Connection Adapters](/ai/chat/connection-adapters) for the underlying behavior. + +On the server, pass an `AbortController` to `toServerSentEventsResponse(stream, { abortController })` so the chat run is cancelled when the client disconnects: + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + const stream = chat({ adapter: openaiText("gpt-5.5"), messages }); + + const abortController = new AbortController(); + return toServerSentEventsResponse(stream, { abortController }); +} +``` + +## Best Practices + +1. **Handle loading states** - Use `isLoading` to show loading indicators +2. **Handle errors** - Check `error` state for stream failures +3. **Cancel on unmount** - Clean up streams when components unmount +4. **Optimize rendering** - Batch updates if needed for performance +5. **Show progress** - Display partial content as it streams + +## Next Steps + +- [Connection Adapters](/ai/chat/connection-adapters) - Learn about different connection types +- [API Reference](/ai/api/ai) - Explore streaming APIs diff --git a/mintlify/ai/chat/structured-outputs.md b/mintlify/ai/chat/structured-outputs.md new file mode 100644 index 000000000..dccea9154 --- /dev/null +++ b/mintlify/ai/chat/structured-outputs.md @@ -0,0 +1,21 @@ +--- +title: Structured Outputs (Moved) +id: structured-outputs +order: 4 +description: "Structured outputs documentation has moved to its own top-level Structured Outputs section, with separate guides for one-shot extraction, streaming UIs, multi-turn chat, and combining with tools." +keywords: + - tanstack ai + - structured outputs + - moved + - redirect +--- + +The structured-outputs guide has moved to its own top-level section, split by what you're building. Pick the journey that fits: + +- **[Overview](/ai/structured-outputs/overview)** — what structured output is, schema library options, provider support, and "which page do I read?" +- **[One-Shot Extraction](/ai/structured-outputs/one-shot)** — single prompt in, single typed object out. Use this when you don't need streaming or chat history. +- **[Streaming UIs](/ai/structured-outputs/streaming)** — `useChat({ outputSchema })` with `partial` and `final` populating a UI field by field. +- **[Multi-Turn Chat](/ai/structured-outputs/multi-turn)** — every assistant turn carries its own typed `StructuredOutputPart`, history stays renderable, and `messages[i].parts.find(p => p.type === "structured-output").data` is typed by your schema. +- **[With Tools](/ai/structured-outputs/with-tools)** — combining `outputSchema` with the agent loop, including pause/resume for server-tool approvals and client-tool invocations. + +> **Note:** This URL is kept for backward compatibility. New content lives under `/structured-outputs/*` — update existing bookmarks when you can. diff --git a/mintlify/ai/chat/thinking-content.md b/mintlify/ai/chat/thinking-content.md new file mode 100644 index 000000000..9e39ad320 --- /dev/null +++ b/mintlify/ai/chat/thinking-content.md @@ -0,0 +1,157 @@ +--- +title: Thinking & Reasoning +id: thinking-content +order: 5 +description: "Render reasoning tokens from thinking models (Claude extended thinking, OpenAI o-series) as streamed ThinkingPart in TanStack AI chat UIs." +keywords: + - tanstack ai + - thinking + - reasoning + - extended thinking + - claude thinking + - o-series + - chain of thought + - ThinkingPart +--- + +Some models expose their internal reasoning as "thinking" content -- Claude with extended thinking, OpenAI o-series models with reasoning, and others. TanStack AI captures this as `ThinkingPart` in messages, streamed to your UI in real-time alongside text and tool calls. + +Thinking content is **UI-only**. It is never sent back to the model in subsequent requests. + +## How It Works + +When a model emits reasoning tokens, the adapter emits AG-UI events for them. Adapters emit `REASONING_MESSAGE_*` events (the preferred, canonical form) **and** the older `STEP_STARTED` / `STEP_FINISHED` events. The stream processor reconciles both into a single `ThinkingPart` on the assistant's `UIMessage`, deduplicating overlapping content. You should rely on the `ThinkingPart` in `message.parts` rather than hand-parsing the raw events: + +```typescript +interface ThinkingPart { + type: "thinking"; + content: string; + stepId?: string; + signature?: string; +} +``` + +The `ThinkingPart` appears in `UIMessage.parts` alongside `TextPart` and `ToolCallPart` entries. As reasoning tokens arrive, its `content` accumulates token by token. + +## Enabling Thinking + +How you enable thinking depends on the provider. + +### Anthropic (Extended Thinking) + +Pass the `thinking` option in `modelOptions` with `type: "enabled"` and a `budget_tokens` (minimum 1024). Keep `budget_tokens` below `modelOptions.max_tokens` so there is room for the visible response in addition to the thinking budget: + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { anthropicText } from "@tanstack/ai-anthropic"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + const stream = chat({ + adapter: anthropicText("claude-sonnet-4-6"), + messages, + modelOptions: { + max_tokens: 32000, + // budget_tokens must be at least 1024 and below max_tokens + thinking: { type: "enabled", budget_tokens: 10000 }, + }, + }); + return toServerSentEventsResponse(stream); +} +``` + +### OpenAI (Reasoning Models) + +OpenAI o-series models (o1, o3, o3-mini, o3-pro) perform reasoning automatically. You can control the depth with the `reasoning` option: + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + const stream = chat({ + adapter: openaiText("o3-mini"), + messages, + modelOptions: { + reasoning: { + effort: "medium", // 'none' | 'minimal' | 'low' | 'medium' | 'high' + summary: "auto", // 'auto' | 'detailed' + }, + }, + }); + return toServerSentEventsResponse(stream); +} +``` + +When `reasoning.summary` is set, the adapter streams reasoning summary text as thinking content. Without it, reasoning tokens are still used internally but may not be surfaced depending on the model. + +GPT-5 and later models also support reasoning. Their `reasoning.effort` accepts `"none" | "minimal" | "low" | "medium" | "high"`, and reasoning activates on any non-`none` value: + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +export async function POST(request: Request) { + const { messages } = await request.json(); + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages, + modelOptions: { + reasoning: { effort: "high" }, + }, + }); + return toServerSentEventsResponse(stream); +} +``` + +## Rendering in React + +Thinking parts appear in `message.parts` just like text and tool calls. A common pattern is to render them in a collapsible element so they don't dominate the UI: + +```tsx +import type { UIMessage } from "@tanstack/ai-react"; + +function MessageContent({ message }: { message: UIMessage }) { + return ( +
+ {message.parts.map((part, idx) => { + if (part.type === "thinking") { + return ( +
+ Thinking... +
{part.content}
+
+ ); + } + if (part.type === "text") { + return

{part.content}

; + } + return null; + })} +
+ ); +} +``` + +The [Quick Start](/ai/getting-started/quick-start) guide shows a simpler inline pattern where thinking is rendered as italic text above the response. + +## Streaming Behavior + +Thinking content streams **before** the final text response. As reasoning tokens arrive, `ThinkingPart.content` accumulates token by token, the same way `TextPart.content` does for the response text. + +The typical streaming order is: + +1. The reasoning block begins (`REASONING_MESSAGE_START`, plus a legacy `STEP_STARTED`) +2. Reasoning tokens stream in (`REASONING_MESSAGE_CONTENT`, plus legacy `STEP_FINISHED` events), accumulating into `ThinkingPart.content` +3. `TEXT_MESSAGE_START` -- the model begins its visible response +4. `TEXT_MESSAGE_CONTENT` (repeated) -- the response text streams in + +Adapters emit both the canonical `REASONING_MESSAGE_*` events and the older `STEP_*` events; the stream processor reconciles them into one `ThinkingPart` so you never have to hand-parse the raw events. If you use `useChat` from `@tanstack/ai-react` (or the Solid/Vue/Svelte equivalents), your `messages` array updates automatically with both thinking and text parts as they arrive. + +## Next Steps + +- [Streaming](/ai/chat/streaming) -- Connection adapters and stream events +- [Agentic Cycle](/ai/chat/agentic-cycle) -- How thinking interacts with tool-calling loops +- [Anthropic Adapter](/ai/adapters/anthropic) -- Full Anthropic provider options +- [OpenAI Adapter](/ai/adapters/openai) -- Full OpenAI provider options diff --git a/mintlify/ai/code-mode/client-integration.md b/mintlify/ai/code-mode/client-integration.md new file mode 100644 index 000000000..db4e156f9 --- /dev/null +++ b/mintlify/ai/code-mode/client-integration.md @@ -0,0 +1,286 @@ +--- +title: Showing Code Mode in the UI +id: code-mode-client-integration +order: 2 +description: "Stream Code Mode execution events to your React app — console output, external calls, and results as they happen, via onCustomEvent." +keywords: + - tanstack ai + - code mode + - react ui + - custom events + - onCustomEvent + - streaming ui + - execution progress +--- + +You have [Code Mode](/ai/code-mode/code-mode) working on your server — the LLM writes and executes TypeScript, and you get results back. But your users see nothing while the sandbox runs. By the end of this guide, your React app will show real-time execution progress: console output, external function calls, and final results as they stream in. + +## How events reach the client + +When code runs inside the sandbox, Code Mode emits **custom events** through the AG-UI streaming protocol. These events travel alongside normal chat chunks (text, tool calls) and arrive in your client via the `onCustomEvent` callback. + +The events emitted during each `execute_typescript` call: + +| Event | When | Key fields | +|-------|------|------------| +| `code_mode:execution_started` | Sandbox begins executing | `timestamp`, `codeLength` | +| `code_mode:console` | Each `console.log/error/warn/info` | `level`, `message`, `timestamp` | +| `code_mode:external_call` | Before an `external_*` function runs | `function`, `args`, `timestamp` | +| `code_mode:external_result` | After a successful `external_*` call | `function`, `result`, `duration` | +| `code_mode:external_error` | When an `external_*` call fails | `function`, `error`, `duration` | + +Every event includes a `toolCallId` that ties it to the specific `execute_typescript` tool call, so you can render events alongside the right message. + +## Listening to events with useChat + +Pass an `onCustomEvent` callback to `useChat`. The callback receives the event type, payload, and a context object with the `toolCallId`: + +```tsx group=code-mode-client +import { useCallback, useRef, useState } from "react"; +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; + +interface VMEvent { + id: string; + eventType: string; + data: unknown; + timestamp: number; +} + +export function CodeModeChat() { + const [toolCallEvents, setToolCallEvents] = useState< + Map> + >(new Map()); + const eventIdCounter = useRef(0); + + const handleCustomEvent = useCallback( + ( + eventType: string, + data: unknown, + context: { toolCallId?: string }, + ) => { + const { toolCallId } = context; + if (!toolCallId) return; + + const event: VMEvent = { + id: `event-${eventIdCounter.current++}`, + eventType, + data, + timestamp: Date.now(), + }; + + setToolCallEvents((prev) => { + const next = new Map(prev); + const events = next.get(toolCallId) || []; + next.set(toolCallId, [...events, event]); + return next; + }); + }, + [], + ); + + const { messages, sendMessage, isLoading } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + onCustomEvent: handleCustomEvent, + }); + + // Render messages with events — see next section +} +``` + +Events are keyed by `toolCallId` so each `execute_typescript` call gets its own event timeline. + +## Rendering execution progress + +When rendering messages, check for `execute_typescript` tool calls and display their events: + +```tsx group=code-mode-client +function MessageList({ + messages, + toolCallEvents, +}: { + messages: Array<{ id: string; role: string; parts: Array }>; + toolCallEvents: Map>; +}) { + return ( +
+ {messages.map((message) => ( +
+ {message.parts.map((part) => { + if (part.type === "text") { + return

{part.content}

; + } + + if ( + part.type === "tool-call" && + part.name === "execute_typescript" + ) { + const events = toolCallEvents.get(part.id) || []; + const result = part.output; + + return ( +
+ +
+ ); + } + + return null; + })} +
+ ))} +
+ ); +} +``` + +## Building an execution panel + +Here's a complete `CodeExecutionPanel` component that shows the generated code, live event stream, and final result: + +```tsx group=code-mode-client +function CodeExecutionPanel({ + code, + events, + result, + isRunning, +}: { + code?: string; + events: Array; + result?: { success: boolean; result?: unknown; logs?: string[]; error?: { message: string } }; + isRunning: boolean; +}) { + return ( +
+ {/* Generated code */} + {code && ( +
+ + TypeScript code + +
+            {code}
+          
+
+ )} + + {/* Live event stream */} + {events.length > 0 && ( +
+
+ Execution log +
+
+ {events.map((event) => ( + + ))} + {isRunning && ( +
Running...
+ )} +
+
+ )} + + {/* Final result */} + {result && ( +
+ {result.error && ( +
Error: {result.error.message}
+ )} + {result.logs && result.logs.length > 0 && ( +
+              {result.logs.join("\n")}
+            
+ )} + {result.success && result.result !== undefined && ( +
+              {JSON.stringify(result.result, null, 2)}
+            
+ )} +
+ )} +
+ ); +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null; +} + +function EventLine({ event }: { event: VMEvent }) { + if (!isRecord(event.data)) return null; + const data = event.data; + + switch (event.eventType) { + case "code_mode:console": + return ( +
+ [{String(data.level)}] {String(data.message)} +
+ ); + + case "code_mode:external_call": + return ( +
+ → {String(data.function)}( + {JSON.stringify(data.args)}) +
+ ); + + case "code_mode:external_result": + return ( +
+ ← {String(data.function)} ({String(data.duration)}ms) +
+ ); + + case "code_mode:external_error": + return ( +
+ ✗ {String(data.function)}: {String(data.error)} +
+ ); + + case "code_mode:execution_started": + return
▶ Execution started
; + + default: + return ( +
+ {event.eventType}: {JSON.stringify(data)} +
+ ); + } +} +``` + +This gives you: +- A collapsible code block showing the TypeScript the model wrote +- A live event log showing console output, external function calls with arguments, results with durations, and errors +- A status-colored result panel with logs and the return value + +## Adapting for other frameworks + +The `onCustomEvent` callback is available through `ChatClient` from `@tanstack/ai-client`, which all framework integrations use under the hood. In Solid, Vue, or Svelte, pass `onCustomEvent` in the same way you pass it to `useChat` in React — the callback signature is identical: + +```typescript ignore +(eventType: string, data: unknown, context: { toolCallId?: string }) => void +``` + +See [Code Mode](/ai/code-mode/code-mode) for setting up the server side, and [Code Mode with Skills](/ai/code-mode/code-mode-with-skills) for adding persistent skill libraries. diff --git a/mintlify/ai/code-mode/code-mode-isolates.md b/mintlify/ai/code-mode/code-mode-isolates.md new file mode 100644 index 000000000..65413458b --- /dev/null +++ b/mintlify/ai/code-mode/code-mode-isolates.md @@ -0,0 +1,221 @@ +--- +title: Code Mode Isolate Drivers +id: code-mode-isolates +order: 4 +description: "Compare Code Mode sandbox drivers — Node isolated-vm, QuickJS WASM, and Cloudflare Workers — and choose the right runtime for your deployment." +keywords: + - tanstack ai + - code mode + - isolate driver + - isolated-vm + - quickjs + - cloudflare workers + - sandbox + - secure execution +--- + +Isolate drivers provide the secure sandbox runtimes that [Code Mode](/ai/code-mode/code-mode) uses to execute generated TypeScript. All drivers implement the same `IsolateDriver` interface, so you can swap them without changing any other code. + +## Choosing a Driver + +| | Node (`isolated-vm`) | QuickJS (WASM) | Cloudflare Workers | +|---|---|---|---| +| **Best for** | Server-side Node.js apps | Browsers, edge, portability | Edge deployments on Cloudflare | +| **Performance** | Fast (V8 JIT) | Slower (interpreted) | Fast (V8 on Cloudflare edge) | +| **Native deps** | Yes (C++ addon) | None | None | +| **Browser support** | No | Yes | N/A | +| **Memory limit** | Configurable | Configurable | N/A | +| **Stack size limit** | N/A | Configurable | N/A | +| **Setup** | `pnpm add` | `pnpm add` | Deploy a Worker first | + +--- + +## Node.js Driver (`@tanstack/ai-isolate-node`) + +Uses V8 isolates via the [`isolated-vm`](https://github.com/laverdet/isolated-vm) native addon. This is the fastest option for server-side Node.js applications because generated code runs in the same V8 engine as the host, under JIT compilation, with no serialization overhead beyond tool call boundaries. + +### Installation + +```bash +pnpm add @tanstack/ai-isolate-node +``` + +`isolated-vm` is a native C++ addon and must be compiled for your platform. It requires Node.js 18 or later. + +### Usage + +```typescript +import { createNodeIsolateDriver } from '@tanstack/ai-isolate-node' + +const driver = createNodeIsolateDriver({ + memoryLimit: 128, // MB + timeout: 30_000, // ms +}) +``` + +### Options + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `memoryLimit` | `number` | `128` | Maximum heap size for the V8 isolate, in megabytes. Execution is terminated if this limit is exceeded. | +| `timeout` | `number` | `30000` | Maximum wall-clock time per execution, in milliseconds. | + +### How it works + +Each `execute_typescript` call creates a fresh V8 isolate. Your tools are bridged into the isolate as async reference functions — when generated code calls `external_myTool(...)`, the call crosses the isolate boundary back into the host Node.js process, executes your tool implementation, and returns the result. Console output (`log`, `error`, `warn`, `info`) is captured and returned in the execution result. The isolate is destroyed after each call. + +--- + +## QuickJS Driver (`@tanstack/ai-isolate-quickjs`) + +Uses [QuickJS](https://bellard.org/quickjs/) compiled to WebAssembly via Emscripten. Because the sandbox is a WASM module, it has no native dependencies and runs anywhere JavaScript runs: Node.js, browsers, Deno, Bun, and Cloudflare Workers (without deploying a separate Worker). + +### Installation + +```bash +pnpm add @tanstack/ai-isolate-quickjs +``` + +### Usage + +```typescript +import { createQuickJSIsolateDriver } from '@tanstack/ai-isolate-quickjs' + +const driver = createQuickJSIsolateDriver({ + memoryLimit: 128, // MB + timeout: 30_000, // ms + maxStackSize: 524288, // bytes (512 KiB) +}) +``` + +### Options + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `memoryLimit` | `number` | `128` | Maximum heap memory for the QuickJS VM, in megabytes. | +| `timeout` | `number` | `30000` | Maximum wall-clock time per execution, in milliseconds. | +| `maxStackSize` | `number` | `524288` | Maximum call stack size in bytes (default: 512 KiB). Increase for deeply recursive code; decrease to catch runaway recursion sooner. | + +### How it works + +QuickJS WASM uses an asyncified execution model — the WASM module can pause while awaiting host async functions (your tools). Executions are serialized through a global queue to prevent concurrent WASM calls, which the asyncify model does not support. Fatal errors (memory exhaustion, stack overflow) are detected, the VM is disposed, and a structured error is returned. Console output is captured and returned with the result. + +> **Performance note:** QuickJS interprets JavaScript rather than JIT-compiling it, so compute-heavy scripts run slower than with the Node driver. For typical LLM-generated scripts that are mostly waiting on `external_*` tool calls, this difference is not significant. + +--- + +## Cloudflare Workers Driver (`@tanstack/ai-isolate-cloudflare`) + +Runs generated code inside a [Cloudflare Worker](https://workers.cloudflare.com/) at the edge. Your application server sends code and tool schemas to the Worker via HTTP; the Worker executes the code and calls back when it needs a tool result. This keeps your tool implementations on your server while sandboxed execution happens on Cloudflare's global network. + +### Installation + +```bash +pnpm add @tanstack/ai-isolate-cloudflare +``` + +### Usage + +```typescript +import { createCloudflareIsolateDriver } from '@tanstack/ai-isolate-cloudflare' + +const driver = createCloudflareIsolateDriver({ + workerUrl: 'https://my-code-mode-worker.my-account.workers.dev', + authorization: process.env.CODE_MODE_WORKER_SECRET, + timeout: 30_000, + maxToolRounds: 10, +}) +``` + +### Options + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `workerUrl` | `string` | — | **Required.** Full URL of the deployed Cloudflare Worker. | +| `authorization` | `string` | — | Optional value sent as the `Authorization` header on every request. Use this to prevent unauthorized access to your Worker. | +| `timeout` | `number` | `30000` | Maximum wall-clock time for the entire execution (including all tool round-trips), in milliseconds. | +| `maxToolRounds` | `number` | `10` | Maximum number of tool-call/result cycles per execution. Prevents infinite loops when generated code calls tools in a loop. | + +### Deploying the Worker + +The package exports a ready-made Worker handler at `@tanstack/ai-isolate-cloudflare/worker`. Create a `wrangler.toml` and a worker entry file: + +```toml +# wrangler.toml +name = "code-mode-worker" +main = "src/worker.ts" +compatibility_date = "2024-01-01" + +[unsafe] +bindings = [{ name = "eval", type = "eval" }] +``` + +```typescript +// src/worker.ts +export { default } from '@tanstack/ai-isolate-cloudflare/worker' +``` + +Deploy: + +```bash +wrangler deploy +``` + +### How it works + +The driver implements a request/response loop for tool execution: + +``` +Driver (your server) Worker (Cloudflare edge) +───────────────────── ───────────────────────── +Send: code + tool schemas ──────▶ Execute code + ◀────── Return: needs tool X with args Y +Execute tool X locally +Send: tool result ──────▶ Resume execution + ◀────── Return: final result / needs tool Z +...repeat until done... +``` + +Each round-trip adds network latency, so the `maxToolRounds` limit both prevents runaway scripts and caps the maximum number of cross-continent hops. Console output from all rounds is aggregated and returned in the final result. + +> **Security:** The Worker requires `UNSAFE_EVAL` (local dev) or the `eval` unsafe binding (production) to execute arbitrary code. Restrict access using the `authorization` option or Cloudflare Access policies. + +--- + +## The `IsolateDriver` Interface + +All three drivers satisfy this interface, exported from `@tanstack/ai-code-mode`: + +```typescript +import type { ToolBinding, NormalizedError } from "@tanstack/ai-code-mode"; + +interface IsolateDriver { + createContext(config: IsolateConfig): Promise +} + +interface IsolateConfig { + bindings: Record + timeout?: number + memoryLimit?: number +} + +interface IsolateContext { + execute(code: string): Promise + dispose(): Promise +} + +interface ExecutionResult { + success: boolean + value?: T + logs: Array + error?: NormalizedError +} +``` + +You can implement this interface to build a custom driver — for example, a Docker-based sandbox or a Deno subprocess. + +## Next Steps + +- [Code Mode](/ai/code-mode/code-mode) — Core setup, API reference, and getting started guide +- [Showing Code Mode in the UI](/ai/code-mode/client-integration) — Display execution progress in your React app +- [Code Mode with Skills](/ai/code-mode/code-mode-with-skills) — Add persistent, reusable skill libraries diff --git a/mintlify/ai/code-mode/code-mode-with-skills.md b/mintlify/ai/code-mode/code-mode-with-skills.md new file mode 100644 index 000000000..5cd6531e0 --- /dev/null +++ b/mintlify/ai/code-mode/code-mode-with-skills.md @@ -0,0 +1,368 @@ +--- +title: Code Mode with Skills +id: code-mode-with-skills +order: 3 +description: "Teach Code Mode to save and reuse working code as named skills backed by persistent storage — faster follow-up requests and composable agent memory." +keywords: + - tanstack ai + - code mode + - skills + - skill library + - register_skill + - reusable snippets + - agent memory + - skill storage +--- + +Skills extend [Code Mode](/ai/code-mode/code-mode) with a persistent library of reusable TypeScript snippets. When the LLM writes a useful piece of code — say, a function that fetches and ranks NPM packages — it can save that code as a _skill_. On future requests, relevant skills are loaded from storage and made available as first-class tools the LLM can call without re-writing the logic. + +> **Different from agent-authoring skills.** The skills on this page are _runtime_ snippets the chat LLM saves and reuses. If you're looking to teach your coding assistant (Claude Code, Cursor, etc.) how TanStack AI itself works, see [Agent Skills (TanStack Intent)](/ai/getting-started/agent-skills). + +## Overview + +The skills system has two integration paths: + +| Approach | Entry point | Skill selection | Best for | +|----------|-------------|----------------|----------| +| **High-level** | `codeModeWithSkills()` | Automatic (LLM-based) | New projects, turnkey setup | +| **Manual** | Individual functions (`skillsToTools`, `createSkillManagementTools`, etc.) | You decide which skills to load | Full control, existing setups | + +Both paths share the same storage, trust, and execution primitives — they differ only in how skills are selected and assembled. + +## How It Works + +A request with skills enabled goes through these stages: + +``` +┌─────────────────────────────────────────────────────┐ +│ 1. Load skill index (metadata only, no code) │ +├─────────────────────────────────────────────────────┤ +│ 2. Select relevant skills (LLM call — fast model) │ +├─────────────────────────────────────────────────────┤ +│ 3. Build tool registry │ +│ ├── execute_typescript (Code Mode sandbox) │ +│ ├── search_skills / get_skill / register_skill │ +│ └── skill tools (one per selected skill) │ +├─────────────────────────────────────────────────────┤ +│ 4. Generate system prompt │ +│ ├── Code Mode type stubs │ +│ └── Skill library documentation │ +├─────────────────────────────────────────────────────┤ +│ 5. Main chat() call (strong model) │ +│ ├── Can call skill tools directly │ +│ ├── Can write code via execute_typescript │ +│ └── Can register new skills for future use │ +└─────────────────────────────────────────────────────┘ +``` + +### LLM calls + +There are **two** LLM interactions per request when using the high-level API: + +1. **Skill selection** (`selectRelevantSkills`) — A single chat call using the adapter you provide. It sends the last 5 conversation messages plus a catalog of skill names/descriptions, and asks the model to return a JSON array of relevant skill names. This should be a cheap/fast model (e.g., `gpt-4o-mini`, `claude-haiku-4-5`). + +2. **Main chat** — The primary `chat()` call with your full model. This is where the LLM reasons, calls tools, writes code, and registers skills. + +The selection call is lightweight — it only sees skill metadata (names, descriptions, usage hints), not full code. If there are no skills in storage or no messages, it short-circuits and skips the LLM call entirely. + +## High-Level API: `codeModeWithSkills()` + +### Installation + +```bash +pnpm add @tanstack/ai-code-mode-skills +``` + +### Usage + +```typescript +import { chat, maxIterations, toServerSentEventsStream } from '@tanstack/ai' +import { createNodeIsolateDriver } from '@tanstack/ai-isolate-node' +import { codeModeWithSkills } from '@tanstack/ai-code-mode-skills' +import { createFileSkillStorage } from '@tanstack/ai-code-mode-skills/storage' +import { openaiText } from '@tanstack/ai-openai' +import { myTool1, myTool2 } from './tools' + +const messages = [{ role: 'user' as const, content: 'Hello' }] +const storage = createFileSkillStorage({ directory: './.skills' }) +const driver = createNodeIsolateDriver() + +const { toolsRegistry, systemPrompt, selectedSkills } = await codeModeWithSkills({ + config: { + driver, + tools: [myTool1, myTool2], + timeout: 60_000, + memoryLimit: 128, + }, + adapter: openaiText('gpt-5-mini'), // cheap model for skill selection + skills: { + storage, + maxSkillsInContext: 5, + }, + messages, // current conversation +}) + +const stream = chat({ + adapter: openaiText('gpt-5.5'), // strong model for reasoning + tools: toolsRegistry.getTools(), + messages, + systemPrompts: ['You are a helpful assistant.', systemPrompt], + agentLoopStrategy: maxIterations(15), +}) +``` + +`codeModeWithSkills` returns: + +| Property | Type | Description | +|----------|------|-------------| +| `toolsRegistry` | `ToolRegistry` | Mutable registry containing all tools. Pass to `chat()` via `tools: toolsRegistry.getTools()`. | +| `systemPrompt` | `string` | Combined Code Mode + skill library documentation. | +| `selectedSkills` | `Array` | Skills the selection model chose for this conversation. | + +### What goes into the registry + +The registry is populated with: + +- **`execute_typescript`** — The Code Mode sandbox tool. Inside the sandbox, skills are also available as `skill_*` functions (loaded dynamically at execution time). +- **`search_skills`** — Search the skill library by query. Returns matching skill metadata. +- **`get_skill`** — Retrieve full details (including code) for a specific skill. +- **`register_skill`** — Save working code as a new skill. Newly registered skills are immediately added to the registry as callable tools. +- **One tool per selected skill** — Each selected skill becomes a direct tool (prefixed with `[SKILL]` in its description) that the LLM can call without going through `execute_typescript`. + +## Manual API + +If you want full control — for example, loading all skills instead of using LLM-based selection — use the lower-level functions directly. This is the approach used in the `ts-code-mode-web` example. + +```typescript +import { chat, maxIterations } from '@tanstack/ai' +import { createCodeMode } from '@tanstack/ai-code-mode' +import { createNodeIsolateDriver } from '@tanstack/ai-isolate-node' +import { + createAlwaysTrustedStrategy, + createSkillManagementTools, + createSkillsSystemPrompt, + skillsToTools, +} from '@tanstack/ai-code-mode-skills' +import { createFileSkillStorage } from '@tanstack/ai-code-mode-skills/storage' +import { openaiText } from '@tanstack/ai-openai' +import { myTool1, myTool2, BASE_PROMPT } from './tools' + +const messages = [{ role: 'user' as const, content: 'Hello' }] +const trustStrategy = createAlwaysTrustedStrategy() +const storage = createFileSkillStorage({ + directory: './.skills', + trustStrategy, +}) +const driver = createNodeIsolateDriver() + +// 1. Create Code Mode tool + prompt +const { tool: codeModeTool, systemPrompt: codeModePrompt } = + createCodeMode({ + driver, + tools: [myTool1, myTool2], + timeout: 60_000, + memoryLimit: 128, + }) + +// 2. Load all skills and convert to tools +const allSkills = await storage.loadAll() +const skillIndex = await storage.loadIndex() + +const skillTools = allSkills.length > 0 + ? skillsToTools({ + skills: allSkills, + driver, + tools: [myTool1, myTool2], + storage, + timeout: 60_000, + memoryLimit: 128, + }) + : [] + +// 3. Create management tools +const managementTools = createSkillManagementTools({ + storage, + trustStrategy, +}) + +// 4. Generate skill library prompt +const skillsPrompt = createSkillsSystemPrompt({ + selectedSkills: allSkills, + totalSkillCount: skillIndex.length, + skillsAsTools: true, +}) + +// 5. Assemble and call chat() +const stream = chat({ + adapter: openaiText('gpt-5.5'), + tools: [codeModeTool, ...managementTools, ...skillTools], + messages, + systemPrompts: [BASE_PROMPT, codeModePrompt, skillsPrompt], + agentLoopStrategy: maxIterations(15), +}) +``` + +This approach skips the selection LLM call entirely — you load whichever skills you want and pass them in directly. + +## Skill Storage + +Skills are persisted through the `SkillStorage` interface. Two implementations are provided: + +### File storage (production) + +`createFileSkillStorage` is Node-only — it imports `node:fs` / `node:path` — so +it lives behind the `/storage` subpath rather than the package root. This keeps +the root export safe to bundle for Cloudflare Workers and browser builds; only +reach for the subpath in a Node runtime. + +```typescript +import { createFileSkillStorage } from '@tanstack/ai-code-mode-skills/storage' +import { createDefaultTrustStrategy } from '@tanstack/ai-code-mode-skills' + +const trustStrategy = createDefaultTrustStrategy() +const storage = createFileSkillStorage({ + directory: './.skills', + trustStrategy, // optional, defaults to createDefaultTrustStrategy() +}) +``` + +Creates a directory structure: + +``` +.skills/ + _index.json # Lightweight catalog for fast loading + fetch_github_stats/ + meta.json # Description, schemas, hints, stats + code.ts # TypeScript source + compare_npm_packages/ + meta.json + code.ts +``` + +### Memory storage (testing & edge runtimes) + +```typescript +import { createMemorySkillStorage } from '@tanstack/ai-code-mode-skills' + +const storage = createMemorySkillStorage() +``` + +Keeps everything in memory — no `node:fs` dependency, so it is re-exported from +the package root and is safe to use in Workers and browsers. Useful for tests, +demos, and edge deployments. (It is also available from the `/storage` subpath.) + +### Storage interface + +Both implementations satisfy this interface: + +| Method | Description | +|--------|-------------| +| `loadIndex()` | Load lightweight metadata for all skills (no code) | +| `loadAll()` | Load all skills with full details including code | +| `get(name)` | Get a single skill by name | +| `save(skill)` | Create or update a skill | +| `delete(name)` | Remove a skill | +| `search(query, options?)` | Search skills by text query | +| `updateStats(name, success)` | Record an execution result for trust tracking | + +## Trust Strategies + +Skills start untrusted and earn trust through successful executions. The trust level is metadata only — it does not currently gate execution. Four built-in strategies are available: + +```typescript group=code-mode-with-skills +import { + createDefaultTrustStrategy, + createAlwaysTrustedStrategy, + createRelaxedTrustStrategy, + createCustomTrustStrategy, +} from '@tanstack/ai-code-mode-skills' +``` + +| Strategy | Initial level | Provisional | Trusted | +|----------|--------------|-------------|---------| +| **Default** | `untrusted` | 10+ runs, ≥90% success | 100+ runs, ≥95% success | +| **Relaxed** | `untrusted` | 3+ runs, ≥80% success | 10+ runs, ≥90% success | +| **Always trusted** | `trusted` | — | — | +| **Custom** | Configurable | Configurable | Configurable | + +```typescript group=code-mode-with-skills +const strategy = createCustomTrustStrategy({ + initialLevel: 'untrusted', + provisionalThreshold: { executions: 5, successRate: 0.85 }, + trustedThreshold: { executions: 50, successRate: 0.95 }, +}) +``` + +## Skill Lifecycle + +### Registration + +When the LLM produces useful code via `execute_typescript`, the system prompt instructs it to call `register_skill` with: + +- `name` — snake_case identifier (becomes the tool name) +- `description` — what the skill does +- `code` — TypeScript source that receives an `input` variable +- `inputSchema` / `outputSchema` — JSON Schema strings +- `usageHints` — when to use this skill +- `dependsOn` — other skills this one calls + +The skill is saved to storage and (if a `ToolRegistry` was provided) immediately added as a callable tool in the current session. + +### Execution + +When a skill tool is called, the system: + +1. Wraps the skill code with `const input = ;` +2. Strips TypeScript syntax to plain JavaScript +3. Creates a fresh sandbox context with `external_*` bindings +4. Executes the code and returns the result +5. Updates execution stats (success/failure count) asynchronously + +### Selection (high-level API only) + +On each new request, `selectRelevantSkills`: + +1. Takes the last 5 conversation messages as context +2. Builds a catalog from the skill index (name + description + first usage hint) +3. Asks the adapter to return a JSON array of relevant skill names (max `maxSkillsInContext`) +4. Loads full skill data for the selected names + +If parsing fails or the model returns invalid JSON, it falls back to an empty selection — the request proceeds without pre-loaded skills, but the LLM can still search and use skills via the management tools. + +## Skills as Tools vs. Sandbox Bindings + +The `skillsAsTools` option (default: `true`) controls how skills are exposed: + +| Mode | How the LLM calls a skill | Pros | Cons | +|------|--------------------------|------|------| +| **As tools** (`true`) | Direct tool call: `skill_name({ ... })` | Simpler for the LLM, shows in tool-call UI, proper input validation | One tool per skill in the tool list | +| **As bindings** (`false`) | Inside `execute_typescript`: `await skill_fetch_data({ ... })` | Skills composable in code, fewer top-level tools | LLM must write code to use them | + +When `skillsAsTools` is enabled, the system prompt documents each skill with its schema, usage hints, and example calls. When disabled, skills appear as typed `skill_*` functions in the sandbox type stubs. + +## Custom Events + +Skill execution emits events through the TanStack AI event system: + +| Event | When | Payload | +|-------|------|---------| +| `code_mode:skill_call` | Skill tool invoked | `{ skill, input, timestamp }` | +| `code_mode:skill_result` | Skill completed successfully | `{ skill, result, duration, timestamp }` | +| `code_mode:skill_error` | Skill execution failed | `{ skill, error, duration, timestamp }` | +| `skill:registered` | New skill saved via `register_skill` | `{ id, name, description, timestamp }` | + +To render these events in your React app alongside Code Mode execution events, see [Showing Code Mode in the UI](/ai/code-mode/client-integration). + +## Tips + +- **Use a cheap model for selection.** The selection call only needs to match skill names to conversation context — `gpt-4o-mini` or `claude-haiku-4-5` work well. +- **Start without skills.** Get Code Mode working first, then add `@tanstack/ai-code-mode-skills` once you have tools that produce reusable patterns. +- **Monitor the skill count.** As the library grows, consider increasing `maxSkillsInContext` or switching to the manual API where you control which skills load. +- **Newly registered skills are available on the next message,** not in the current turn's tool list (unless using `ToolRegistry` with the high-level API, which adds them immediately). +- **Skills can call other skills.** Inside the sandbox, both `external_*` and `skill_*` functions are available. Set `dependsOn` when registering to document these relationships. + +## Next Steps + +- [Code Mode](/ai/code-mode/code-mode) — Core Code Mode setup and API reference +- [Showing Code Mode in the UI](/ai/code-mode/client-integration) — Display execution progress in your React app +- [Isolate Drivers](/ai/code-mode/code-mode-isolates) — Compare sandbox runtimes diff --git a/mintlify/ai/code-mode/code-mode.md b/mintlify/ai/code-mode/code-mode.md new file mode 100644 index 000000000..acd5c8cad --- /dev/null +++ b/mintlify/ai/code-mode/code-mode.md @@ -0,0 +1,295 @@ +--- +title: Code Mode +id: code-mode +order: 1 +description: "Let LLMs write and execute TypeScript programs that orchestrate tools in a secure sandbox with TanStack AI Code Mode — fewer loops, richer logic." +keywords: + - tanstack ai + - code mode + - sandbox + - typescript execution + - tool orchestration + - execute_typescript + - ai agents +--- + +Code Mode lets an LLM write and execute TypeScript programs inside a secure sandbox. Instead of making one tool call at a time, the model writes a short script that orchestrates multiple tools with loops, conditionals, `Promise.all`, and data transformations — then returns a single result. + +You already have a chat app that uses [tools](/ai/tools/tools). By the end of this guide, you'll have Code Mode set up so the LLM can compose those tools in TypeScript and execute them in a single sandbox call. + +## Why Code Mode? + +### Reduced context window usage + +In a traditional agentic loop, every tool call adds a round-trip of messages: the model's tool-call request, the tool result, then the model's next reasoning step. A task that touches five tools can easily consume thousands of tokens in back-and-forth. + +With Code Mode the model emits one `execute_typescript` call containing a complete program. The five tool invocations happen inside the sandbox, and only the final result comes back — one request, one response. + +### The LLM decides how to interpret tool output + +When tools are called individually, the model must decide what to do with each result in a new turn. With Code Mode, the model writes the logic up front: filter, aggregate, compare, branch. It can `Promise.all` ten API calls, pick the best result, and return a summary — all in a single execution. + +### Type-safe tool execution + +Tools you pass to Code Mode are converted to typed function stubs that appear in the system prompt. The model sees exact input/output types, so it generates correct calls without guessing parameter names or shapes. TypeScript annotations in the generated code are stripped automatically before execution. + +### Secure sandboxing + +Generated code runs in an isolated environment (V8 isolate, QuickJS WASM, or Cloudflare Worker) with no access to the host file system, network, or process. The sandbox has configurable timeouts and memory limits. + +## Getting Started + +### 1. Install packages + +```bash +pnpm add @tanstack/ai @tanstack/ai-code-mode zod +``` + +Pick an isolate driver: + +```bash +# Node.js — fastest, uses V8 isolates (requires native compilation) +pnpm add @tanstack/ai-isolate-node + +# QuickJS WASM — no native deps, works in browsers and edge runtimes +pnpm add @tanstack/ai-isolate-quickjs + +# Cloudflare Workers — run on the edge +pnpm add @tanstack/ai-isolate-cloudflare +``` + +### 2. Define tools + +Define your tools with `toolDefinition()` and provide a server-side implementation with `.server()`. These become the `external_*` functions available inside the sandbox. + +```typescript group=code-mode +import { toolDefinition } from "@tanstack/ai"; +import { z } from "zod"; + +const fetchWeather = toolDefinition({ + name: "fetchWeather", + description: "Get current weather for a city", + inputSchema: z.object({ location: z.string() }), + outputSchema: z.object({ + temperature: z.number(), + condition: z.string(), + }), +}).server(async ({ location }) => { + const res = await fetch(`https://api.weather.example/v1?city=${location}`); + return res.json(); +}); +``` + +### 3. Create the Code Mode tool and system prompt + +```typescript group=code-mode +import { createCodeMode } from "@tanstack/ai-code-mode"; +import { createNodeIsolateDriver } from "@tanstack/ai-isolate-node"; + +const { tool, systemPrompt } = createCodeMode({ + driver: createNodeIsolateDriver(), + tools: [fetchWeather], + timeout: 30_000, +}); +``` + +### 4. Use with `chat()` + +```typescript group=code-mode +import { chat } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +const result = await chat({ + adapter: openaiText("gpt-5.5"), + systemPrompts: [ + "You are a helpful weather assistant.", + systemPrompt, + ], + tools: [tool], + messages: [ + { + role: "user", + content: "Compare the weather in Tokyo, Paris, and New York City", + }, + ], +}); +``` + +The model will generate something like: + +```typescript ignore +const cities = ["Tokyo", "Paris", "New York City"]; +const results = await Promise.all( + cities.map((city) => external_fetchWeather({ location: city })) +); + +const warmest = results.reduce((prev, curr) => + curr.temperature > prev.temperature ? curr : prev +); + +return { + comparison: results.map((r, i) => ({ + city: cities[i], + temperature: r.temperature, + condition: r.condition, + })), + warmest: cities[results.indexOf(warmest)], +}; +``` + +All three API calls happen in parallel inside the sandbox. The model receives one structured result instead of three separate tool-call round-trips. + +## API Reference + +### `createCodeMode(config)` + +Creates both the `execute_typescript` tool and its matching system prompt from a single config object. This is the recommended entry point. + +```typescript ignore +const { tool, systemPrompt } = createCodeMode({ + driver, // IsolateDriver — required + tools, // Array — required, at least one + timeout, // number — execution timeout in ms (default: 30000) + memoryLimit, // number — memory limit in MB (default: 128, Node + QuickJS drivers) + getSkillBindings, // () => Promise> — optional dynamic bindings +}); +``` + +**Config properties:** + +| Property | Type | Description | +|----------|------|-------------| +| `driver` | `IsolateDriver` | The sandbox runtime to execute code in | +| `tools` | `Array` | Tools exposed as `external_*` functions. Must have `.server()` implementations | +| `timeout` | `number` | Execution timeout in milliseconds (default: 30000) | +| `memoryLimit` | `number` | Memory limit in MB (default: 128). Supported by Node and QuickJS drivers | +| `getSkillBindings` | `() => Promise>` | Optional function returning additional bindings at execution time | + +The tool returns a `CodeModeToolResult`: + +```typescript +interface CodeModeToolResult { + success: boolean; + result?: unknown; // Return value from the executed code + logs?: Array; // Captured console output + error?: { + message: string; + name?: string; + line?: number; + }; +} +``` + +### `createCodeModeTool(config)` / `createCodeModeSystemPrompt(config)` + +Lower-level functions if you need only the tool or only the prompt. `createCodeMode` calls both internally. + +```typescript +import { createCodeModeTool, createCodeModeSystemPrompt } from "@tanstack/ai-code-mode"; +import { config } from "./config"; + +const tool = createCodeModeTool(config); +const prompt = createCodeModeSystemPrompt(config); +``` + +### `IsolateDriver` + +The interface that sandbox runtimes implement. You do not implement this yourself — pick one of the provided drivers: + +```typescript +import type { IsolateConfig, IsolateContext } from "@tanstack/ai-code-mode"; + +interface IsolateDriver { + createContext(config: IsolateConfig): Promise; +} +``` + +**Available drivers:** + +| Package | Factory function | Environment | +|---------|-----------------|-------------| +| `@tanstack/ai-isolate-node` | `createNodeIsolateDriver()` | Node.js | +| `@tanstack/ai-isolate-quickjs` | `createQuickJSIsolateDriver()` | Node.js, browser, edge | +| `@tanstack/ai-isolate-cloudflare` | `createCloudflareIsolateDriver()` | Cloudflare Workers | + +For full configuration options for each driver, see [Isolate Drivers](/ai/code-mode/code-mode-isolates). + +### Advanced + +These utilities are used internally and are exported for custom pipelines: + +- **`stripTypeScript(code)`** — Strips TypeScript syntax using sucrase (edge-safe, no native binary), converting to plain JavaScript. +- **`toolsToBindings(tools, prefix?)`** — Converts TanStack AI tools into `Record` for sandbox injection. +- **`generateTypeStubs(bindings, options?)`** — Generates TypeScript type declarations from tool bindings for system prompts. + +## Choosing a Driver + +For a full comparison of drivers with all configuration options, see [Isolate Drivers](/ai/code-mode/code-mode-isolates). + +In brief: use the **Node driver** for server-side Node.js (fastest, V8 JIT), **QuickJS** for browsers or portable edge deployments (no native deps), and the **Cloudflare driver** when you deploy to Cloudflare Workers. + +## Custom Events + +Code Mode emits custom events during execution that you can observe through the TanStack AI event system. These are useful for building UIs that show execution progress, debugging, or logging. + +| Event | When | Payload | +|-------|------|---------| +| `code_mode:execution_started` | Code execution begins | `{ timestamp, codeLength }` | +| `code_mode:console` | Each `console.log/error/warn/info` call | `{ level, message, timestamp }` | +| `code_mode:external_call` | Before an `external_*` function runs | `{ function, args, timestamp }` | +| `code_mode:external_result` | After a successful `external_*` call | `{ function, result, duration }` | +| `code_mode:external_error` | When an `external_*` call fails | `{ function, error, duration }` | + +To display these events in your React app, see [Showing Code Mode in the UI](/ai/code-mode/client-integration). + +## Model Compatibility + +Code Mode asks the model to write valid TypeScript that calls your tools through the sandbox bridge. Not every model handles this equally — many small or older models mishandle the `external_*` calling conventions even when the system prompt is explicit. We track a single multi-step benchmark (joining three tables, filtering customers who bought from every product category, aggregating spend per category) against a gold reference. The full harness lives at `packages/ai-code-mode/models-eval/`. + +| Rank | Model | Stars | Acc | Comp | TS | CME | Latency | Tokens | +|------|-------|:-----:|:---:|:----:|:--:|:---:|--------:|-------:| +| 1 | `grok:grok-4-1-fast-non-reasoning` | ★★★ | 10 | 9 | 6 | 10 | 7.0s | — | +| 2 | `ollama:gpt-oss:20b` | ★★★ | 10 | 8 | 6 | 5 | 45.1s | 23.6k | +| 3 | `anthropic:claude-haiku-4-5` | ★★★ | 10 | 10 | 7 | 10 | 9.4s | 8.5k | +| 4 | `gemini:gemini-2.5-flash` | ★★★ | 10 | 7 | 5 | 9 | 7.3s | 6.9k | +| 5 | `ollama:nemotron-cascade-2` | ★★★ | 10 | 9 | 5 | 5 | 60.4s | 11.7k | +| 6 | `openai:gpt-4o-mini` | ★★☆ | 10 | 8 | 8 | 10 | 19.2s | 8.7k | +| 7 | `ollama:gemma4:31b` | ★★☆ | 10 | 8 | 4 | 5 | 264.2s | 6.4k | + +**Columns** + +- **Stars** — overall weighted rating (1-3) combining accuracy, comprehensiveness, code quality, code-mode efficiency, speed, token efficiency, and stability. +- **Acc / Comp / TS / CME** — Anthropic-judged subscores out of 10: accuracy vs gold, comprehensiveness, TypeScript quality, code-mode efficiency (fewer wasted attempts is better). +- **Latency** — wall-clock time for the full agentic loop. +- **Tokens** — total prompt + completion tokens. Grok's adapter does not report usage. + +**Takeaways** + +- **Strongest cloud picks:** Grok 4.1 Fast, Claude Haiku 4.5, and Gemini 2.5 Flash all finish under 10s and handle the multi-step task cleanly. Claude Haiku 4.5 has the highest comprehensiveness score (10/10). +- **Strongest local pick:** `ollama:gpt-oss:20b` is the best local performer at 45s with zero compilation failures. `ollama:nemotron-cascade-2` is a close second. +- **Avoid:** the smaller `gemma4` (9.6 GB) and the other local models commented out at the top of `eval-config.ts` (`granite4:3b`, `ministral-3`, `mistral:7b`, `qwen3:8b`, etc.) — they either ignore the `external_queryTable` shape, hallucinate results, or refuse to invoke `execute_typescript`. +- **Caveat:** this is a single‑prompt benchmark. Local model results can vary noticeably between runs; use these as a rough capability filter rather than a definitive ranking. + +Reproduce locally: + +```bash +cd packages/ai-code-mode/models-eval +pnpm install +pnpm eval # full suite (needs cloud API keys + Anthropic for judging) +pnpm eval -- --ollama-only # local models only +pnpm eval -- --no-judge # skip Anthropic-based judging +``` + +## Tips + +- **Start simple.** Give the model 2-3 tools and a clear task. Code Mode works best when the model has a focused set of capabilities. +- **Prefer `Promise.all` tasks.** Code Mode shines when the model can parallelize work that would otherwise be sequential tool calls. +- **Use `console.log` for debugging.** Logs are captured and returned in the result, making it easy to see what happened inside the sandbox. +- **Keep tools focused.** Each tool should do one thing well. The model will compose them in code. +- **Check the system prompt.** Call `createCodeModeSystemPrompt(config)` and inspect the output to see exactly what the model will see, including generated type stubs. + +## Next Steps + +- [Showing Code Mode in the UI](/ai/code-mode/client-integration) — Display execution progress in your React app +- [Code Mode with Skills](/ai/code-mode/code-mode-with-skills) — Add persistent, reusable skill libraries +- [Isolate Drivers](/ai/code-mode/code-mode-isolates) — Compare Node, QuickJS, and Cloudflare sandbox runtimes diff --git a/mintlify/ai/code-mode/lazy-tools.md b/mintlify/ai/code-mode/lazy-tools.md new file mode 100644 index 000000000..ee158b2dd --- /dev/null +++ b/mintlify/ai/code-mode/lazy-tools.md @@ -0,0 +1,206 @@ +--- +title: Lazy Tools +id: lazy-tools +order: 5 +description: "Keep large tool catalogs out of the Code Mode system prompt with lazy tools — the model fetches TypeScript signatures on demand via a discover_tools call." +keywords: + - tanstack ai + - code mode + - lazy tools + - discover_tools + - progressive disclosure + - prompt size + - tool catalog +--- + +Large tool catalogs bloat the `execute_typescript` system prompt. Every tool you pass to `createCodeMode` becomes a full TypeScript type stub in that prompt — and at 50+ tools, those stubs can push the effective prompt into the tens of thousands of tokens before the model has even seen your user message. + +Lazy tools fix this with **progressive disclosure**: mark rarely-used tools `lazy: true` and they are withheld from the initial system prompt. The model sees only their names in a short "Discoverable APIs" catalog. When it needs one, it calls the `discover_tools` sibling tool to fetch the TypeScript signature on demand, then uses it inside `execute_typescript`. All sandbox bindings are always injected — lazy only defers _documentation_, not callability. + +## Marking a Tool Lazy + +Add `lazy: true` to the `toolDefinition` config for any tool you want to defer: + +```typescript group=lazy-tools +import { toolDefinition } from "@tanstack/ai"; +import { z } from "zod"; + +// Always eager — documented upfront +const fetchWeather = toolDefinition({ + name: "fetchWeather", + description: "Get current weather for a city", + inputSchema: z.object({ location: z.string() }), + outputSchema: z.object({ temperature: z.number(), condition: z.string() }), +}).server(async ({ location }) => { + const res = await fetch(`https://api.weather.example/v1?city=${location}`); + return res.json(); +}); + +// Lazy — kept out of the system prompt until discovered +const fetchArchive = toolDefinition({ + name: "fetchArchive", + description: "Retrieve historical weather archive data for a date range", + inputSchema: z.object({ + location: z.string(), + from: z.string(), + to: z.string(), + }), + outputSchema: z.array(z.object({ date: z.string(), temperature: z.number() })), + lazy: true, +}).server(async ({ location, from, to }) => { + const res = await fetch( + `https://api.weather.example/v1/archive?city=${location}&from=${from}&to=${to}` + ); + return res.json(); +}); +``` + +Eager tools continue to receive full type stubs in the system prompt. Lazy tools appear only by name. + +## Server Setup + +Pass both eager and lazy tools to `createCodeMode`. When at least one tool is lazy, `createCodeMode` also returns a `discover_tools` sibling tool — include it in the `tools` array you pass to `chat()`: + +```typescript group=lazy-tools +// server/route.ts +import { chat, maxIterations, toServerSentEventsStream } from "@tanstack/ai"; +import { createCodeMode } from "@tanstack/ai-code-mode"; +import { createNodeIsolateDriver } from "@tanstack/ai-isolate-node"; +import { openaiText } from "@tanstack/ai-openai"; + +const { tools, systemPrompt } = createCodeMode({ + driver: createNodeIsolateDriver(), + tools: [fetchWeather, fetchArchive], // fetchArchive is lazy +}); + +// tools is [execute_typescript, discover_tools] +// — discover_tools is included automatically because fetchArchive is lazy + +export async function POST(req: Request) { + const { messages } = await req.json(); + + const stream = chat({ + adapter: openaiText("gpt-5.5"), + systemPrompts: ["You are a helpful weather assistant.", systemPrompt], + tools: [...tools], + messages, + agentLoopStrategy: maxIterations(10), + }); + + return toServerSentEventsStream(stream); +} +``` + +`createCodeMode` returns `{ tool, discoveryTool, tools, systemPrompt }`: + +| Field | Type | Description | +|-------|------|-------------| +| `tool` | `ServerTool` | The `execute_typescript` tool (backward compatible) | +| `discoveryTool` | `ServerTool \| null` | The `discover_tools` tool, or `null` when there are no lazy tools | +| `tools` | `Array` | `[tool]` or `[tool, discoveryTool]` — spread into `chat({ tools })` | +| `systemPrompt` | `string` | The matching system prompt | + +If no tools are lazy, `discoveryTool` is `null` and `tools` contains only `execute_typescript`. + +## The `discover_tools` Flow + +When the model encounters a task that requires a lazy tool, it: + +1. Calls `discover_tools` with the tool name (bare name, no `external_` prefix). +2. Receives the TypeScript type stub and description for that tool. +3. Writes `execute_typescript` code using the now-documented `external_fetchArchive(...)` call. + +The bindings are always injected into the sandbox — discovering a tool only retrieves documentation, it does not enable the binding. The model could call `external_fetchArchive` without discovering it first, but it would be writing blind without the type signature. + +## Tuning the Discoverable APIs Catalog + +By default, lazy tools appear in the system prompt as bare names with no description: + +```text +### Discoverable APIs + +- external_fetchArchive +- external_runReport +- external_exportData +``` + +If you want the model to have a hint about what each tool does before deciding whether to discover it, use `lazyToolsConfig.includeDescription`: + +```typescript +import { createCodeMode } from "@tanstack/ai-code-mode"; +import { createNodeIsolateDriver } from "@tanstack/ai-isolate-node"; +import { + fetchWeather, + fetchArchive, + runReport, + exportData, +} from "./tools"; + +const { tools, systemPrompt } = createCodeMode({ + driver: createNodeIsolateDriver(), + tools: [fetchWeather, fetchArchive, runReport, exportData], + lazyToolsConfig: { + includeDescription: "first-sentence", // 'none' | 'first-sentence' | 'full' + }, +}); +``` + +With `'first-sentence'` the catalog becomes: + +```text +### Discoverable APIs + +- external_fetchArchive — Retrieve historical weather archive data for a date range. +- external_runReport — Generate a summary report for a given time period. +- external_exportData — Export query results to CSV or JSON format. +``` + +| Value | Effect | +|-------|--------| +| `'none'` (default) | Bare names only — smallest possible prompt addition | +| `'first-sentence'` | Name plus the first sentence of the tool's description | +| `'full'` | Name plus the complete description | + +The full type stub and input/output schema are always returned on discovery — `includeDescription` only affects the pre-discovery catalog. + +## Lazy Tools with Plain `chat()` + +The same `lazyToolsConfig` option works for lazy tools used directly with `chat()`, outside of Code Mode. Tools marked `lazy: true` are withheld from the `__lazy__tool__discovery__` catalog description until the model calls for them. Pass `lazyToolsConfig` directly to `chat()`: + +```typescript +import { chat, maxIterations, toServerSentEventsStream } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { fetchWeather, fetchArchive, runReport } from "./tools"; + +// Non-code-mode: lazy tools in a regular chat agent +export async function POST(req: Request) { + const { messages } = await req.json(); + + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages, + tools: [fetchWeather, fetchArchive, runReport], + lazyToolsConfig: { + includeDescription: "first-sentence", + }, + agentLoopStrategy: maxIterations(10), + }); + + return toServerSentEventsStream(stream); +} +``` + +The `includeDescription` behavior is identical — `'none'` lists bare tool names, `'first-sentence'` appends the first sentence, `'full'` appends the complete description. + +## Tips + +- **Start with `'none'`.** The bare-names catalog is enough for models that reason well about tool names. Add `'first-sentence'` only if the model frequently discovers irrelevant tools. +- **Lazy tools are always callable.** Their `external_*` bindings are injected into the sandbox regardless of whether the model has called `discover_tools`. Discovery only reveals documentation. +- **Use `discoveryTool` for observability.** You can inspect `discoveryTool.name` (`"discover_tools"`) to confirm the tool is wired up, or log its calls for analytics. +- **Partition by frequency, not capability.** Mark tools lazy when they are rarely needed for a typical request. Core tools that most requests use should stay eager. + +## Next Steps + +- [Code Mode](/ai/code-mode/code-mode) — Core Code Mode setup and API reference +- [Code Mode with Skills](/ai/code-mode/code-mode-with-skills) — Persistent reusable skill libraries +- [Isolate Drivers](/ai/code-mode/code-mode-isolates) — Compare Node, QuickJS, and Cloudflare sandbox runtimes diff --git a/mintlify/ai/community-adapters/cencori.md b/mintlify/ai/community-adapters/cencori.md new file mode 100644 index 000000000..dc6f52c9e --- /dev/null +++ b/mintlify/ai/community-adapters/cencori.md @@ -0,0 +1,206 @@ +--- +title: Cencori +id: cencori-adapter +order: 3 +description: "Access 14+ AI providers (OpenAI, Anthropic, Google, xAI, and more) through Cencori's unified interface with built-in security, observability, and cost tracking in TanStack AI." +keywords: + - tanstack ai + - cencori + - multi-provider + - observability + - cost tracking + - security + - community adapter +--- + +The Cencori adapter provides access to 14+ AI providers (OpenAI, Anthropic, Google, xAI, and more) through a unified interface with built-in security, observability, and cost tracking. + +## Installation + +```bash +npm install @cencori/ai-sdk +``` + +## Basic Usage + +```typescript ignore +// ignore: @cencori/ai-sdk/tanstack is a subpath export; kiira's paths["*"] wildcard maps it +// to a flat directory lookup and does not consult the package.json exports field, +// so the subpath cannot be resolved until kiira.config.ts adds an explicit path entry. +import { chat } from "@tanstack/ai"; +import { cencori } from "@cencori/ai-sdk/tanstack"; + +const adapter = cencori("o1"); + +for await (const chunk of chat({ + adapter, + messages: [{ role: "user", content: "Hello!" }], +})) { + if (chunk.type === "TEXT_MESSAGE_CONTENT") { + console.log(chunk.delta); + } +} +``` + +## Configuration + +```typescript ignore +// ignore: @cencori/ai-sdk/tanstack subpath not resolvable via kiira's paths["*"] wildcard. +import { createCencori } from "@cencori/ai-sdk/tanstack"; + +const myCencori = createCencori({ + apiKey: process.env.CENCORI_API_KEY!, + baseUrl: "https://cencori.com", // Optional +}); + +const adapter = myCencori("o1"); +``` + +## Streaming + +```typescript ignore +// ignore: @cencori/ai-sdk/tanstack subpath not resolvable via kiira's paths["*"] wildcard. +import { chat } from "@tanstack/ai"; +import { cencori } from "@cencori/ai-sdk/tanstack"; + +const adapter = cencori("claude-3-5-sonnet"); + +for await (const chunk of chat({ + adapter, + messages: [{ role: "user", content: "Tell me a story" }], +})) { + if (chunk.type === "TEXT_MESSAGE_CONTENT") { + process.stdout.write(chunk.delta); + } else if (chunk.type === "RUN_FINISHED") { + console.log("\nDone"); + } +} +``` + + +## Tool Calling + +```typescript ignore +// ignore: @cencori/ai-sdk/tanstack subpath not resolvable via kiira's paths["*"] wildcard. +import { chat, toolDefinition } from "@tanstack/ai"; +import { cencori } from "@cencori/ai-sdk/tanstack"; +import { z } from "zod"; + +const adapter = cencori("o1"); + +const getWeatherDef = toolDefinition({ + name: "getWeather", + description: "Get weather for a location", + inputSchema: z.object({ location: z.string() }), +}); + +const getWeather = getWeatherDef.server(async ({ location }) => { + // Look up the weather for `location` + return { temperature: 72, conditions: "Sunny" }; +}); + +for await (const chunk of chat({ + adapter, + messages: [{ role: "user", content: "What's the weather in NYC?" }], + tools: [getWeather], +})) { + if (chunk.type === "TOOL_CALL_START") { + console.log("Tool call:", chunk.toolCallName); + } else if (chunk.type === "TOOL_CALL_END") { + console.log("Tool call finished:", chunk.toolCallId); + } +} +``` + + +## Multi-Provider Support + +Switch between providers with a single parameter: + +```typescript ignore +// ignore: @cencori/ai-sdk/tanstack subpath not resolvable via kiira's paths["*"] wildcard. +import { cencori } from "@cencori/ai-sdk/tanstack"; + +// OpenAI-compatible +const openaiCompat = cencori("o1"); + +// Anthropic +const anthropic = cencori("claude-3-5-sonnet"); + +// Google +const google = cencori("gemini-2.5-flash"); + +// xAI +const grok = cencori("grok-3"); + +// DeepSeek +const deepseek = cencori("deepseek-v3.2"); +``` + +All responses use the same unified format regardless of provider. + +## Supported Models + +| Provider | Models | +|----------|--------| +| OpenAI | `gpt-5`, `gpt-4o`, `gpt-4o-mini`, `o3`, `o1` | +| Anthropic | `claude-opus-4`, `claude-sonnet-4`, `claude-3-5-sonnet` | +| Google | `gemini-3-pro`, `gemini-2.5-flash`, `gemini-2.0-flash` | +| xAI | `grok-4`, `grok-3` | +| Mistral | `mistral-large`, `codestral`, `devstral` | +| DeepSeek | `deepseek-v3.2`, `deepseek-reasoner` | +| + More | Groq, Cohere, Perplexity, Together, Qwen, OpenRouter | + +> **Note:** Cencori is an external package and its catalogue changes over time. Verify the model ids above against [Cencori's current catalogue](https://cencori.com/docs) before relying on them. + +## Environment Variables + +```bash +CENCORI_API_KEY=csk_your_api_key_here +``` + +## Getting an API Key + +1. Go to [Cencori](https://cencori.com) +2. Create an account and generate an API key +3. Add it to your environment variables + +## Why Cencori? + +- **🔒 Security** — PII filtering, jailbreak detection, content moderation +- **📊 Observability** — Request logs, latency metrics, cost tracking +- **💰 Cost Control** — Budgets, alerts, per-route analytics +- **🔌 Multi-Provider** — One API key for 14+ AI providers +- **🛠️ Tool Calling** — Full support for function calling across providers +- **🔄 Failover** — Automatic retry and fallback to alternative providers + +## API Reference + +### `cencori(model)` + +Creates a Cencori adapter using environment variables. + +**Parameters:** + +- `model` - Model name (e.g., `"gpt-4o"`, `"claude-3-5-sonnet"`, `"gemini-2.5-flash"`) + +**Returns:** A Cencori TanStack AI adapter instance. + +### `createCencori(config)` + +Creates a custom Cencori adapter factory. + +**Parameters:** + +- `config.apiKey` - Your Cencori API key +- `config.baseUrl?` - Custom base URL (optional) + +**Returns:** A function that creates adapter instances for specific models. + +## Next Steps + +- [Cencori Dashboard](https://cencori.com) — View analytics, logs, and costs +- [Documentation](https://cencori.com/docs) — Complete API reference +- [GitHub Repository](https://github.com/cencori/cencori) — SDK source code +- [Streaming Guide](/ai/chat/streaming) — Learn about streaming responses +- [Tools Guide](/ai/tools/tools) — Learn about tool calling diff --git a/mintlify/ai/community-adapters/cloudflare.md b/mintlify/ai/community-adapters/cloudflare.md new file mode 100644 index 000000000..b37f35ceb --- /dev/null +++ b/mintlify/ai/community-adapters/cloudflare.md @@ -0,0 +1,319 @@ +--- +title: Cloudflare +id: cloudflare-adapter +order: 3 +description: "Use Cloudflare Workers AI and AI Gateway with TanStack AI for edge inference, caching, rate limiting, and unified billing across providers." +keywords: + - tanstack ai + - cloudflare + - workers ai + - ai gateway + - edge inference + - caching + - rate limiting + - community adapter +--- + +The Cloudflare adapter provides access to [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/) models and [AI Gateway](https://developers.cloudflare.com/ai-gateway/) for routing requests to OpenAI, Anthropic, Gemini, Grok, and OpenRouter with caching, rate limiting, and unified billing. + +## Installation + +```bash +npm install @cloudflare/tanstack-ai @tanstack/ai +``` + +For AI Gateway with third-party providers, install the provider SDKs you need: + +```bash +npm install @tanstack/ai-openai # For OpenAI via Gateway +npm install @tanstack/ai-anthropic # For Anthropic via Gateway +npm install @tanstack/ai-gemini # For Gemini via Gateway +npm install @tanstack/ai-grok # For Grok via Gateway +npm install @tanstack/ai-openrouter # For OpenRouter via Gateway +``` + +## Basic Usage + +```typescript +import { chat, toHttpResponse } from "@tanstack/ai"; +import { createWorkersAiChat } from "@cloudflare/tanstack-ai"; +import { env } from "./env"; + +const adapter = createWorkersAiChat( + "@cf/meta/llama-4-scout-17b-16e-instruct", + { binding: env.AI }, +); + +const response = chat({ + adapter, + stream: true, + messages: [{ role: "user", content: "Hello!" }], +}); + +const httpResponse = toHttpResponse(response); +``` + +## Workers AI + +The simplest way to use AI in a Cloudflare Worker. No API keys needed when using the `env.AI` binding. + +### Chat + +```typescript +import { chat, toHttpResponse } from "@tanstack/ai"; +import { createWorkersAiChat } from "@cloudflare/tanstack-ai"; +import { env } from "./env"; + +const adapter = createWorkersAiChat( + "@cf/meta/llama-4-scout-17b-16e-instruct", + { binding: env.AI }, +); + +const response = chat({ + adapter, + stream: true, + messages: [{ role: "user", content: "Hello!" }], +}); + +const httpResponse = toHttpResponse(response); +``` + +### Chat with REST Credentials + +If you're not running inside a Worker, use account ID and API key instead: + +```typescript +import { createWorkersAiChat } from "@cloudflare/tanstack-ai"; + +const adapter = createWorkersAiChat( + "@cf/meta/llama-4-scout-17b-16e-instruct", + { + accountId: "your-account-id", + apiKey: "your-api-key", + }, +); +``` + +### Image Generation + +```typescript +import { generateImage } from "@tanstack/ai"; +import { createWorkersAiImage } from "@cloudflare/tanstack-ai"; +import { env } from "./env"; + +const adapter = createWorkersAiImage( + "@cf/stabilityai/stable-diffusion-xl-base-1.0", + { binding: env.AI }, +); + +const result = await generateImage({ + adapter, + prompt: "a cat in space", +}); + +console.log(result.images[0]?.b64Json); +``` + +### Transcription (Speech-to-Text) + +Supports Whisper and Deepgram models: + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { createWorkersAiTranscription } from "@cloudflare/tanstack-ai"; +import { env, audioArrayBuffer } from "./env"; + +const adapter = createWorkersAiTranscription( + "@cf/openai/whisper-large-v3-turbo", + { binding: env.AI }, +); + +const result = await generateTranscription({ + adapter, + audio: audioArrayBuffer, +}); + +console.log(result.text); +console.log(result.segments); +``` + +Supported transcription models: `@cf/openai/whisper`, `@cf/openai/whisper-tiny-en`, `@cf/openai/whisper-large-v3-turbo`, `@cf/deepgram/nova-3` + +### Text-to-Speech + +```typescript +import { generateSpeech } from "@tanstack/ai"; +import { createWorkersAiTts } from "@cloudflare/tanstack-ai"; +import { env } from "./env"; + +const adapter = createWorkersAiTts("@cf/deepgram/aura-2-en", { + binding: env.AI, +}); + +const result = await generateSpeech({ + adapter, + text: "Hello world", +}); + +console.log(result.audio); +``` + +### Summarization + +```typescript +import { summarize, type AnySummarizeAdapter } from "@tanstack/ai"; +import { createWorkersAiSummarize } from "@cloudflare/tanstack-ai"; +import { env } from "./env"; + +const adapter: AnySummarizeAdapter = createWorkersAiSummarize("@cf/facebook/bart-large-cnn", { + binding: env.AI, +}); + +const result = await summarize({ + adapter, + text: "Long article here...", +}); + +console.log(result.summary); +``` + +## AI Gateway + +Route AI requests through Cloudflare's AI Gateway for caching, rate limiting, and unified billing. Supports both Workers AI and third-party providers. + +### Workers AI through Gateway + +```typescript +import { createWorkersAiChat } from "@cloudflare/tanstack-ai"; +import { env } from "./env"; + +const adapter = createWorkersAiChat( + "@cf/meta/llama-4-scout-17b-16e-instruct", + { + binding: env.AI.gateway("my-gateway-id"), + apiKey: env.WORKERS_AI_TOKEN, + }, +); +``` + +### Third-Party Providers through Gateway + +Use the binding approach (recommended for Cloudflare Workers): + +```typescript +import { + createOpenAiChat, + createAnthropicChat, + createGeminiChat, + createGrokChat, + createOpenRouterChat, +} from "@cloudflare/tanstack-ai"; +import { env } from "./env"; + +const openai = createOpenAiChat("gpt-4o", { + binding: env.AI.gateway("my-gateway-id"), +}); + +const anthropic = createAnthropicChat("claude-sonnet-4-5", { + binding: env.AI.gateway("my-gateway-id"), +}); + +const grok = createGrokChat("grok-4.3", { + binding: env.AI.gateway("my-gateway-id"), +}); + +const openrouter = createOpenRouterChat("openai/gpt-4o", { + binding: env.AI.gateway("my-gateway-id"), +}); +``` + +Or use credentials for non-Worker environments: + +```typescript +import { createOpenAiChat } from "@cloudflare/tanstack-ai"; + +const adapter = createOpenAiChat("gpt-4o", { + accountId: "your-account-id", + gatewayId: "your-gateway-id", + cfApiKey: "your-cf-api-key", + apiKey: "provider-api-key", +}); +``` + +### Cache Options + +Both binding and credentials modes support cache configuration: + +```typescript +import { createOpenAiChat } from "@cloudflare/tanstack-ai"; +import { env } from "./env"; + +const adapter = createOpenAiChat("gpt-4o", { + binding: env.AI.gateway("my-gateway-id"), + skipCache: false, + cacheTtl: 3600, + customCacheKey: "my-key", + metadata: { user: "test" }, +}); +``` + +## Configuration Modes + +Workers AI supports four configuration modes: + +| Mode | Config | Description | +|------|--------|-------------| +| Plain binding | `{ binding: env.AI }` | Direct access, no gateway | +| Plain REST | `{ accountId, apiKey }` | REST API, no gateway | +| Gateway binding | `{ binding: env.AI.gateway(id) }` | Through AI Gateway via binding | +| Gateway REST | `{ accountId, gatewayId, ... }` | Through AI Gateway via REST | + +Third-party providers (OpenAI, Anthropic, Gemini, Grok, OpenRouter) only support the gateway modes. + +## Supported Capabilities + +| Provider | Chat | Summarize | Image Gen | Transcription | TTS | +|----------|------|-----------|-----------|---------------|-----| +| **Workers AI** | ✅ | ✅ | ✅ | ✅ | ✅ | +| **OpenAI** | ✅ | ✅ | ✅ | ✅ | ✅ | +| **Anthropic** | ✅ | ✅ | ❌ | ❌ | ❌ | +| **Gemini** | ✅ | ✅ | ✅ | ❌ | ✅ | +| **Grok** | ✅ | ✅ | ✅ | ❌ | ❌ | +| **OpenRouter** | ✅ | ✅ | ✅ | ❌ | ❌ | + +## Environment Variables + +For the REST credential path (outside of Cloudflare Workers): + +```bash +CLOUDFLARE_ACCOUNT_ID=your-account-id +CLOUDFLARE_API_KEY=your-api-key +``` + +When using the `env.AI` binding inside a Worker, no environment variables are needed. + +## API Reference + +### Workers AI + +- `createWorkersAiChat(model, config)` — Chat and structured output +- `createWorkersAiImage(model, config)` — Image generation +- `createWorkersAiTranscription(model, config)` — Speech-to-text (Whisper, Deepgram) +- `createWorkersAiTts(model, config)` — Text-to-speech (Deepgram Aura) +- `createWorkersAiSummarize(model, config)` — Summarization (BART-large-CNN) + +### Gateway Providers + +- `createOpenAiChat(model, config)` / `createOpenAiSummarize` / `createOpenAiImage` / `createOpenAiTranscription` / `createOpenAiTts` +- `createAnthropicChat(model, config)` / `createAnthropicSummarize` +- `createGeminiChat(model, config)` / `createGeminiSummarize` / `createGeminiImage` / `createGeminiTts` +- `createGrokChat(model, config)` / `createGrokSummarize` / `createGrokImage` +- `createOpenRouterChat(model, config)` / `createOpenRouterSummarize` / `createOpenRouterImage` + +## Next Steps + +- [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/) — Workers AI documentation +- [Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/) — AI Gateway documentation +- [GitHub Repository](https://github.com/cloudflare/ai) — Source code and issues +- [Streaming Guide](/ai/chat/streaming) — Learn about streaming responses +- [Tools Guide](/ai/tools/tools) — Learn about tool calling diff --git a/mintlify/ai/community-adapters/decart.md b/mintlify/ai/community-adapters/decart.md new file mode 100644 index 000000000..f5db63f7d --- /dev/null +++ b/mintlify/ai/community-adapters/decart.md @@ -0,0 +1,262 @@ +--- +title: Decart +id: decart-adapter +order: 2 +description: "Generate images and videos with Decart's AI models in TanStack AI via the Decart community adapter." +keywords: + - tanstack ai + - decart + - image generation + - video generation + - community adapter +--- + +The Decart adapter provides access to Decart's image and video generation models. + +## Installation + +```bash +npm install @decartai/tanstack-ai-adapter +``` + +## Basic Usage + +```typescript +import { generateImage } from "@tanstack/ai"; +import { decartImage } from "@decartai/tanstack-ai-adapter"; + +const result = await generateImage({ + adapter: decartImage("lucy-pro-t2i"), + prompt: "A serene mountain landscape at sunset", +}); +``` + +## Basic Usage - Custom API Key + +```typescript +import { generateImage } from "@tanstack/ai"; +import { createDecartImage } from "@decartai/tanstack-ai-adapter"; + +const adapter = createDecartImage("lucy-pro-t2i", process.env.DECART_API_KEY!); + +const result = await generateImage({ + adapter, + prompt: "A serene mountain landscape at sunset", +}); +``` + +## Configuration + +```typescript +import { createDecartImage, type DecartImageConfig } from "@decartai/tanstack-ai-adapter"; + +const config: Omit = { + baseUrl: "https://api.decart.ai", // Optional, for custom endpoints +}; + +const adapter = createDecartImage("lucy-pro-t2i", process.env.DECART_API_KEY!, config); +``` + +## Image Generation + +Generate images with `lucy-pro-t2i`: + +```typescript +import { generateImage } from "@tanstack/ai"; +import { decartImage } from "@decartai/tanstack-ai-adapter"; + +const result = await generateImage({ + adapter: decartImage("lucy-pro-t2i"), + prompt: "A futuristic cityscape at night", +}); + +console.log(result.images[0]?.b64Json); +``` + +### Image Model Options + +```typescript +import { generateImage } from "@tanstack/ai"; +import { decartImage } from "@decartai/tanstack-ai-adapter"; + +const result = await generateImage({ + adapter: decartImage("lucy-pro-t2i"), + prompt: "A portrait of a robot artist", + modelOptions: { + resolution: "720p", + orientation: "portrait", + seed: 42, + }, +}); +``` + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `resolution` | `"720p"` | `"720p"` | Output resolution | +| `orientation` | `"portrait" \| "landscape"` | `"landscape"` | Image orientation | +| `seed` | `number` | - | Seed for reproducible generation | + +## Video Generation + +Video generation uses an async job/polling architecture. + +### Creating a Video Job + +```typescript +import { generateVideo } from "@tanstack/ai"; +import { decartVideo } from "@decartai/tanstack-ai-adapter"; + +const { jobId } = await generateVideo({ + adapter: decartVideo("lucy-pro-t2v"), + prompt: "A cat playing with a ball of yarn", +}); + +console.log("Job started:", jobId); +``` + +### Polling for Status + +```typescript +import { getVideoJobStatus } from "@tanstack/ai"; +import { decartVideo } from "@decartai/tanstack-ai-adapter"; + +const jobId = "example-job-id"; +const status = await getVideoJobStatus({ + adapter: decartVideo("lucy-pro-t2v"), + jobId, +}); + +console.log("Status:", status.status); // "pending" | "processing" | "completed" | "failed" + +if (status.status === "completed" && status.url) { + console.log("Video URL:", status.url); +} +``` + +### Complete Example with Polling + +```typescript +import { generateVideo, getVideoJobStatus } from "@tanstack/ai"; +import { decartVideo } from "@decartai/tanstack-ai-adapter"; + +async function createVideo(prompt: string) { + const adapter = decartVideo("lucy-pro-t2v"); + + // Create the job + const { jobId } = await generateVideo({ adapter, prompt }); + console.log("Job created:", jobId); + + // Poll for completion + let status = "pending"; + while (status !== "completed" && status !== "failed") { + await new Promise((resolve) => setTimeout(resolve, 5000)); + + const result = await getVideoJobStatus({ adapter, jobId }); + status = result.status; + console.log(`Status: ${status}`); + + if (result.status === "failed") { + throw new Error("Video generation failed"); + } + + if (result.status === "completed" && result.url) { + return result.url; + } + } +} + +const videoUrl = await createVideo("A drone shot over a tropical beach"); +``` + +### Video Model Options + +```typescript +import { generateVideo } from "@tanstack/ai"; +import { decartVideo } from "@decartai/tanstack-ai-adapter"; + +const { jobId } = await generateVideo({ + adapter: decartVideo("lucy-pro-t2v"), + prompt: "A timelapse of a blooming flower", + modelOptions: { + resolution: "720p", + orientation: "landscape", + seed: 42, + }, +}); +``` + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `resolution` | `"720p" \| "480p"` | `"720p"` | Output resolution | +| `orientation` | `"portrait" \| "landscape"` | `"landscape"` | Video orientation | +| `seed` | `number` | - | Seed for reproducible generation | + +## Environment Variables + +Set your API key in environment variables: + +```bash +DECART_API_KEY=your-api-key-here +``` + +## Getting an API Key + +1. Go to [Decart Platform](https://platform.decart.ai) +2. Create an account and generate an API key +3. Add it to your environment variables + +## API Reference + +### `decartImage(model, config?)` + +Creates a Decart image adapter using environment variables. + +**Parameters:** + +- `model` - Model name (`"lucy-pro-t2i"`) +- `config.baseUrl?` - Custom base URL (optional) + +**Returns:** A Decart image adapter instance. + +### `createDecartImage(model, apiKey, config?)` + +Creates a Decart image adapter with an explicit API key. + +**Parameters:** + +- `model` - Model name (`"lucy-pro-t2i"`) +- `apiKey` - Your Decart API key +- `config.baseUrl?` - Custom base URL (optional) + +**Returns:** A Decart image adapter instance. + +### `decartVideo(model, config?)` + +Creates a Decart video adapter using environment variables. + +**Parameters:** + +- `model` - Model name (`"lucy-pro-t2v"`) +- `config.baseUrl?` - Custom base URL (optional) + +**Returns:** A Decart video adapter instance. + +### `createDecartVideo(model, apiKey, config?)` + +Creates a Decart video adapter with an explicit API key. + +**Parameters:** + +- `model` - Model name (`"lucy-pro-t2v"`) +- `apiKey` - Your Decart API key +- `config.baseUrl?` - Custom base URL (optional) + +**Returns:** A Decart video adapter instance. + +## Next Steps + +- [Decart Platform](https://platform.decart.ai) - Visit Decart's platform to generate API keys +- [API Documentation](https://docs.platform.decart.ai) - View complete API reference +- [GitHub Repository](https://github.com/decartai/tanstack-ai) - Explore the adapter source code +- [Image Generation Guide](/ai/media/image-generation) - Learn about image generation +- [Video Generation Guide](/ai/media/video-generation) - Learn about video generation diff --git a/mintlify/ai/community-adapters/guide.md b/mintlify/ai/community-adapters/guide.md new file mode 100644 index 000000000..7690cb09a --- /dev/null +++ b/mintlify/ai/community-adapters/guide.md @@ -0,0 +1,217 @@ +--- +title: "Community Adapters Guide" +slug: /community-adapters/guide +order: 1 +description: "Build and publish a community adapter for TanStack AI — package conventions, implementing the adapter interface, and publishing to npm." +keywords: + - tanstack ai + - community adapters + - build adapter + - custom adapter + - provider integration + - adapter authoring + - contribute +--- + +# Community Adapters Guide + +This guide explains how to create and contribute community adapters for the TanStack AI ecosystem. + +Community adapters extend TanStack AI by integrating external services, APIs, or custom model logic. They are authored and maintained by the community and can be reused across projects. + +## What is a Community Adapter? + +A community adapter is a reusable module that connects TanStack AI to an external provider or system. + +Common use cases include: +- Integrating third-party AI model providers +- Implementing custom inference or routing logic +- Exposing provider-specific tools or capabilities +- Connecting to non-LLM AI services (e.g. images, embeddings, video) + +Community adapters are **not maintained by the core TanStack AI team**, and can be reused across different projects. + +## Creating a Community Adapter + +Follow the steps below to build a well-structured, type-safe adapter. + +### 1. Set up your project + +Start by reviewing the [existing internal adapter implementations in the TanStack AI GitHub repository](https://github.com/tanstack/ai/tree/main/packages). These define the expected structure, conventions, and integration patterns. + +For a complete, detailed reference, use the [OpenAI adapter](https://github.com/tanstack/ai/tree/main/packages/ai-openai), which is the most fully featured implementation. + +### 2. Define model metadata + +Model metadata describes each model’s capabilities and constraints and is used by TanStack AI for compatibility checks and feature selection. + +Your metadata should define, at a minimum: + +- Model name and identifier +- Supported input and output modalities +- Supported features (e.g. streaming, tools, structured output) +- Pricing or cost information (if available) +- Any provider-specific notes or limitations + +Refer to the [OpenAI adapter’s model metadata](https://github.com/TanStack/ai/blob/main/packages/ai-openai/src/model-meta.ts) for a concrete example. + +### 3. Define model capability arrays + +After defining metadata, group models by supported functionality using exported arrays. These arrays allow TanStack AI to automatically select compatible models for a given task. + +Example: +```typescript ignore +export const OPENAI_CHAT_MODELS = [ + // Frontier models + GPT5_2.name, + GPT5_2_PRO.name, + GPT5_2_CHAT.name, + GPT5_1.name, + GPT5_1_CODEX.name, + GPT5.name, + GPT5_MINI.name, + GPT5_NANO.name, + GPT5_PRO.name, + GPT5_CODEX.name, + // ...other models +] as const +export const OPENAI_IMAGE_MODELS = [ + GPT_IMAGE_1.name, + GPT_IMAGE_1_MINI.name, + DALL_E_3.name, + DALL_E_2.name, +] as const + +export const OPENAI_VIDEO_MODELS = [SORA2.name, SORA2_PRO.name] as const +``` +Each array should only include models that fully support the associated functionality. + +### 4. Define model provider options + +Each model exposes a different set of configurable options. These options must be typed per model name so that users only see valid configuration options. + +Example: +```typescript ignore +export type OpenAIChatModelProviderOptionsByName = { + [GPT5_2.name]: OpenAIBaseOptions & + OpenAIReasoningOptions & + OpenAIStructuredOutputOptions & + OpenAIToolsOptions & + OpenAIStreamingOptions & + OpenAIMetadataOptions + [GPT5_2_CHAT.name]: OpenAIBaseOptions & + OpenAIReasoningOptions & + OpenAIStructuredOutputOptions & + OpenAIToolsOptions & + OpenAIStreamingOptions & + OpenAIMetadataOptions + // ... repeat for each model +} + +``` +This ensures strict type safety and feature correctness at compile time. + +### 5. Define supported input modalities + +Models typically support different input modalities (e.g. text, images, audio). These must be defined per model to prevent invalid usage. + +Example: +```typescript ignore +export type OpenAIModelInputModalitiesByName = { + [GPT5_2.name]: typeof GPT5_2.supports.input + [GPT5_2_PRO.name]: typeof GPT5_2_PRO.supports.input + [GPT5_2_CHAT.name]: typeof GPT5_2_CHAT.supports.input + // ... repeat for each model +} +``` + +### 6. Define model option fragments + +Model options should be composed from reusable fragments rather than duplicated per model. + +A common pattern is: +- Base options shared by all models +- Feature fragments that are stitched together per model + +Example (based on [OpenAI models](https://github.com/TanStack/ai/blob/main/packages/ai-openai/src/text/text-provider-options.ts)): +```typescript +export interface OpenAIBaseOptions { + // base options that every chat model supports +} + +// Feature fragments that can be stitched per-model + +/** + * Reasoning options for models + */ +export interface OpenAIReasoningOptions { + //... +} + +/** + * Structured output options for models. + */ +export interface OpenAIStructuredOutputOptions { + //... +} +``` + + +Models can then opt into only the features they support: + +```typescript ignore +export type OpenAIChatModelProviderOptionsByName = { + [GPT5_2.name]: OpenAIBaseOptions & + OpenAIReasoningOptions & + OpenAIStructuredOutputOptions & + OpenAIToolsOptions & + OpenAIStreamingOptions & + OpenAIMetadataOptions +} +``` + +There is no single correct composition; this structure should reflect the capabilities of the provider you are integrating. + +### 7. Implement adapter logic + +Finally, implement the adapter’s runtime logic. + +This includes: +- Sending requests to the external service +- Handling streaming and non-streaming responses +- Mapping provider responses to TanStack AI types +- Enforcing model-specific options and constraints + +Adapters are implemented per capability, so only implement what your provider supports: + +- Text adapter +- Chat adapter +- Image adapter +- Embeddings adapter +- Video adapter + +Refer to the [OpenAI adapter](https://github.com/TanStack/ai/blob/main/packages/ai-openai/src/adapters/text.ts) for a complete, end-to-end implementation example. + +### 8. Publish and submit a PR + +Once your adapter is complete: +1. Publish it as an npm package +2. Open a PR to the [TanStack AI repository](https://github.com/TanStack/ai/pulls) +3. Add your adapter to the [Community Adapters list in the documentation](https://github.com/TanStack/ai/tree/main/docs/community-adapters) + +### 9. Sync documentation configuration + +After adding your adapter, run the `pnpm run sync-docs-config` in the root of the TanStack AI monorepo. This ensures your adapter appears correctly in the documentation navigation. Open a PR with the generated changes. + +### 10. Maintain your adapter + +As a community adapter author, you are responsible for ongoing maintenance. + +This includes: + +- Tracking upstream provider API changes +- Keeping compatibility with TanStack AI releases +- Addressing issues and feedback from users +- Updating documentation when features change + +If you add new features or breaking changes, open a follow-up PR to keep the docs in sync. \ No newline at end of file diff --git a/mintlify/ai/community-adapters/mynth.md b/mintlify/ai/community-adapters/mynth.md new file mode 100644 index 000000000..5686bc87c --- /dev/null +++ b/mintlify/ai/community-adapters/mynth.md @@ -0,0 +1,303 @@ +--- +title: Mynth +id: mynth-adapter +description: "Generate images with Mynth models like Flux, Recraft, Gemini, Qwen, Seedream, Wan, and Grok Imagine in TanStack AI via the Mynth community adapter." +keywords: + - tanstack ai + - mynth + - image generation + - flux + - recraft + - qwen + - seedream + - community adapter +--- + +# Mynth + +The Mynth adapter gives you access to Mynth image generation models through TanStack AI. It is a community adapter for `generateImage()` with typed model IDs, normalized image results, image-to-image support, and Mynth-specific request options through `modelOptions`. + +Mynth is image-only in this package. Reach for it when you want TanStack AI's image generation workflow with Mynth models such as Flux, Recraft, Gemini, Qwen, Seedream, Wan, and Grok Imagine. + +Quick note: Mynth is in public beta, so the model lineup and a few request options are still settling. The adapter tracks the Mynth SDK closely, and we welcome feedback on the API and integration experience. + +## Installation + +```sh +# bun +bun add @mynthio/tanstack-ai-adapter @tanstack/ai + +# pnpm +pnpm add @mynthio/tanstack-ai-adapter @tanstack/ai + +# npm +npm install @mynthio/tanstack-ai-adapter @tanstack/ai +``` + +The adapter targets `@tanstack/ai` 0.34 and newer. + +## Authentication + +Set your Mynth API key in the environment: + +```sh +MYNTH_API_KEY=mak_... +``` + +Keep `MYNTH_API_KEY` on the server only. Never expose it in browser code or public client environment variables, or it may end up in a client bundle. + +You can also pass `apiKey` directly in the adapter config. `baseUrl` is optional and useful for proxies, tests, or custom deployments. + +If you need a key, create one in the [Mynth API keys dashboard](https://mynth.io/dashboard/keys). + +## Quick Start + +```ts +import { generateImage } from "@tanstack/ai"; +import { mynthImage } from "@mynthio/tanstack-ai-adapter"; + +const result = await generateImage({ + adapter: mynthImage("black-forest-labs/flux.2-dev"), + prompt: "Editorial product photo of a ceramic mug on a linen tablecloth", + numberOfImages: 1, + size: "square", +}); + +console.log(result.id); +console.log(result.model); +console.log(result.images[0]?.url); +``` + +TanStack AI adapters are model-bound, so you choose the Mynth model when you create the adapter. + +## Reusable Provider + +Use `createMynthImage()` when you want to share config across multiple adapters: + +```ts +import { generateImage } from "@tanstack/ai"; +import { createMynthImage } from "@mynthio/tanstack-ai-adapter"; + +const mynth = createMynthImage({ + apiKey: process.env.MYNTH_API_KEY!, + baseUrl: "https://api.mynth.io", +}); + +const result = await generateImage({ + adapter: mynth("google/gemini-3.1-flash-image"), + prompt: "A playful paper-cut illustration of a city park in spring", +}); + +console.log(result.images[0]?.url); +``` + +You can still override shared config per adapter: + +```ts +import { createMynthImage } from "@mynthio/tanstack-ai-adapter"; + +const mynth = createMynthImage(); + +const adapter = mynth("auto", { + baseUrl: "https://proxy.example.com", +}); +``` + +## Model Options + +Use TanStack's top-level fields for common options such as `prompt`, `numberOfImages`, and shorthand `size`. Use `modelOptions` for Mynth-specific options: + +```ts +import { generateImage } from "@tanstack/ai"; +import { mynthImage } from "@mynthio/tanstack-ai-adapter"; + +const result = await generateImage({ + adapter: mynthImage("recraft/recraft-v4"), + prompt: "Modern poster design for a jazz festival", + numberOfImages: 2, + size: "portrait", + modelOptions: { + negativePrompt: "watermark, blurry text", + magicPrompt: true, + size: { + type: "aspect_ratio", + aspectRatio: "4:5", + scale: "4k", + }, + output: { + format: "png", + quality: 90, + }, + rating: true, + metadata: { + requestId: "req_123", + }, + }, +}); +``` + +Notes: + +- `modelOptions.negativePrompt` maps to Mynth's `negative_prompt`, and `modelOptions.magicPrompt` maps to `magic_prompt` +- `modelOptions.rating` configures content rating on the result. The older `contentRating` name still works as an alias +- `modelOptions.promptStructured` is still supported for compatibility and expands into `prompt`, `negative_prompt`, and `magic_prompt`. When set, its `positive` overrides the plain `prompt` +- `modelOptions.size` overrides the top-level `size`. Use it when you need structured Mynth size objects, including aspect ratios and an optional `scale: "4k"` +- Top-level `size` is for shorthand values such as `"auto"` and preset strings like `"square"` or `"landscape"` +- `modelOptions.destination` delivers the generation to a configured Mynth destination, overriding any adapter-level or env default + +## Image Inputs (image-to-image) + +Models that accept image inputs work with TanStack AI's content-part prompts, so you can mix instruction text with reference images for image-to-image, reference-guided, edit, and try-on flows. The adapter maps the image parts onto Mynth's `inputs`: + +```ts +import { generateImage } from "@tanstack/ai"; +import { mynthImage } from "@mynthio/tanstack-ai-adapter"; + +const result = await generateImage({ + adapter: mynthImage("black-forest-labs/flux.2-dev"), + prompt: [ + { type: "text", content: "Restyle this scene as a watercolor painting" }, + { + type: "image", + source: { type: "url", value: "https://example.com/photo.jpg" }, + }, + ], +}); +``` + +A few things worth knowing: + +- Only models in `MYNTH_IMAGE_INPUT_MODELS` accept image parts. Passing image parts to a text-only model is a compile-time error +- Both URL sources (`{ type: "url", value }`) and inline data sources (`{ type: "data", value, mimeType }`, encoded as a data URI) are supported +- A part's `metadata.role` maps to Mynth's input intent. TanStack's `"character"` maps directly, and the other generic roles fall back to Mynth's automatic detection +- For Mynth's finer-grained intents (`person`, `garment`, `pose`, `style`, `background`, `product`, `object`), pass `modelOptions.inputs` with an explicit `as` +- Image parts from the prompt and entries in `modelOptions.inputs` are combined, with prompt parts first + +## Available Models + +The adapter exports a runtime list and a type union for supported image models: + +```ts +import { MYNTH_IMAGE_MODELS, type MynthImageModel } from "@mynthio/tanstack-ai-adapter"; + +const defaultModel: MynthImageModel = "auto"; + +for (const model of MYNTH_IMAGE_MODELS) { + console.log(model); +} +``` + +This is handy for model selectors, validation, and keeping client and server code in sync. There is a matching `MYNTH_IMAGE_INPUT_MODELS` list (and `MynthImageInputModel` type) for the subset that accepts image inputs. + +Mynth supports model IDs across multiple providers, including `auto`, Flux, Recraft, Gemini, Qwen, Seedream, Imagine, Wan, Grok Imagine, and try-on models. The exported list is a fixed snapshot for type safety. For the live catalog with pricing, use the models endpoint below. + +### Models endpoint + +Mynth exposes a public catalog at `https://api.mynth.io/models`. It does not require an API key, and it carries pricing today with room for more metadata over time. This is the source of truth if you want to render a picker with live pricing rather than the static exported list. + +```ts +const response = await fetch("https://api.mynth.io/models"); +const { data } = await response.json(); + +for (const model of data) { + console.log(model.id, model.displayName, model.pricing); +} +``` + +Each entry looks roughly like this: + +```jsonc +{ + "id": "black-forest-labs/flux.2-dev", + "displayName": "FLUX.2 Dev", + "pricing": { + "perImage": { "base": "0.01", "4k": "0.04" }, + "perInput": "0.002", + }, +} +``` + +If you already use the Mynth SDK, the same data is available through `new Mynth().models.list()`. + +## Streaming Example + +This adapter also works with TanStack AI's streaming image workflow: + +```ts +import { generateImage, toServerSentEventsResponse } from "@tanstack/ai"; +import { mynthImage } from "@mynthio/tanstack-ai-adapter"; + +export async function POST(request: Request) { + const { prompt, model } = await request.json(); + + const stream = generateImage({ + adapter: mynthImage(model ?? "auto"), + prompt, + numberOfImages: 1, + stream: true, + }); + + return toServerSentEventsResponse(stream); +} +``` + +For a full example using `useGenerateImage()`, see the [TanStack Start + Mynth adapter demo](https://github.com/mynthio/oss/tree/main/examples/tanstack-start-ai-mynth-adapter). + +## Supported Capabilities + +- Image generation with `generateImage()` +- Image-to-image with content-part prompts on input-capable models +- Streaming image generation with `stream: true` +- Typed model IDs through `MYNTH_IMAGE_MODELS` and `MynthImageModel` +- Mynth-specific request options through `modelOptions` + +The adapter returns TanStack AI's normalized image result shape: + +- `id`: the Mynth task id +- `model`: the resolved model returned by Mynth, or the requested model as a fallback +- `images`: only successful images are included +- `images[*].revisedPrompt`: included when Mynth enhances the prompt + +## API Reference + +### `mynthImage(model, config?)` + +Creates a Mynth image adapter directly. + +- `model`: a `MynthImageModel` +- `config.apiKey?`: optional override for `MYNTH_API_KEY` +- `config.baseUrl?`: optional base URL override +- `config.destination?`: optional default destination for generated images + +Returns a `MynthImageAdapter` for use with `generateImage()`. + +### `createMynthImage(config?)` + +Creates a reusable provider factory that returns model-bound adapters. + +### `MYNTH_IMAGE_MODELS` + +Readonly array of supported Mynth image model IDs. + +### `MynthImageModel` + +Type union of supported Mynth image model IDs. + +### `MYNTH_IMAGE_INPUT_MODELS` + +Readonly array of model IDs that accept image inputs (image-to-image, try-on). + +### `MynthImageInputModel` + +Type union of the model IDs that accept image inputs. + +## Limitations + +- This package only provides an image adapter for `generateImage()` +- It does not provide chat or text-generation adapters + +## Next Steps + +- [Mynth SDK README](https://github.com/mynthio/oss/tree/main/packages/sdk) +- [TanStack Start + Mynth adapter demo](https://github.com/mynthio/oss/tree/main/examples/tanstack-start-ai-mynth-adapter) +- [Mynth](https://mynth.io) diff --git a/mintlify/ai/community-adapters/soniox.md b/mintlify/ai/community-adapters/soniox.md new file mode 100644 index 000000000..eab0b8c30 --- /dev/null +++ b/mintlify/ai/community-adapters/soniox.md @@ -0,0 +1,257 @@ +--- +title: Soniox +id: soniox-adapter +order: 3 +description: "Transcribe audio with Soniox speech-to-text models in TanStack AI via the Soniox community adapter." +keywords: + - tanstack ai + - soniox + - transcription + - speech-to-text + - asr + - community adapter +--- + +The Soniox adapter provides access to Soniox transcription models. + +## Installation + +```bash +npm install @soniox/tanstack-ai-adapter +``` + +## Authentication + +Set `SONIOX_API_KEY` in your environment or pass `apiKey` when creating the adapter. Get your API key from the [Soniox Console](https://console.soniox.com). + +## Basic Usage + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { sonioxTranscription } from "@soniox/tanstack-ai-adapter"; +import { audioFile } from "./audio"; + +const result = await generateTranscription({ + adapter: sonioxTranscription("stt-async-v3"), + audio: audioFile, + modelOptions: { + enableLanguageIdentification: true, + enableSpeakerDiarization: true, + }, +}); + +console.log(result.text); +console.log(result.segments); +``` + +## Basic Usage - Custom API Key + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { createSonioxTranscription } from "@soniox/tanstack-ai-adapter"; +import { audioFile } from "./audio"; + +const adapter = createSonioxTranscription("stt-async-v3", process.env.SONIOX_API_KEY!); + +const result = await generateTranscription({ + adapter, + audio: audioFile, +}); +``` + +## Adapter Configuration + +Use `createSonioxTranscription` to customize the adapter instance: + +```typescript +import { createSonioxTranscription } from "@soniox/tanstack-ai-adapter"; + +const adapter = createSonioxTranscription("stt-async-v3", process.env.SONIOX_API_KEY!, { + baseUrl: "https://api.soniox.com", + pollingIntervalMs: 1000, + timeout: 180000, +}); +``` + +Options: + +- `apiKey` - Override `SONIOX_API_KEY` (required when using `createSonioxTranscription`) +- `baseUrl` - Custom API base URL. Default is `https://api.soniox.com` +- `headers` - Additional request headers +- `timeout` - Transcription timeout in milliseconds (default: 180000) +- `pollingIntervalMs` - Transcription polling interval in milliseconds (default: 1000) + +See the [Soniox regional endpoints](https://soniox.com/docs/stt/data-residency#regional-endpoints) if you need data residency. + +## Transcription Options + +Per-request options are passed via `modelOptions`: + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { sonioxTranscription } from "@soniox/tanstack-ai-adapter"; +import { audio } from "./audio"; + +const result = await generateTranscription({ + adapter: sonioxTranscription("stt-async-v3"), + audio, + modelOptions: { + languageHints: ["en", "es"], + enableLanguageIdentification: true, + enableSpeakerDiarization: true, + context: { + terms: ["Soniox", "TanStack"], + }, + }, +}); +``` + +Available options: + +- `languageHints` - Array of ISO language codes to bias recognition +- `languageHintsStrict` - When true, rely more heavily on language hints (not supported by all models) +- `enableLanguageIdentification` - Automatically detect spoken language +- `enableSpeakerDiarization` - Identify and separate different speakers +- `context` - Additional context to improve accuracy +- `clientReferenceId` - Optional client-defined reference ID +- `webhookUrl` - Webhook URL for completion notifications +- `webhookAuthHeaderName` - Webhook authentication header name +- `webhookAuthHeaderValue` - Webhook authentication header value +- `translation` - Translation configuration + +Check the [Soniox API reference](https://soniox.com/docs/stt/api-reference/transcriptions/create_transcription) for more details. + +## Language Hints + +Soniox automatically detects and transcribes speech in [60+ languages](https://soniox.com/docs/stt/concepts/supported-languages). When you know which languages are likely to appear in your audio, provide `languageHints` to improve accuracy by biasing recognition toward those languages. + +Language hints do not restrict recognition. If you pass the TanStack `language` option, this adapter merges it into `languageHints`. + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { sonioxTranscription } from "@soniox/tanstack-ai-adapter"; +import { audio } from "./audio"; + +const result = await generateTranscription({ + adapter: sonioxTranscription("stt-async-v3"), + audio, + modelOptions: { + languageHints: ["en", "es"], + }, +}); +``` + +For more details, see the [Soniox language hints documentation](https://soniox.com/docs/stt/concepts/language-hints). + +## Context + +Provide custom context to improve transcription and translation accuracy. Context helps the model understand your domain, recognize important terms, and apply custom vocabulary. + +The `context` object supports four optional sections: + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { sonioxTranscription } from "@soniox/tanstack-ai-adapter"; +import { audio } from "./audio"; + +const result = await generateTranscription({ + adapter: sonioxTranscription("stt-async-v3"), + audio, + modelOptions: { + context: { + general: [ + { key: "domain", value: "Healthcare" }, + { key: "topic", value: "Diabetes management consultation" }, + { key: "doctor", value: "Dr. Martha Smith" }, + ], + text: "The patient has a history of...", + terms: ["Celebrex", "Zyrtec", "Xanax"], + translationTerms: [ + { source: "Mr. Smith", target: "Sr. Smith" }, + { source: "MRI", target: "RM" }, + ], + }, + }, +}); +``` + +For more details, see the [Soniox context documentation](https://soniox.com/docs/stt/concepts/context). + +## Translation + +Configure translation for your transcriptions: + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { sonioxTranscription } from "@soniox/tanstack-ai-adapter"; +import { audio } from "./audio"; + +const result = await generateTranscription({ + adapter: sonioxTranscription("stt-async-v3"), + audio, + modelOptions: { + translation: { + type: "one_way", + targetLanguage: "es", + }, + }, +}); +``` + +For two-way translation: + +```typescript +import { generateTranscription } from "@tanstack/ai"; +import { sonioxTranscription } from "@soniox/tanstack-ai-adapter"; +import { audio } from "./audio"; + +const result = await generateTranscription({ + adapter: sonioxTranscription("stt-async-v3"), + audio, + modelOptions: { + translation: { + type: "two_way", + languageA: "en", + languageB: "es", + }, + }, +}); +``` + +When using translation, the API returns both transcription tokens and translation tokens. The `segments` array always includes only transcription tokens. To access translation tokens, use `providerMetadata` and filter by `translation_status === "translation"`. + +### Accessing Raw Tokens + +When using translation or working with multilingual audio, you may need access to raw tokens with per-token language information and translation status. The adapter attaches a non-standard `providerMetadata` field at runtime: + +```typescript ignore +// ignore: providerMetadata is a non-standard runtime extension not on TranscriptionResult; +// accessing it requires a cast that cannot be expressed without `as`. +const result = await generateTranscription({ + adapter: sonioxTranscription("stt-async-v3"), + audio, + modelOptions: { + translation: { type: "one_way", targetLanguage: "es" }, + }, +}); + +const rawTokens = (result as any).providerMetadata?.soniox?.tokens; + +if (rawTokens) { + rawTokens.forEach((token) => { + // token.text - token text + // token.start_ms - start time in milliseconds + // token.end_ms - end time in milliseconds + // token.language - detected language for this token + // token.translation_status - translation status (if translation enabled) + // token.speaker - speaker identifier + // token.confidence - confidence score + }); +} +``` + +## Next Steps + +- [Soniox Console](https://console.soniox.com) - Manage API keys and projects +- [Soniox API docs](https://soniox.com/docs) - Complete Soniox API reference +- [Transcription Guide](/ai/media/transcription) - Learn about transcription in TanStack AI diff --git a/mintlify/ai/comparison/vercel-ai-sdk.md b/mintlify/ai/comparison/vercel-ai-sdk.md new file mode 100644 index 000000000..7a04e5100 --- /dev/null +++ b/mintlify/ai/comparison/vercel-ai-sdk.md @@ -0,0 +1,755 @@ +--- +title: TanStack AI vs Vercel AI SDK +id: vercel-ai-sdk +order: 1 +description: "How TanStack AI compares to the Vercel AI SDK — feature matrix, philosophy, type safety, tool calling, streaming, and framework support." +keywords: + - tanstack ai + - vercel ai sdk + - comparison + - ai sdk + - alternatives + - typescript ai sdk + - tool calling + - llm +--- + +Both TanStack AI and Vercel AI SDK are open-source TypeScript toolkits for building AI-powered applications. They share common ground - streaming chat, tool calling, multi-provider support, and deploy-anywhere flexibility - but they approach the problem from fundamentally different directions. + +TanStack AI treats AI as a **library composition problem**. Every piece - adapters, tools, agent loops, transport, UI - is a composable building block. You import what you need, compose it how you want, and ship it wherever you want. No platform layer, no gateway abstraction, no implicit associations. + +Vercel AI SDK treats AI as a **full-stack platform problem**. It provides a broad surface area of primitives with optional platform integration for gateway routing, observability, and deployment optimization. + +This article compares the two SDKs from TanStack AI's perspective, with honest acknowledgment of where each excels. + +## Feature Comparison + +Versions referenced below: TanStack AI as of this writing; Vercel AI SDK `ai@6.x` (v6.0.0 shipped December 2025; v7 is in pre-release at the time of writing). + +| Feature | TanStack AI | Vercel AI SDK | +|---------|------------|---------------| +| License | MIT | Apache 2.0 | +| Hosting | Works anywhere | Works anywhere | +| Providers | 9 official + community; OpenRouter routes to 100s of models and the `openaiCompatible` adapter connects to any OpenAI-compatible endpoint | ~38 first-party provider packages (plus community providers); 100+ models via AI Gateway | +| Framework Hooks | React, Solid, Svelte, Vue, Preact (+ React Native) | React, Vue, Svelte, Angular (Solid is community-maintained) | +| Generation UI Hooks | One hook per activity: chat, structured output, image, audio, speech, transcription, summarize, video, realtime | `useChat`, `useCompletion`, `useObject` | +| Wire Protocol | Native AG-UI events end to end | Proprietary UI Message Stream; AG-UI via external translation layer | +| Streaming | Built-in with configurable chunk strategies | Built-in with progressive delivery | +| Tool Calling | Isomorphic `.server()` / `.client()` system | `tool()` objects; client execution via `onToolCall` | +| Agent Loop Control | Composable strategy functions `(state) => boolean` | `stopWhen` conditions + `Agent` (`ToolLoopAgent`) class | +| Tool Approval | Per-tool `needsApproval` with batched approval flow | Per-tool `needsApproval` (human-in-the-loop) | +| Type Safety | Per-model type narrowing | Per-provider types | +| Tree-Shaking | Separate adapter per activity (text, image, speech, etc.) | Monolithic provider packages | +| Lazy Tool Discovery | Built-in - works across every provider | Anthropic-only tool search with `deferLoading` (provider-hosted) | +| Connection Adapters | SSE, HTTP stream, XHR (SSE/stream), RPC, direct async iterables, `fetcher`, custom | SSE-based data stream protocol (`ChatTransport`) | +| Middleware | App-level lifecycle hooks (config, iterations, chunks, tool calls, usage, errors) | Model-level wrapping via `wrapLanguageModel()` | +| Extend Adapter | Add custom/fine-tuned models with per-model type narrowing | `customProvider()` / `createProviderRegistry()` (string model ids) | +| Structured Outputs | Typed `StructuredOutputPart`, streamed alongside tools and preserved per turn in message history | `generateObject()` / `streamObject()` / `Output` API (per-call; no structured-output message part) | +| Image Generation | Stable API with per-model type safety (OpenAI, Gemini, Grok, OpenRouter, fal.ai) | `generateImage()` (stable) | +| Video Generation | Stable API with async job lifecycle (OpenAI, fal.ai) | `experimental_generateVideo()` | +| Text-to-Speech | Stable API, 6 output formats, speed control (OpenAI, Gemini, Grok, ElevenLabs, fal.ai) | `generateSpeech()` (experimental) | +| Transcription | Stable API with word timestamps and diarization (OpenAI, Grok, ElevenLabs, fal.ai) | `transcribe()` (experimental) | +| Audio / Music Generation | `generateAudio()` for music & sound effects (Gemini, ElevenLabs, fal.ai) | - | +| Summarization | Dedicated `summarize()` with streaming and style options | - | +| Code Execution | Node.js, Cloudflare Workers, QuickJS sandboxes you run yourself | Provider-hosted code-execution tools (Anthropic, xAI, OpenAI) | +| Code Mode Skills | LLM-writable persistent skill library | - | +| Coding Agent Sandboxes | First-party Grok Build, Claude Code, Codex, OpenCode harnesses **+ any ACP agent** via `acpCompatible`; runs on local-process, Docker, Daytona, Vercel, Sprites, or Cloudflare | `HarnessAgent` (experimental) — Claude Code, Codex, Pi, OpenCode, Deep Agents; centered on Vercel Sandbox | +| Realtime Voice | OpenAI, Grok, and ElevenLabs with VAD modes and tool support | - | +| DevTools | Isomorphic in-app panel via TanStack DevTools (all frameworks, media previews) | `devToolsMiddleware` + local inspector (server-side, dev-only) | +| Debug Logging | One flag, per-category toggles, pluggable logger | Warning logs + experimental telemetry hooks | +| MCP Client | Standalone host-side client (`@tanstack/ai-mcp`) + provider-routed `mcpTool()` | Built-in (`@ai-sdk/mcp`, stable) | +| MCP Apps (Interactive Widgets) | `ui://` widgets via `@mcp-ui/client`; React + Preact + framework-agnostic bridge; multi-server routing, pluggable session store, link-scheme hardening | `experimental_MCPAppRenderer` (React only); model-vs-app tool split; iframe sandbox + allowlist | +| Platform Association | None - pure library | Optional Vercel integration | + +## Where TanStack AI Excels + +### Per-Model Type Safety + +When you select a provider and model, TypeScript narrows the exact options, capabilities, and input modalities available for that specific model - not a union of everything the provider supports. + +Each provider adapter contains a comprehensive `model-meta.ts` that maps every model to its capabilities: supported input modalities, context windows, and provider-specific options. When you write `openaiText('gpt-5.5')`, the type system knows exactly what that model can do. + +```ts +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +// TypeScript knows gpt-5.5 supports text + image input +const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages: [{ + role: 'user', + content: [ + { type: 'text', content: 'What is in this image?' }, + { type: 'image', source: { type: 'url', value: 'https://example.com/photo.jpg' } }, + ], + }], +}) +``` + +If you pass an image content part to a text-only model, TypeScript catches it at compile time. + +### Tree-Shakeable Adapters + +Every AI activity - chat, summarization, image generation, speech, transcription, video - is a separate import. Every provider exposes separate adapter functions per activity. If your app only uses chat, image generation code never enters your bundle. + +```ts ignore +// Only chat code is bundled - nothing else +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +// vs. importing activities you actually need +import { chat, generateImage } from '@tanstack/ai' +import { openaiText, openaiImage } from '@tanstack/ai-openai' +``` + +This is architectural, not incidental. Each adapter implements a specific interface (`TextAdapter`, `ImageAdapter`, `TTSAdapter`, etc.) and lives in its own module. Modern bundlers eliminate everything you don't import. + +### Isomorphic Tools + +`toolDefinition()` creates a shared contract - name, description, input schema, output schema - that can be implemented for different runtimes. `.server()` adds a server-side implementation with access to databases and APIs. `.client()` adds a client-side implementation that runs in the browser. + +```ts +import { toolDefinition } from '@tanstack/ai' +import { z } from 'zod' +import { db } from './db' + +// Define once - shared validation contract +const addToCartDef = toolDefinition({ + name: 'addToCart', + description: 'Add an item to the shopping cart', + inputSchema: z.object({ + itemId: z.string(), + quantity: z.number(), + }), + outputSchema: z.object({ + success: z.boolean(), + cartId: z.string(), + }), +}) + +// Server implementation - database access +const addToCartServer = addToCartDef.server(async ({ itemId, quantity }) => { + const cart = await db.carts.addItem(itemId, quantity) + return { success: true, cartId: cart.id } +}) + +// Client implementation - runs in the browser +const addToCartClient = addToCartDef.client(async ({ itemId, quantity }) => { + const res = await fetch(`/api/cart`, { + method: 'POST', + body: JSON.stringify({ itemId, quantity }), + }) + return res.json() +}) +``` + +The same schema validates inputs and outputs on both sides. The type system tracks whether a tool is a `ServerTool` or `ClientTool` at compile time. + +Vercel AI SDK defines tools with a `tool()` helper and does support client-side execution - a tool with no `execute` function is handled in the browser via the UI hook's `onToolCall` callback, with the result returned through `addToolOutput` (renamed from `addToolResult` in v6). What it doesn't have is a single shared contract that produces separate `.server()` and `.client()` implementations: server and client tool code are declared independently rather than derived from one definition. + +### Composable Agent Loop Strategies + +TanStack AI provides agent loop control as composable pure functions. Each strategy is `(state) => boolean` - return `true` to continue, `false` to stop. + +```ts +import { chat, maxIterations, untilFinishReason, combineStrategies } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { tools } from './tools' + +const messages = [{ role: 'user' as const, content: 'Help me plan a trip.' }] + +const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages, + tools, + agentLoopStrategy: combineStrategies([ + maxIterations(10), + untilFinishReason(['stop', 'length']), + ]), +}) +``` + +`combineStrategies` composes them with AND logic - all strategies must agree to continue. You can add custom strategies alongside built-in ones: + +```ts +import { maxIterations, untilFinishReason, combineStrategies } from '@tanstack/ai' +import { estimatedCost, budget } from './cost' + +combineStrategies([ + maxIterations(10), + untilFinishReason(['stop']), + // Custom: stop if budget exceeded + ({ iterationCount }) => estimatedCost(iterationCount) < budget, +]) +``` + +Vercel AI SDK (v5+) controls agent loops via `stopWhen`, which accepts composable stopping conditions like `stepCountIs(n)` and `hasToolCall(name)` (the default is `stepCountIs(20)`), and v6 adds a dedicated `Agent` abstraction (the `ToolLoopAgent` class) that bundles model, tools, instructions, and loop settings into a reusable object. The remaining difference is in the composition model: TanStack AI's strategies are arbitrary `(state) => boolean` predicates you write inline and combine with `combineStrategies`, so a stopping condition can encode any business logic (token budgets, cost ceilings, custom state checks) without waiting for a built-in condition to exist. Vercel's `stopWhen` also accepts custom functions, so the gap here is smaller than it once was. + +### Lazy Tool Discovery + +When your application has dozens of tools, sending all their schemas to the LLM on every request wastes tokens. TanStack AI solves this with lazy tool discovery. + +Mark tools as `lazy: true` and they won't be sent to the LLM initially. Instead, a synthetic discovery tool is injected that lets the LLM request tool schemas on demand: + +```ts +import { toolDefinition } from '@tanstack/ai' +import { z } from 'zod' + +const searchProducts = toolDefinition({ + name: 'searchProducts', + description: 'Search the product catalog', + lazy: true, // Not sent to LLM initially + inputSchema: z.object({ query: z.string() }), + outputSchema: z.array(z.object({ id: z.string(), name: z.string() })), +}) +``` + +The LLM sees a lightweight `__lazy__tool__discovery__` tool listing available tool names. When it needs one, it calls the discovery tool to get the full schema, then uses the real tool. For applications with large tool inventories, this significantly reduces per-request token costs. + +Vercel AI SDK 6 added a provider-specific analogue for Anthropic: the tool search provider tool (`toolSearchBm25` / `toolSearchRegex`) with per-tool `deferLoading`, where deferred tools are excluded from the initial prompt and discovered on demand. It's Anthropic-only and runs provider-side; TanStack AI's lazy discovery works across every provider and runs in your own agent loop. + +### Model Context Protocol (MCP) + +TanStack AI connects to MCP servers two ways, and you can mix them in a single `chat()` run: + +- **Host-side client** (`@tanstack/ai-mcp`) - your server connects directly to any MCP server. `createMCPClient` (single server) and `createMCPClients` (multi-server pool) discover and execute tools, read resources, and fetch prompts over Streamable HTTP, SSE, or stdio transports, with OAuth 2.1 (`authProvider`) and static-token auth. +- **Provider-routed** (`mcpTool()`) - the *provider* connects to the MCP server on your behalf (OpenAI Responses API, Anthropic), so no MCP traffic flows through your server at all. + +The host-side client goes beyond basic discovery: + +- **Managed lifecycle** - hand clients to `chat()` via the `mcp` option and it discovers tools and closes connections when the run ends - no `try/finally` per route. +- **Multi-server pools** - `createMCPClients` connects to many servers in parallel, auto-prefixing each server's tools to prevent name collisions. +- **Three modes of type safety** - untyped auto-discovery, `toolDefinition()`-typed allowlists with Zod validation, or fully generated per-server types via the `tanstack-ai-mcp` CLI. +- **Lazy discovery** - `tools({ lazy: true })` defers sending tool schemas to the LLM, plugging into TanStack AI's lazy tool discovery to cut token usage on tool-heavy servers. +- **Resources & prompts** - inject MCP resources and prompts into a run with `mcpResourceToContentPart` and `mcpPromptToMessages`. + +```ts +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { createMCPClient } from '@tanstack/ai-mcp' + +const messages = [{ role: 'user' as const, content: 'What tools are available?' }] + +const mcp = await createMCPClient({ + transport: { type: 'http', url: 'https://my-mcp-server.example.com/mcp' }, +}) + +// chat() discovers the tools and closes the client when the run ends +const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages, + mcp: { clients: [mcp] }, +}) +``` + +Vercel AI SDK's `@ai-sdk/mcp` (`createMCPClient`) is a stable host-side client with HTTP/SSE transports, OAuth, resource reading, and prompt templates. TanStack AI's `@tanstack/ai-mcp` matches that surface and adds generated end-to-end types, multi-server pools, lazy discovery, a managed `chat()` lifecycle, and the provider-routed `mcpTool()` alternative. + +### MCP Apps (Interactive Widgets) + +Both SDKs implement [MCP Apps](https://modelcontextprotocol.io) — the ratified MCP extension (standardized 2026-01-26) where a server returns a `ui://` resource so a tool result renders as an interactive widget in a sandboxed iframe instead of raw JSON. Both keep the widget HTML out of model input and render it in a sandboxed iframe with a tool allowlist and safe link handling. TanStack AI's implementation is more built out along three axes: + +- **More than one framework.** Widgets render via `@tanstack/ai-react/mcp-apps` and `@tanstack/ai-preact/mcp-apps`, and the bridge that routes widget actions (`createMcpAppBridge`) lives in the framework-agnostic `@tanstack/ai-client`, so a new framework only needs a thin renderer. Vercel's `experimental_MCPAppRenderer` and its bridge live in `@ai-sdk/react` — React only. +- **Multi-server routing.** Each `UIResourcePart` carries a `serverId` (the pool prefix from `createMCPClients`), and interactive calls route back to the exact server that produced the widget — automatically when you run a multi-server pool. The call handler also enforces an unconditional same-server exposure check (`toolName` must be a tool that server actually exposes) with an optional `allowTool` restriction AND-ed on top. +- **Session persistence & serverless-safety.** The call handler reconnects per call from a transport descriptor (stateless, serverless-safe by default), and stateful transports opt into a pluggable `McpSessionStore` — an `inMemoryMcpSessionStore` ships, and SQL/KV backends drop in behind the same interface. + +```tsx +import { useChat, useMcpAppBridge } from '@tanstack/ai-react' +import { fetchServerSentEvents } from '@tanstack/ai-client' +import { MCPAppResource } from '@tanstack/ai-react/mcp-apps' + +export function Chat() { + const { messages, sendMessage } = useChat({ + connection: fetchServerSentEvents('/api/chat'), + }) + + // Routes widget tool-calls to /api/mcp-apps/call by serverId; only http/https/mailto links pass through. + const bridge = useMcpAppBridge({ + threadId: 'weather-chat', + callEndpoint: '/api/mcp-apps/call', + chat: { sendMessage: async (content) => void sendMessage({ content }) }, + onLink: (url) => window.open(url, '_blank', 'noopener'), + }) + + return ( + <> + {messages.map((m) => + m.parts.map((part, i) => + part.type === 'ui-resource' ? ( + + ) : null, + ), + )} + + ) +} +``` + +Vercel AI SDK 7 covers the core flow and adds one thing TanStack AI doesn't: `splitMCPAppTools`, which separates *model-visible* tools from *app-only* tools the widget can call but the model never sees. Both implementations are new — Vercel marks its renderer `experimental_`, and TanStack AI's writeback of widget tool-calls into chat history is still out of scope. See the [MCP Apps guide](/ai/mcp/apps) for the full API. + +### Headless Client Architecture + +`ChatClient` is a framework-agnostic class that manages the entire chat lifecycle - streaming, message state, tool execution, approval flows, and connection management. Every framework integration wraps this single client: + +- `@tanstack/ai-react` - `useChat` hook wraps `ChatClient` +- `@tanstack/ai-solid` - `useChat` hook wraps `ChatClient` +- `@tanstack/ai-vue` - `useChat` composable wraps `ChatClient` +- `@tanstack/ai-svelte` - `createChat` wraps `ChatClient` (Svelte 5 runes) +- `@tanstack/ai-preact` - `useChat` hook wraps `ChatClient` + +No framework-specific logic in the core. If a new framework emerges, it only needs a thin reactive wrapper. + +`ChatClient` also accepts a persistence adapter (`ChatClientPersistence`) for saving and restoring conversations client-side, and a typed runtime `context` that flows through to tools and middleware. + +### Connection Adapters + +TanStack AI ships six built-in connection adapters plus a custom adapter interface: + +```ts +import { + fetchServerSentEvents, + fetchHttpStream, + xhrServerSentEvents, + xhrHttpStream, + stream, + rpcStream, +} from '@tanstack/ai-client' +import { chatOnServer } from './server' +import { api } from './api' + +// Server-Sent Events (standard) +fetchServerSentEvents('/api/chat') + +// Raw HTTP streaming (newline-delimited JSON) +fetchHttpStream('/api/chat') + +// XHR-based SSE / HTTP streaming (React Native / Expo, where fetch streaming is unavailable) +xhrServerSentEvents('/api/chat') +xhrHttpStream('/api/chat') + +// Direct async iterables (TanStack Start server functions) +stream((messages) => chatOnServer({ messages })) + +// RPC-based transport +rpcStream((messages, data) => api.streamResponse(messages, data)) + +// Or implement your own ConnectionAdapter +``` + +Each adapter accepts static or dynamic (function-based) URLs and options. There's also a lighter-weight `fetcher` option on `ChatClient` / `useChat` for wiring a server function directly without a full adapter. Swap transport without changing application code. Vercel AI SDK centers on its SSE-based data stream protocol and a `ChatTransport` interface for extensibility, but doesn't ship the same breadth of built-in adapters - notably the XHR variants for React Native. + +### Extend Adapter + +When you use fine-tuned models, OpenAI-compatible proxies, or custom model endpoints, `extendAdapter()` lets you add them to any provider adapter with full type safety: + +```ts +import { extendAdapter, createModel } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +const customModels = [ + createModel('my-fine-tuned-gpt4', ['text', 'image']), + createModel('company-internal-llm', ['text']), +] as const + +const myOpenai = extendAdapter(openaiText, customModels) + +// Full autocomplete - original models + custom models +const adapter = myOpenai('my-fine-tuned-gpt4') +``` + +Your custom models appear in autocomplete alongside official ones. Vercel AI SDK covers the registration half of this with the now-stable `customProvider()` (custom and aliased model ids, settings overrides) and `createProviderRegistry()`; the difference is type-safety depth - registry model ids are plain strings, while `extendAdapter()` gives custom models the same literal-type narrowing and per-model option gating as official ones. + +### Middleware + +TanStack AI's middleware system hooks into every stage of the `chat()` lifecycle: configuration, streaming, tool execution, usage tracking, and completion. Each middleware is a plain object with named hooks that fire at specific phases. + +```ts +import { chat, EventType, type ChatMiddleware, type StreamChunk } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +const messages = [{ role: 'user' as const, content: 'Hello' }] + +const logger: ChatMiddleware = { + name: 'logger', + onStart: (ctx) => { + console.log(`[${ctx.requestId}] Chat started`) + }, + onChunk: (ctx, chunk) => { + // Transform, expand, or drop chunks + if ('delta' in chunk && 'messageId' in chunk) { + return { ...chunk, delta: chunk.delta!.replace(/\b\d{3}-\d{2}-\d{4}\b/g, '[REDACTED]') } + } + }, + onBeforeToolCall: (ctx, hookCtx) => { + // Intercept tool calls: transform args, skip, or abort + if (hookCtx.toolName === 'deleteDatabase') { + return { type: 'abort', reason: 'Dangerous operation blocked' } + } + }, + onAfterToolCall: (ctx, info) => { + console.log(`${info.toolName}: ${info.ok ? 'success' : 'failed'} in ${info.duration}ms`) + }, + onFinish: (ctx, info) => { + console.log(`Done in ${info.duration}ms, ${info.usage?.totalTokens} tokens`) + }, +} + +const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages, + middleware: [logger], +}) +``` + +The available hooks cover the full lifecycle: + +| Hook | Purpose | +|------|---------| +| `onConfig` | Transform messages, tools, temperature, system prompts per iteration | +| `onStructuredOutputConfig` | Transform the structured-output schema/config before the call | +| `onStart` | Setup tasks (timers, logging) | +| `onIteration` | Observe each agent-loop iteration | +| `onChunk` | Transform, expand, or drop individual stream chunks | +| `onBeforeToolCall` | Intercept tool calls: transform args, skip execution, or abort the run | +| `onAfterToolCall` | Observe tool results, timing, and errors | +| `onToolPhaseComplete` | Observe the full batch of tool results for an iteration (e.g. aggregate approval state) | +| `onUsage` | Track token usage per iteration | +| `onFinish` / `onAbort` / `onError` | Terminal hooks (exactly one fires per run) | + +Middleware compose naturally. `onConfig` pipes through each middleware in order. `onChunk` pipes chunks through each middleware (if one drops a chunk, later middleware never see it). `onBeforeToolCall` uses first-win semantics: the first middleware that returns a decision short-circuits the rest. + +TanStack AI ships several built-in middleware. `toolCacheMiddleware` and `contentGuardMiddleware` come from the `@tanstack/ai/middlewares` subpath, and `otelMiddleware` from `@tanstack/ai/middlewares/otel` (kept on its own subpath so `@opentelemetry/api` stays an optional peer). `toolCacheMiddleware` caches tool results by name and arguments with configurable TTL, LRU eviction, and pluggable storage backends (Redis, localStorage, etc.). + +```ts +import { toolCacheMiddleware, contentGuardMiddleware } from '@tanstack/ai/middlewares' +import { otelMiddleware } from '@tanstack/ai/middlewares/otel' +``` + +Vercel AI SDK takes a different approach: `wrapLanguageModel()` wraps a model instance with middleware that can intercept and transform calls (and v6 adds `wrapEmbeddingModel()`). It ships several built-in middleware (`extractReasoningMiddleware`, `simulateStreamingMiddleware`, `defaultSettingsMiddleware`, and the new `devToolsMiddleware`), but these all operate at the model level rather than the application level. v6 also exposes per-call options that cover slices of this surface: `experimental_transform` for stream transforms, `experimental_onToolCallStart` / `experimental_onToolCallFinish` callbacks, `prepareStep` for per-step config changes, and `experimental_repairToolCall`. What it doesn't have is a unified middleware system at the application level - named, reusable middleware objects whose hooks span the whole lifecycle, compose in order, and can short-circuit tool calls with first-win semantics. + +### No Platform Association + +TanStack AI is a pure library. There's no optional platform layer, no gateway abstraction, no hosting-specific features, and no deployment-specific optimizations. Your AI code carries no implicit association with any deployment platform. + +This isn't just philosophical - it means no accidental dependencies on platform-specific features, no gateway abstractions that subtly encourage vendor adoption, and no marketing surface embedded in your technical stack. + +### Code Execution Sandboxes + +TanStack AI provides three isolate drivers for safe code execution in AI workflows: + +- **`@tanstack/ai-isolate-node`** - Node.js sandbox via `isolated-vm` +- **`@tanstack/ai-isolate-cloudflare`** - Cloudflare Workers sandbox +- **`@tanstack/ai-isolate-quickjs`** - QuickJS lightweight sandbox + +All three implement the same `IsolateDriver` interface, so you can swap execution environments without changing application code. This powers TanStack AI's code mode - where the LLM writes and executes code as part of the agent loop. A companion `@tanstack/ai-code-mode-skills` package lets you give code mode a persistent, reusable library of runtime skills. Skills are LLM-writable: the model can save working TypeScript snippets, list and reuse them across sessions, with trust strategies controlling what gets promoted to a first-class tool. The closest AI SDK analogues - Anthropic's provider-hosted code execution and developer-uploaded skills, or pre-authored file skills loaded into a sandbox - are provider-specific and static; none give the model a persistent, provider-agnostic skill library it builds itself. + +Vercel AI SDK does not provide built-in code execution sandboxes (though some providers expose their own server-side code execution as provider-executed tools). + +### Coding Agent Sandboxes + +Separately from the JS isolates above, TanStack AI can put a full **coding-agent CLI** — Claude Code, Codex, Grok Build, OpenCode, or any ACP-compliant agent — inside an isolated sandbox with a real filesystem, shell, and a cloned repo, and stream its work back through `chat()` like any other run. A sandboxed run composes three swappable pieces: a **provider** (where it runs), a **workspace** (what the agent sees), and a **harness adapter** (which agent runs). The sandbox is a `chat()` middleware, so the agent's edits and commands arrive as the same AG-UI stream every `useChat` UI already renders. + +```ts +import { chat } from '@tanstack/ai' +import { grokBuildText } from '@tanstack/ai-grok-build' +import { defineSandbox, defineWorkspace, githubRepo, withSandbox } from '@tanstack/ai-sandbox' +import { dockerSandbox } from '@tanstack/ai-sandbox-docker' +import { messages, threadId } from './chat-context' + +const sandbox = defineSandbox({ + id: 'repo-agent', + provider: dockerSandbox({ image: 'node:22' }), + workspace: defineWorkspace({ + source: githubRepo({ repo: 'TanStack/ai' }), + packageManager: 'pnpm', + }), +}) + +const stream = chat({ + threadId, + adapter: grokBuildText('grok-build'), + messages, + middleware: [withSandbox(sandbox)], +}) +``` + +Two axes are open where the AI SDK's is narrower: + +- **Any agent, not a fixed list.** Grok Build, Claude Code, Codex, and OpenCode ship as first-party harness packages, and `acpCompatible` (from `@tanstack/ai-acp`) turns *any* [Agent Client Protocol](https://agentclientprotocol.com) agent — `pi`, `gemini --acp`, and [dozens of others](https://agentclientprotocol.com/get-started/agents) — into a harness by describing how to launch it. Adding an agent doesn't require a dedicated adapter to exist. +- **Any sandbox, not one cloud.** The same run executes on `localProcessSandbox` (host dev loop), `dockerSandbox` (real container isolation), Daytona, Vercel Sandbox, Sprites, or Cloudflare — swap the provider without touching the harness or workspace. Providers declare their `capabilities()` (`fs`, `exec`, `ports`, `snapshots`, `fork`, `durableFilesystem`, …) so code degrades gracefully across them. + +Vercel AI SDK 7 added a `HarnessAgent` API for the same idea — running a coding-agent harness in a sandbox and returning AI SDK-compatible `generate()` / `stream()` results. It's marked experimental, ships harnesses for Claude Code, Codex, Pi, OpenCode, and Deep Agents, and the documented path runs them in Vercel Sandbox. There's no generic ACP-compatible escape hatch (each supported harness is its own dedicated package), and sandbox support centers on Vercel's own microVM rather than a provider-swappable contract. + +### Media Generation + +TanStack AI provides stable, dedicated APIs for every media generation activity - image, video, speech, transcription, and summarization. Each is a separate, tree-shakeable function with its own adapter per provider. + +Vercel AI SDK has added several of these capabilities. As of v6, `generateImage()` is stable; video generation is still experimental (`experimental_generateVideo()`); and `generateSpeech()` / `transcribe()` are exported without the `experimental_` prefix but are still documented as experimental features. TanStack AI's media APIs are stable across the board and go further in several areas: + +**Image generation** - `generateImage()` with per-model type safety. TypeScript knows that `gpt-image-2` and `dall-e-3` expose different size constraints. Five providers ship adapters: OpenAI (GPT Image, DALL-E), Gemini (Imagen), Grok, OpenRouter, and fal.ai (600+ community models including Flux, SDXL, and more). + +```ts +import { generateImage } from '@tanstack/ai' +import { openaiImage } from '@tanstack/ai-openai' + +const result = await generateImage({ + adapter: openaiImage('gpt-image-2'), + prompt: 'A sunset over mountains', + size: '1536x1024', + numberOfImages: 1, +}) +``` + +**Video generation** - `generateVideo()` handles the full async job lifecycle automatically. Video generation APIs are inherently asynchronous - you submit a job, poll for status, and eventually get a result. TanStack AI manages this entire lifecycle with configurable polling intervals and timeouts, streaming status updates back to the client. + +```ts +import { generateVideo } from '@tanstack/ai' +import { openaiVideo } from '@tanstack/ai-openai' + +const stream = generateVideo({ + adapter: openaiVideo('sora-2'), + prompt: 'A cat playing piano', + size: '1280x720', + duration: 8, + stream: true, // Stream job lifecycle events + pollingInterval: 2000, // Poll every 2 seconds +}) + +for await (const chunk of stream) { + // Receive: job created → status updates → final video URL +} +``` + +Vercel AI SDK's `experimental_generateVideo()` returns the video directly without exposing the job lifecycle or streaming status updates. + +**Text-to-speech** - `generateSpeech()` supports 6 audio output formats (mp3, opus, aac, flac, wav, pcm), speed control (0.25x to 4x), and five providers: OpenAI (11 voices), Gemini (30+ voices with language hints), Grok, ElevenLabs, and fal.ai. + +```ts +import { generateSpeech } from '@tanstack/ai' +import { openaiSpeech } from '@tanstack/ai-openai' + +const result = await generateSpeech({ + adapter: openaiSpeech('tts-1-hd'), + text: 'Hello, world!', + voice: 'nova', + format: 'opus', + speed: 1.2, +}) +``` + +**Transcription** - `generateTranscription()` supports common output formats (json, text, srt, verbose_json, vtt), word-level timestamps with confidence scores, and four providers (OpenAI, Grok, ElevenLabs, fal.ai), with speaker diarization via OpenAI's `gpt-4o-transcribe-diarize` model. + +```ts +import { generateTranscription } from '@tanstack/ai' +import { openaiTranscription } from '@tanstack/ai-openai' +import { audioFile } from './audio' + +const result = await generateTranscription({ + adapter: openaiTranscription('gpt-4o-transcribe'), + audio: audioFile, + responseFormat: 'verbose_json', // Includes word-level timestamps +}) + +// result.words → [{ word: 'Hello', start: 0.0, end: 0.42 }, ...] +``` + +**Audio & music generation** - `generateAudio()` generates music and sound effects across Gemini (Lyria), ElevenLabs (music + sound effects), and fal.ai. Vercel AI SDK has no equivalent. + +**Summarization** - `summarize()` is a dedicated activity with style control (`bullet-points`, `paragraph`, `concise`), focus topics, and streaming support. Vercel AI SDK has no equivalent - summarization requires calling `generateText()` with a prompt. + +**Realtime voice** - `realtimeToken()` enables bidirectional audio streaming with Voice Activity Detection modes (server, semantic, manual), tool calling during voice sessions, and simultaneous audio + text output. Three providers ship realtime adapters: OpenAI (Realtime API), Grok, and ElevenLabs. Vercel AI SDK has no realtime/bidirectional voice primitive - its audio support is batch-only (`generateSpeech` and `transcribe`). + +All media activities follow the same adapter pattern as chat - tree-shakeable imports, per-model type safety, and streaming support. If your app only uses chat, none of this media code enters your bundle. + +### Native AG-UI Protocol + +The events TanStack AI streams between server and client are [AG-UI](https://docs.ag-ui.com/) events (`RUN_STARTED`, `TEXT_MESSAGE_*`, `TOOL_CALL_*`, `RUN_FINISHED`), imported directly from `@ag-ui/core` - not a bespoke format with an AG-UI export bolted on. Anything that speaks AG-UI can sit on either side of the wire: AG-UI-compliant agent frameworks behind a TanStack AI frontend, or a TanStack AI client in front of an agent server written in another language entirely. + +Vercel AI SDK streams its own proprietary UI Message Stream protocol. AG-UI interop requires an external translation layer (`@ag-ui/vercel-ai-sdk`, built and maintained by the AG-UI project), and native support remains an open feature request on the AI SDK repo. + +### Hooks for Every Activity + +Chat isn't the only activity with a hook. Every activity ships one - `useGeneration` (streaming structured output), `useGenerateImage`, `useGenerateAudio`, `useGenerateSpeech`, `useTranscription`, `useSummarize`, `useGenerateVideo`, and `useRealtimeChat` - with the same connection-adapter wiring and devtools integration as `useChat`, across React, Solid, Vue, Svelte, and Preact. + +Vercel AI SDK's UI layer has three hooks: `useChat`, `useCompletion`, and `useObject`. Its media functions (`generateImage()`, `experimental_generateVideo()`, speech, transcription) are server-side only - surfacing them in a UI means hand-rolling your own routes and client state. + +### Multi-Turn Structured Output + +Structured output in TanStack AI is part of the conversation, not a separate call. Pass `outputSchema` to `useChat` and every assistant turn carries its own typed `StructuredOutputPart` - streamed as a `partial`, validated as a `final`, preserved in message history, with the schema generic threading all the way down to `messages[i].parts[j].data`. + +Vercel AI SDK's structured output (`generateObject` / `streamObject` / `Output`) is per-call: the typed object lives on the call result, the message-part union has no structured-output type, and combining `useChat` with typed structured output means manually parsing model text into custom data parts. + +### Debug Logging + +Set `debug: true` on any activity and the pipeline prints itself: raw provider chunks, post-middleware output, middleware hook inputs and outputs, tool execution, agent-loop iterations, config transforms, and request metadata - each category individually toggleable, with a pluggable `logger` for structured output. Vercel AI SDK's built-in logging covers provider warnings; richer observability goes through experimental telemetry hooks or the dev-only DevTools recorder rather than a debug log you can flip on anywhere. + +### Community Adapter Ecosystem + +TanStack AI publishes an open adapter specification. The community has already built adapters for Decart, Cencori, Cloudflare, Soniox, and Mynth - with a [guide for building your own](/ai/community-adapters/guide). The adapter interface is simple enough that adding a new provider is a focused, self-contained task. + +## Where Vercel AI SDK Excels + +**Provider breadth.** Vercel AI SDK ships ~38 first-party, individually typed provider packages, plus a large community list. If you want a specific provider as a dedicated, maintained package without writing an adapter, their coverage is broader today. Raw model *count* is not the differentiator, though - TanStack AI's OpenRouter adapter reaches OpenRouter's full catalog (several hundred models), and the `openaiCompatible` adapter connects to any OpenAI-compatible endpoint. + +**Angular support.** Vercel AI SDK has an official Angular integration. TanStack AI supports React, Solid, Svelte, Vue, and Preact, but not Angular. (Solid now cuts the other way: AI SDK's Solid package is community-maintained and pinned to an older SDK major, while TanStack AI ships an official, current Solid integration.) + +**Agent abstraction.** Vercel AI SDK v6 ships a dedicated `Agent` abstraction (the `ToolLoopAgent` class) that packages a model, tools, instructions, and loop settings into a reusable object with `.generate()` and `.stream()` methods, plus `InferAgentUIMessage` for end-to-end type safety. TanStack AI composes these pieces per call rather than offering a single agent class. + +**AI Gateway.** Vercel's optional AI Gateway adds centralized provider management - failover routing, caching, and a single key across providers - integrated with the Vercel platform (and used by default when no provider is configured). TanStack AI ships no gateway of its own; for the same centralized routing across a large model catalog, it recommends its first-class OpenRouter adapter, with no platform association attached. + +**React Server Components.** Vercel AI SDK has an RSC integration via `@ai-sdk/rsc` (`AIState`, `StreamableValue`, `streamUI`). Note that Vercel documents this as experimental and recommends AI SDK UI for production - so it's an option for Next.js RSC apps rather than the primary path. + +## Side-by-Side: Key Differences + +### Tool Definition + +**TanStack AI** - Isomorphic definitions with separate runtime implementations: + +```ts +import { toolDefinition } from '@tanstack/ai' +import { z } from 'zod' +import { weatherApi } from './weather' + +const getWeather = toolDefinition({ + name: 'getWeather', + description: 'Get current weather for a location', + inputSchema: z.object({ city: z.string() }), + outputSchema: z.object({ temp: z.number(), condition: z.string() }), +}) + +// Server implementation +const getWeatherServer = getWeather.server(async ({ city }) => { + const data = await weatherApi.get(city) + return { temp: data.temperature, condition: data.condition } +}) + +// Client implementation +const getWeatherClient = getWeather.client(async ({ city }) => { + const res = await fetch(`/api/weather?city=${city}`) + return res.json() +}) +``` + +**Vercel AI SDK** - Tool objects via the `tool()` helper: + +```ts +import { generateText, tool } from 'ai' +import { openai } from '@ai-sdk/openai' +import { z } from 'zod' +import { weatherApi } from './weather' + +const result = await generateText({ + model: openai('gpt-5.5'), + tools: { + getWeather: tool({ + description: 'Get current weather for a location', + inputSchema: z.object({ city: z.string() }), + execute: async ({ city }) => { + const data = await weatherApi.get(city) + return { temp: data.temperature, condition: data.condition } + }, + }), + }, + prompt: "What's the weather in Tokyo?", +}) +``` + +The TanStack approach separates the tool contract from its implementation, making tools reusable across server and client contexts. + +### Agent Loop Control + +**TanStack AI** - Composable strategies: + +```ts +import { chat, combineStrategies, maxIterations, untilFinishReason } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { tools } from './tools' +import { estimatedTokens } from './cost' + +const messages = [{ role: 'user' as const, content: 'Help me plan a trip.' }] + +const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages, + tools, + agentLoopStrategy: combineStrategies([ + maxIterations(10), + untilFinishReason(['stop']), + ({ iterationCount }) => estimatedTokens(iterationCount) < 50_000, + ]), +}) +``` + +**Vercel AI SDK** - `stopWhen` conditions (v5+): + +```ts +import { generateText, stepCountIs } from 'ai' +import { openai } from '@ai-sdk/openai' +import { tools } from './tools' + +const result = await generateText({ + model: openai('gpt-5.5'), + tools, + stopWhen: stepCountIs(10), // also: hasToolCall('name'), or a custom function + prompt: 'Help me plan a trip.', +}) +``` + +Both let you compose multiple stopping conditions - `stopWhen` accepts an array of conditions including custom functions, and v6 adds a reusable `Agent` class. The remaining nuance is ergonomic: TanStack AI's strategies are plain `(state) => boolean` predicates combined with `combineStrategies`, so token budgets and custom business logic are first-class without reaching for a built-in condition. + +### Tree-Shaking + +**TanStack AI** - Separate adapters per activity: + +```ts +// Only bundles chat + OpenAI text adapter +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +``` + +**Vercel AI SDK** - Single provider import: + +```ts +// Provider package includes all model types +import { openai } from '@ai-sdk/openai' +``` + +In TanStack AI, each activity (chat, image, speech, video, transcription, summarization) is a separate adapter function. You import `openaiText` for chat and `openaiImage` for image generation - they're independent modules. Vercel AI SDK's provider packages are more monolithic. + +## When to Choose TanStack AI + +- **Bundle size matters** - Tree-shakeable adapters per activity mean smaller bundles +- **AG-UI native** - The wire protocol is AG-UI end to end; interoperate with the agent-UI ecosystem and non-TypeScript agent servers without a translation layer +- **Solid, Preact, or React Native** - One headless core covers React, Solid, Vue, Svelte, Preact, and React Native (via XHR adapters), all officially maintained +- **Hooks beyond chat** - `useGeneration`, `useGenerateImage`, `useSummarize`, and the rest of the generation hook family across every supported framework +- **Isomorphic tools** - Define a tool once and derive `.server()` / `.client()` implementations from one contract +- **App-level middleware** - Lifecycle hooks for chunks, tool calls, usage, and errors - not just model wrapping +- **Realtime voice** - Bidirectional audio across OpenAI, Grok, and ElevenLabs +- **No vendor association** - Pure library with no platform layer +- **Per-model type safety** - TypeScript narrows options per model, not per provider +- **Code execution** - Built-in sandboxed execution environments +- **Coding agent sandboxes** - Run Claude Code, Codex, Grok Build, OpenCode, or any ACP agent in a swappable sandbox (local, Docker, Daytona, Vercel, Sprites, Cloudflare), streamed through `chat()` +- **Flexible transport** - SSE, HTTP streams, XHR, RPC, direct iterables, or custom adapters +- **MCP, two ways** - A standalone host-side client (`@tanstack/ai-mcp`) with pools, codegen, and managed `chat()` lifecycle, plus a provider-routed `mcpTool()` + +## When to Choose Vercel AI SDK + +- **Need a first-party package for a specific provider** - ~38 dedicated, individually typed provider packages today (TanStack reaches comparable model breadth via OpenRouter + `openaiCompatible`) +- **Angular support** - Official Angular integration +- **Agent abstraction** - A reusable `Agent` (`ToolLoopAgent`) class with end-to-end UI message types +- **Vercel platform** - AI Gateway, observability, and deployment optimization +- **React Server Components** - RSC primitives via `@ai-sdk/rsc` (experimental; AI SDK UI is the recommended production path) + +## Getting Started + +```bash +npm install @tanstack/ai @tanstack/ai-openai +# or +pnpm add @tanstack/ai @tanstack/ai-openai +``` + +See the [Quick Start Guide](/ai/getting-started/quick-start) to build your first chat application, or explore the [full documentation](/ai/getting-started/overview). diff --git a/mintlify/ai/getting-started/agent-skills.md b/mintlify/ai/getting-started/agent-skills.md new file mode 100644 index 000000000..b1aeede80 --- /dev/null +++ b/mintlify/ai/getting-started/agent-skills.md @@ -0,0 +1,109 @@ +--- +title: Agent Skills (TanStack Intent) +id: agent-skills +order: 6 +description: "Use TanStack Intent to wire TanStack AI's bundled Agent Skills into Claude Code, Cursor, GitHub Copilot, and other AI coding assistants." +keywords: + - tanstack ai + - tanstack intent + - agent skills + - claude code + - cursor + - github copilot + - ai coding agents + - SKILL.md + - AGENTS.md +--- + +You're building with TanStack AI and using an AI coding agent — Claude Code, Cursor, GitHub Copilot, or similar. The agent keeps suggesting Vercel-AI-SDK patterns like `streamText()` or `createOpenAI()`, or it wires streams manually instead of using `toServerSentEventsResponse()`. By the end of this guide, your agent will load TanStack AI's bundled skills automatically whenever you work on AI code — and those skills will stay in sync with whichever `@tanstack/ai` version your project installs. + +> **Looking for runtime skills inside Code Mode?** Those are a different feature — see [Code Mode with Skills](/ai/code-mode/code-mode-with-skills). This page is about _agent-authoring_ skills: markdown files that teach your coding assistant how TanStack AI works. + +## What are Agent Skills? + +Agent Skills are markdown documents (`SKILL.md`) that ship inside npm packages and tell AI coding agents how to use a library correctly — which functions to use, which patterns to avoid, and when to reach for which module. The format is an open standard supported by Claude Code, Cursor, GitHub Copilot, Codex, and others. + +TanStack AI publishes skills inside its packages so the guidance travels with `npm update` instead of being pinned in a model's training data or copy-pasted into `CLAUDE.md` manually. + +## Skills Shipped by TanStack AI + +| Package | Skill | What it teaches | +|---------|-------|-----------------| +| `@tanstack/ai` | `ai-core` | Chat experience, tool calling, adapters, middleware, structured outputs, media generation, AG-UI protocol, custom backends | +| `@tanstack/ai-code-mode` | `ai-code-mode` | Setting up Code Mode with a sandbox driver and registering server tools | + +Each skill lives under `node_modules//skills//SKILL.md` once the package is installed. + +## Step 1: Install TanStack AI + +If you haven't already, install `@tanstack/ai` plus any adapter packages you need. See the [Quick Start](/ai/getting-started/quick-start) for a full walkthrough. + +```bash +pnpm add @tanstack/ai @tanstack/ai-openai +``` + +## Step 2: Run `intent install` + +From the root of your project, run: + +```bash +npx @tanstack/intent@latest install +``` + +The CLI walks your agent through the setup. It scans `node_modules` for every package that ships skills (any package with the `tanstack-intent` keyword), asks your agent to propose task-to-skill mappings that match your codebase, and writes them into your agent's config file. + +By default the mappings land in `AGENTS.md`. The CLI can also target: + +- `CLAUDE.md` — Claude Code +- `.cursorrules` — Cursor +- any other agent config file you point it at + +## Step 3: Review the Generated Mappings + +The install command appends (or creates) an `intent-skills` block that looks like this: + +```yaml + +# Skill mappings — when working in these areas, load the linked skill file into context. +skills: + - task: "Building chat, tool calling, adapters, or streaming with TanStack AI" + load: "node_modules/@tanstack/ai/skills/ai-core/SKILL.md" + - task: "Setting up Code Mode with TanStack AI" + load: "node_modules/@tanstack/ai-code-mode/skills/ai-code-mode/SKILL.md" + +``` + +Check that the `task:` descriptions match areas you actually work in. Tighten or reword them if needed — they're how your agent decides when to pull the skill into context. + +## Step 4: Confirm It's Wired Up + +Open a fresh session in your coding agent and ask it to build something with TanStack AI — for example: _"Add a streaming chat endpoint using `@tanstack/ai` and the OpenAI adapter."_ + +You should see: + +- The agent uses `chat()`, not `streamText()`. +- The adapter is imported as `openaiText()` from `@tanstack/ai-openai`, not `createOpenAI()`. +- The response is wrapped with `toServerSentEventsResponse()` instead of manual SSE wiring. +- Middleware is used for lifecycle events (no `onFinish` callback on `chat()`). + +If the agent still falls back to other-SDK patterns, re-open its config file and confirm the `intent-skills` block is present and the `task:` descriptions clearly cover the area you're asking about. + +## Keeping Skills Current + +Skills are versioned with the package. When you bump `@tanstack/ai`, the `SKILL.md` files under `node_modules` update with it — no CLI re-run needed. Re-run `npx @tanstack/intent@latest install` only when you _add_ a new intent-enabled package (for example, adding `@tanstack/ai-code-mode` later) or want to refresh the task mappings. + +## Using Skills Without the CLI + +If you'd rather wire skills in yourself, you can reference them directly from `node_modules` in any agent config file. The minimum your agent needs is a pointer to the file: + +```markdown +When working on TanStack AI code, read and follow: +node_modules/@tanstack/ai/skills/ai-core/SKILL.md +``` + +The CLI is recommended because it discovers packages automatically and stays consistent with the agent-skills standard, but the underlying file paths are stable. + +## Learn More + +- [TanStack Intent documentation](/intent/overview) — the CLI's full reference, including `scaffold`, `validate`, and CI setup for library maintainers. +- [Agent Skills registry](https://tanstack.com/intent/registry) — browse other intent-enabled packages. diff --git a/mintlify/ai/getting-started/devtools.md b/mintlify/ai/getting-started/devtools.md new file mode 100644 index 000000000..259363da6 --- /dev/null +++ b/mintlify/ai/getting-started/devtools.md @@ -0,0 +1,145 @@ +--- +title: Devtools +id: devtools +order: 3 +description: "Inspect and debug TanStack AI apps with the TanStack Devtools panel — live chat messages, tool call inputs and outputs, state, and errors." +keywords: + - tanstack ai + - devtools + - debugging + - tool inspection + - chat inspector + - react devtools + - observability +--- + +TanStack Devtools is a unified devtools panel for inspecting and debugging TanStack libraries, including TanStack AI. It provides real-time insights into AI interactions, tool calls, and state changes, making it easier to develop and troubleshoot AI-powered applications. + +## Features +- **Hook dashboard** - Discover every active TanStack AI hook on the page, including chat, structured output, image, video, audio, speech, transcription, and summarize hooks. +- **Run timeline** - Inspect user turns, linked runs, stream events, client snapshots, and server-only events by `threadId` and `runId`. +- **Real-time Monitoring** - View live chat messages, tool invocations, and AI responses. +- **Tool Call Inspection** - Inspect input and output of tool calls. +- **Tool Fixture Replay** - Build tool payloads from a tool's standard-schema input, append the result into chat messages, and save fixtures in localStorage for repeated UI iteration. +- **State Visualization** - Visualize chat state and message history. +- **Error Tracking** - Monitor errors and exceptions in AI interactions. + +## Hook Dashboard + +The AI devtools panel listens for active TanStack AI clients and shows them in the left sidebar. Hooks register when they are created, emit a snapshot immediately, and respond again whenever the devtools panel opens or requests state. This keeps hooks discoverable even when the panel is opened after the app has already rendered. + +Each hook entry includes its type, lifecycle, message count, run count, and the latest linked `threadId`. Selecting a hook opens the full timeline for that hook. Chat hooks keep the current turn-based view: a user message wraps every run and event that happened while answering that turn. The details view also includes lightweight client/server state snapshots between runs so you can see exactly what changed. + +### Naming Hooks + +When a page has more than one AI hook, pass `devtools.name` to give each hook a user-facing label in the dashboard. The configured name is display-only; hook type, framework, thread id, and run correlation still come from the TanStack AI client. + +```tsx +import { fetchServerSentEvents, useChat } from '@tanstack/ai-react' + +export function SupportChat() { + const chat = useChat({ + id: 'support-chat', + connection: fetchServerSentEvents('/api/chat'), + devtools: { + name: 'Support Chat', + }, + }) + + // render your chat UI with `chat.messages`, `chat.sendMessage`, etc. +} +``` + +The same display option works for specialized generation hooks: + +```tsx +import { fetchServerSentEvents, useGenerateImage } from '@tanstack/ai-react' + +export function ImageStudio() { + const image = useGenerateImage({ + id: 'generation-hooks:useGenerateImage', + connection: fetchServerSentEvents('/api/image'), + devtools: { + name: 'Image Studio', + }, + }) + + // render your image generation UI with `image.generate` and `image.result` +} +``` + +## Tool Fixtures + +When a `useChat` hook receives tools, the devtools panel lists those tools and their schemas. For standard-schema-compatible inputs, the panel renders a small form from the input schema so you can create a tool call payload without hand-writing JSON. + +Applying a tool fixture appends the tool call and result into the real chat messages for that hook. Saved fixtures are stored in browser localStorage under the AI devtools namespace so they are available the next time you open the panel. + +## Event Sources + +Client-visible state is emitted by the headless client. Server-only details, such as middleware and provider stream events that never exist on the client, are emitted from the server counterpart. Events include a source descriptor and stable envelope id so the panel can link related events and avoid displaying duplicates. + +## Installation +To use TanStack Devtools with TanStack AI, install the `@tanstack/react-ai-devtools` package: + +```bash +npm install -D @tanstack/react-ai-devtools @tanstack/react-devtools +``` + +Or the `@tanstack/solid-ai-devtools` package for SolidJS: +```bash +npm install -D @tanstack/solid-ai-devtools @tanstack/solid-devtools +``` + +Or the `@tanstack/preact-ai-devtools` package for Preact: +```bash +npm install -D @tanstack/preact-ai-devtools @tanstack/preact-devtools +``` + +## Usage + +Import and include the TanStackDevtools component in your application: + +```tsx +import { TanStackDevtools } from '@tanstack/react-devtools' +import { aiDevtoolsPlugin } from '@tanstack/react-ai-devtools' + +const App = () => { + return ( + <> + + + ) +} +``` + +## Using with Next.js (or without a Vite plugin) + +`connectToServerBus: true` relies on a WebSocket/SSE server on port 4206 that is normally started by `@tanstack/devtools-vite`. If you're using Next.js (or any non-Vite bundler), you need to start `ServerEventBus` manually at server boot. + +In Next.js, do this in `instrumentation.ts`: + +```ts ignore +export async function register() { + if ( + process.env["NEXT_RUNTIME"] === "nodejs" && + process.env.NODE_ENV === "development" + ) { + const { ServerEventBus } = await import( + "@tanstack/devtools-event-bus/server" + ); + const bus = new ServerEventBus(); + await bus.start(); + } +} +``` + +This sets globalThis.__TANSTACK_EVENT_TARGET__ so the server-side devtoolsMiddleware (which runs automatically inside every chat() call) can emit tool call events to the bus, which then forwards them to the devtools panel. diff --git a/mintlify/ai/getting-started/overview.md b/mintlify/ai/getting-started/overview.md new file mode 100644 index 000000000..85f7d1ad7 --- /dev/null +++ b/mintlify/ai/getting-started/overview.md @@ -0,0 +1,123 @@ +--- +title: Overview +id: overview +order: 1 +description: "TanStack AI is a type-safe, provider-agnostic TypeScript SDK for building streaming chat, tool calling, and AI features that work across any framework." +keywords: + - tanstack ai + - ai sdk + - typescript ai + - streaming chat + - tool calling + - isomorphic tools + - framework agnostic + - llm sdk +--- + +TanStack AI is a lightweight, type-safe SDK for building production-ready AI experiences. Its framework-agnostic core provides type-safe tool/function calling, streaming responses, and first-class React and Solid integrations, with adapters for multiple LLM providers — enabling predictable, composable, and testable AI features across any stack. + +## Key Features + +- ✅ **Type-Safe** - Full TypeScript support with Zod schema inference +- ✅ **Streaming** - Built-in streaming support for real-time responses +- ✅ **Isomorphic Tools** - Define once with `toolDefinition()`, implement with `.server()` or `.client()` +- ✅ **Framework Agnostic** - Core library works anywhere +- ✅ **Multiple Providers** - OpenRouter, OpenAI, Anthropic, Gemini, Ollama, and more +- ✅ **Approval Flow** - Built-in support for tool approval workflows +- ✅ **Automatic Execution** - Both server and client tools execute automatically + +## Framework Agnostic + +The framework-agnostic core of TanStack AI provides the building blocks for creating AI experiences in any environment, including: + +- **Next.js** - API routes and App Router +- **TanStack Start** - React Start or Solid Start (recommended!) +- **React Native / Expo** - Native chat screens with `useChat`, absolute server URLs, and XHR streaming transports +- **Express** - Node.js server +- **React Router v7** - Loaders and actions + +TanStack AI lets you define a tool once and provide environment-specific implementations. Using `toolDefinition()` to declare the tool's input/output types and the server behavior with `.server()` (or a client implementation with `.client()`). These isomorphic tools can be invoked from the AI runtime regardless of framework. + +```typescript +import { chat, toolDefinition } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { z } from 'zod' +import { db } from './db' + +// Define a tool +const getProductsDef = toolDefinition({ + name: 'getProducts', + description: 'Search for products by query', + inputSchema: z.object({ query: z.string() }), + outputSchema: z.array(z.object({ id: z.string(), name: z.string() })), +}) + +// Create server implementation +const getProducts = getProductsDef.server(async ({ query }) => { + return await db.products.search(query) +}) + +// Use in AI chat +chat({ + adapter: openaiText('gpt-5.2'), + messages: [{ role: 'user', content: 'Find products' }], + tools: [getProducts] +}) +``` + +## Core Packages + +The TanStack AI ecosystem consists of several packages: + +### `@tanstack/ai` +The core AI library that provides: +- AI adapter interface for connecting to LLM providers +- Chat completion and streaming +- Isomorphic tool/function calling system +- Agent loop strategies +- Type-safe tool definitions with `toolDefinition()` +- Type-safe Model Options based on adapter & model selection +- Type-safe content modalities (text, image, audio, video, document) based on model capabilities + +### `@tanstack/ai-client` +A framework-agnostic headless client for managing chat state: +- Message management with full type safety +- Streaming support +- Connection adapters (SSE, HTTP stream, custom) +- Automatic tool execution (server and client) +- Tool approval flow handling + +### `@tanstack/ai-react` +React hooks for TanStack AI: +- `useChat` hook for chat interfaces +- Automatic state management +- Tool approval flow support +- Type-safe message handling with `InferChatMessages` + +### `@tanstack/ai-solid` +Solid hooks for TanStack AI: +- `useChat` hook for chat interfaces +- Automatic state management +- Tool approval flow support +- Type-safe message handling with `InferChatMessages` + +## Adapters + +With the help of adapters, TanStack AI can connect to various LLM providers. Available adapters include: + +- **@tanstack/ai-openrouter** - OpenRouter (300+ models via a single API key — recommended) +- **@tanstack/ai-openai** - OpenAI (GPT series) +- **@tanstack/ai-anthropic** - Anthropic (Claude) +- **@tanstack/ai-gemini** - Google Gemini +- **@tanstack/ai-ollama** - Ollama (local models) +- **@tanstack/ai-groq** - Groq +- **@tanstack/ai-grok** - xAI Grok +- **@tanstack/ai-bedrock** - Amazon Bedrock (Claude, Nova, Llama, and more via AWS) +- **@tanstack/ai-fal** - fal (image & video generation) + +## Next Steps + +- [Quick Start Guide](/ai/getting-started/quick-start) - Get up and running in minutes +- [Quick Start: React Native](/ai/getting-started/quick-start-react-native) - Add mobile chat with Expo and a server-owned provider boundary +- [Tools Guide](/ai/tools/tools) - Learn about the isomorphic tool system +- [API Reference](/ai/api/ai) - Explore the full API diff --git a/mintlify/ai/getting-started/quick-start-angular.md b/mintlify/ai/getting-started/quick-start-angular.md new file mode 100644 index 000000000..92e14292d --- /dev/null +++ b/mintlify/ai/getting-started/quick-start-angular.md @@ -0,0 +1,224 @@ +--- +title: "Quick Start: Angular" +id: quick-start-angular +order: 4 +description: "Build a streaming TanStack AI chat component in an Angular app using the injectChat function and the OpenAI adapter." +keywords: + - tanstack ai + - angular + - quick start + - injectChat + - streaming chat + - openai + - signals +--- + +You have an Angular app and want to add AI chat. By the end of this guide, you'll have a streaming chat component powered by TanStack AI and OpenAI. + +> **Tip:** If you'd prefer not to sign up with individual AI providers, [OpenRouter](/ai/adapters/openrouter) gives you access to 300+ models with a single API key and is the easiest way to get started. + +## Installation + +```bash +npm install @tanstack/ai @tanstack/ai-angular @tanstack/ai-openai +# or +pnpm add @tanstack/ai @tanstack/ai-angular @tanstack/ai-openai +# or +yarn add @tanstack/ai @tanstack/ai-angular @tanstack/ai-openai +``` + +## Server Setup + +Angular apps typically use a separate backend. Here's an Express server that streams chat responses: + +```typescript ignore +import express from 'express' +import { chat, toServerSentEventsResponse } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +const app = express() +app.use(express.json()) + +app.post('/api/chat', async (req, res) => { + const { messages } = req.body + + if (!process.env.OPENAI_API_KEY) { + res.status(500).json({ error: 'OPENAI_API_KEY not configured' }) + return + } + + try { + // `chat()` uses the AG-UI `threadId` for devtools correlation + // when available — no need to plumb `conversationId` manually. + const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages, + }) + + const response = toServerSentEventsResponse(stream) + res.writeHead(response.status, Object.fromEntries(response.headers)) + + const body = response.body + if (body) { + const reader = body.getReader() + const pump = async () => { + const { done, value } = await reader.read() + if (done) { + res.end() + return + } + res.write(value) + await pump() + } + await pump() + } + } catch (error) { + res.status(500).json({ + error: error instanceof Error ? error.message : 'An error occurred', + }) + } +}) + +app.listen(3000, () => console.log('Server running on port 3000')) +``` + +> **Tip:** Any backend that returns the TanStack AI SSE format works — you can use Fastify, Hono, Nitro, or any other Node.js framework. + +## Client Setup + +Create a standalone `ChatComponent` using the `injectChat` function: + +```typescript group=quick-start-angular +import { Component, signal } from '@angular/core' +import { FormsModule } from '@angular/forms' +import { injectChat } from '@tanstack/ai-angular' +import { fetchServerSentEvents } from '@tanstack/ai-client' + +@Component({ + selector: 'app-chat', + standalone: true, + imports: [FormsModule], + template: ` +
+
+ @for (message of chat.messages(); track message.id) { +
+ {{ message.role === 'assistant' ? 'Assistant' : 'You' }} + @for (part of message.parts; track $index) { + @if (part.type === 'text') { +

{{ part.content }}

+ } + } +
+ } +
+ +
+ + +
+
+ `, +}) +export class ChatComponent { + // injectChat is called in a field initializer — this is a valid injection context. + chat = injectChat({ + connection: fetchServerSentEvents('/api/chat'), + }) + + input = signal('') + + handleSubmit() { + const text = this.input().trim() + if (text && !this.chat.isLoading()) { + this.chat.sendMessage(text) + this.input.set('') + } + } +} +``` + +## Environment Variables + +Create a `.env` file (or `.env.local` depending on your setup) with your API key: + +```bash +# OpenRouter (recommended — access 300+ models with one key) +OPENROUTER_API_KEY=sk-or-... + +# OpenAI +OPENAI_API_KEY=your-openai-api-key +``` + +Your server reads this key at runtime. Never expose it to the browser. + +## Angular-Specific Notes + +**State is exposed as Angular `Signal`s.** The `injectChat` function returns state wrapped in read-only `Signal`s. Read them by calling them as functions: + +```typescript ignore +// In component class +if (this.chat.isLoading()) { /* ... */ } +const count = this.chat.messages().length + +// In template — same syntax, no .value needed +``` + +```html + +@if (chat.isLoading()) { +

Thinking...

+} +{{ chat.messages().length }} messages +``` + +**`injectChat` must be called in an injection context.** Angular's dependency injection requires that `inject()` is called during component construction. The recommended approach is a field initializer (shown above). You can also call it in the constructor or inside `runInInjectionContext`: + +```typescript +import { injectChat } from '@tanstack/ai-angular' +import { fetchServerSentEvents } from '@tanstack/ai-client' + +// Field initializer (recommended) +export class MyComponentA { + chat = injectChat({ connection: fetchServerSentEvents('/api/chat') }) +} + +// Constructor +export class MyComponentB { + chat: ReturnType + constructor() { + this.chat = injectChat({ connection: fetchServerSentEvents('/api/chat') }) + } +} +``` + +Calling `injectChat` outside an injection context — for example, in a lifecycle hook like `ngOnInit` — will throw a runtime error. + +**Automatic cleanup.** The function subscribes to `DestroyRef` internally, so in-flight requests are stopped when the component is destroyed. No manual cleanup needed. + +**Same API shape as React and Vue.** If you're coming from `@tanstack/ai-react` or `@tanstack/ai-vue`, `injectChat` returns the same properties (`messages`, `sendMessage`, `isLoading`, `error`, `status`, `stop`, `reload`, `clear`). The only difference is that each property is an Angular `Signal` rather than a React state value or a Vue `ShallowRef`. + +## That's It! + +You now have a working Angular chat application. The `injectChat` function handles: + +- Message state management +- Streaming responses +- Loading states +- Error handling + +## Next Steps + +- Learn about [Tools](/ai/tools/tools) to add function calling +- Check out the [Adapters](/ai/adapters/openai) to connect to different providers +- See the [React Quick Start](/ai/getting-started/quick-start) if you're comparing frameworks diff --git a/mintlify/ai/getting-started/quick-start-react-native.md b/mintlify/ai/getting-started/quick-start-react-native.md new file mode 100644 index 000000000..36e16051b --- /dev/null +++ b/mintlify/ai/getting-started/quick-start-react-native.md @@ -0,0 +1,305 @@ +--- +title: "Quick Start: React Native" +id: quick-start-react-native +order: 3 +description: "Build a React Native or Expo chat screen with TanStack AI's useChat hook, a server-only OpenAI backend, and mobile-compatible streaming transports." +keywords: + - tanstack ai + - react native + - expo + - mobile + - useChat + - streaming + - xhrHttpStream + - openai +--- + +You have a React Native or Expo app and you want to add streaming AI chat +without putting provider SDKs or API keys in the native bundle. By the end of +this guide, your app will call a server-owned Hono route with `useChat` from +`@tanstack/ai-react`, stream responses over a mobile-compatible transport, and +keep `OPENAI_API_KEY` / `OPENAI_MODEL` on the server. + +> **Coming from the web quick start?** The hook is the same, but the URL and +> transport are different. React Native needs an absolute backend URL, not +> `/api/chat`, and most Expo runtimes should start with `xhrHttpStream()`. + +## 1. Install packages + +If you are starting from scratch, create an Expo app first: + +```bash +npx create-expo-app@latest my-ai-chat +``` + +Install TanStack AI, the React hook package, the OpenAI adapter for your +server, and Hono for the example backend: + +```bash +pnpm add @tanstack/ai @tanstack/ai-react @tanstack/ai-openai hono @hono/node-server zod +``` + +If your Expo app lives in a workspace, run the command from the app package or +use your workspace filter. + +## 2. Keep OpenAI on the server + +Create a Hono route that owns the model, API key, and response format. The +native app sends chat messages to this route; it never imports +`@tanstack/ai-openai` and never receives `OPENAI_API_KEY`. + +```ts +// server.ts +import { serve } from '@hono/node-server' +import { chat, toHttpResponse, toServerSentEventsResponse } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { Hono } from 'hono' +import { model } from './config' + +const app = new Hono() + +function requireOpenAIKey() { + if (!process.env.OPENAI_API_KEY) { + throw new Error('OPENAI_API_KEY is not configured on the server') + } +} + +app.get('/health', (c) => c.json({ ok: true })) + +app.post('/chat/http', async (c) => { + requireOpenAIKey() + const body = await c.req.json() + const stream = chat({ + adapter: openaiText(model), + messages: body.messages, + }) + + return toHttpResponse(stream, { + headers: { + 'Content-Type': 'application/x-ndjson', + 'Cache-Control': 'no-cache', + }, + }) +}) + +app.post('/chat/sse', async (c) => { + requireOpenAIKey() + const body = await c.req.json() + const stream = chat({ + adapter: openaiText(model), + messages: body.messages, + }) + + return toServerSentEventsResponse(stream) +}) + +serve({ + fetch: app.fetch, + hostname: '0.0.0.0', + port: Number(process.env.PORT ?? 8787), +}) +``` + +Set server-only environment variables where the Hono process runs: + +```env +OPENAI_API_KEY=sk-... +OPENAI_MODEL=gpt-5.2 +``` + +Run the Hono server before starting the native app. For a TypeScript-only +example, install `tsx` and add a script: + +```bash +pnpm add -D tsx +pnpm pkg set scripts.dev:server="tsx server.ts" +pnpm dev:server +``` + +> **Route pairing matters:** `xhrHttpStream()` and `fetchHttpStream()` expect +> the newline-delimited JSON response from `toHttpResponse()`. +> `xhrServerSentEvents()` expects the `text/event-stream` response from +> `toServerSentEventsResponse()`. + +## 3. Configure a native-reachable URL + +React Native is not served from your backend origin, so `/api/chat` cannot work +as a default. Expose the backend URL to Expo with a public variable: + +```env +EXPO_PUBLIC_TANSTACK_AI_BASE_URL=http://192.168.1.10:8787 +``` + +Use the address your device can reach: + +- iOS simulator: `http://127.0.0.1:8787` often works. +- Android emulator: use `http://10.0.2.2:8787`. +- Physical device: use your computer's LAN IP, for example + `http://192.168.1.10:8787`, or a tunneled HTTPS URL. + +Only `EXPO_PUBLIC_*` values are bundled into the app. Keep provider keys as +plain server variables such as `OPENAI_API_KEY`. + +## 4. Use `useChat` in your native screen + +Start with `xhrHttpStream()` for Expo and React Native. It reads the same +newline-delimited JSON produced by `toHttpResponse()` and relies on XHR progress +events, which are usually more reliable on phone runtimes than streaming +`fetch`. + +```tsx +// ChatScreen.tsx +import { useState } from 'react' +import { Button, ScrollView, Text, TextInput, View } from 'react-native' +import { useChat, xhrHttpStream } from '@tanstack/ai-react' + +const baseUrl = + process.env.EXPO_PUBLIC_TANSTACK_AI_BASE_URL ?? 'http://127.0.0.1:8787' + +export function ChatScreen() { + const [input, setInput] = useState('') + const { messages, sendMessage, isLoading, error } = useChat({ + connection: xhrHttpStream(`${baseUrl}/chat/http`), + }) + + async function send() { + const text = input.trim() + if (!text || isLoading) return + setInput('') + await sendMessage(text) + } + + return ( + + + {messages.map((message) => ( + + {message.role} + {message.parts.map((part, index) => + part.type === 'text' ? ( + {part.content} + ) : null, + )} + + ))} + + + {error ? {error.message} : null} + + + + + +``` + +## Environment Variables + +Create a `.env` file with your API key: + +```bash +# OpenRouter (recommended -- access 300+ models with one key) +OPENROUTER_API_KEY=sk-or-... + +# OpenAI +OPENAI_API_KEY=your-openai-api-key +``` + +Your SvelteKit server reads this key at runtime. Never expose it to the browser. + +## Svelte-Specific Notes + +**`createChat`, not `useChat`.** The Svelte integration uses `createChat` instead of `useChat` to follow Svelte's naming conventions. The returned object has the same properties as the React and Vue versions (`messages`, `sendMessage`, `isLoading`, `error`, `status`, `stop`, `reload`, `clear`). + +**Svelte 5 runes.** The examples above use Svelte 5 runes (`$state`). The `createChat` return object uses reactive getters internally, so `chat.messages` and `chat.isLoading` are reactive without any extra wrappers -- no `.value` like Vue, no signals to unwrap. + +**No automatic cleanup.** Unlike the React and Vue integrations, `createChat` does not register automatic cleanup. If your component can unmount while a response is streaming, call `chat.stop()` in an `onDestroy` callback: + +```svelte + +``` + +## That's It! + +You now have a working SvelteKit chat application. The `createChat` function handles: + +- Message state management +- Streaming responses +- Loading states +- Error handling + +## Next Steps + +- Learn about [Tools](/ai/tools/tools) to add function calling +- Check out the [Adapters](/ai/adapters/openai) to connect to different providers +- See the [React Quick Start](/ai/getting-started/quick-start) if you're comparing frameworks diff --git a/mintlify/ai/getting-started/quick-start-vue.md b/mintlify/ai/getting-started/quick-start-vue.md new file mode 100644 index 000000000..15671d1c4 --- /dev/null +++ b/mintlify/ai/getting-started/quick-start-vue.md @@ -0,0 +1,189 @@ +--- +title: "Quick Start: Vue" +id: quick-start-vue +order: 3 +description: "Build a streaming TanStack AI chat component in a Vue 3 app using the useChat composable and the OpenAI adapter." +keywords: + - tanstack ai + - vue + - vue 3 + - quick start + - useChat + - streaming chat + - openai + - composable +--- + +You have a Vue 3 app and want to add AI chat. By the end of this guide, you'll have a streaming chat component powered by TanStack AI and OpenAI. + +> **Tip:** If you'd prefer not to sign up with individual AI providers, [OpenRouter](/ai/adapters/openrouter) gives you access to 300+ models with a single API key and is the easiest way to get started. + +## Installation + +```bash +npm install @tanstack/ai @tanstack/ai-vue @tanstack/ai-openai +# or +pnpm add @tanstack/ai @tanstack/ai-vue @tanstack/ai-openai +# or +yarn add @tanstack/ai @tanstack/ai-vue @tanstack/ai-openai +``` + +## Server Setup + +Vue apps typically use a separate backend. Here's an Express server that streams chat responses: + +```typescript ignore +import express from 'express' +import { chat, toServerSentEventsResponse } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' + +const app = express() +app.use(express.json()) + +app.post('/api/chat', async (req, res) => { + const { messages } = req.body + + if (!process.env.OPENAI_API_KEY) { + res.status(500).json({ error: 'OPENAI_API_KEY not configured' }) + return + } + + try { + // `chat()` uses the AG-UI `threadId` for devtools correlation + // when available — no need to plumb `conversationId` manually. + const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages, + }) + + const response = toServerSentEventsResponse(stream) + res.writeHead(response.status, Object.fromEntries(response.headers)) + + const body = response.body + if (body) { + const reader = body.getReader() + const pump = async () => { + const { done, value } = await reader.read() + if (done) { + res.end() + return + } + res.write(value) + await pump() + } + await pump() + } + } catch (error) { + res.status(500).json({ + error: error instanceof Error ? error.message : 'An error occurred', + }) + } +}) + +app.listen(3000, () => console.log('Server running on port 3000')) +``` + +> **Tip:** Any backend that returns the TanStack AI SSE format works -- you can use Fastify, Hono, Nitro, or any other Node.js framework. + +## Client Setup + +Create a `Chat.vue` component using the `useChat` composable: + +```vue + + + +``` + +## Environment Variables + +Create a `.env` file (or `.env.local` depending on your setup) with your API key: + +```bash +# OpenRouter (recommended — access 300+ models with one key) +OPENROUTER_API_KEY=sk-or-... + +# OpenAI +OPENAI_API_KEY=your-openai-api-key +``` + +Your server reads this key at runtime. Never expose it to the browser. + +## Vue-Specific Notes + +**Reactive state uses `ShallowRef`.** The `useChat` composable returns state wrapped in `DeepReadonly>`. In ` + + +``` + +**Automatic cleanup.** The composable calls `onScopeDispose` internally, so in-flight requests are stopped when the component unmounts. No manual cleanup needed. + +**Same API shape as React.** If you're coming from `@tanstack/ai-react`, the Vue composable returns the same properties (`messages`, `sendMessage`, `isLoading`, `error`, `status`, `stop`, `reload`, `clear`). The only difference is the `ShallowRef` wrapper. + +## That's It! + +You now have a working Vue chat application. The `useChat` composable handles: + +- Message state management +- Streaming responses +- Loading states +- Error handling + +## Next Steps + +- Learn about [Tools](/ai/tools/tools) to add function calling +- Check out the [Adapters](/ai/adapters/openai) to connect to different providers +- See the [React Quick Start](/ai/getting-started/quick-start) if you're comparing frameworks diff --git a/mintlify/ai/getting-started/quick-start.md b/mintlify/ai/getting-started/quick-start.md new file mode 100644 index 000000000..39270cda2 --- /dev/null +++ b/mintlify/ai/getting-started/quick-start.md @@ -0,0 +1,291 @@ +--- +title: "Quick Start: React" +id: quick-start +order: 2 +description: "Add a streaming TanStack AI chat to a React app in minutes using the useChat hook and the OpenAI adapter." +keywords: + - tanstack ai + - react + - quick start + - useChat + - streaming chat + - openai + - tutorial + - ai chatbot +--- + +Get started with TanStack AI in minutes. This guide will walk you through creating a simple chat application using the React integration and OpenAI adapter. + +> **Using a different framework?** See quick-starts for [Vue](/ai/getting-started/quick-start-vue), [Svelte](/ai/getting-started/quick-start-svelte), or [server-only Node.js](/ai/getting-started/quick-start-server). + +> **React Native or Expo app?** Use the headless React hooks with an absolute +> server URL and a mobile-compatible transport. See +> [Quick Start: React Native](/ai/getting-started/quick-start-react-native). + +> **Tip:** If you'd prefer not to sign up with individual AI providers, [OpenRouter](/ai/adapters/openrouter) gives you access to 300+ models with a single API key and is the easiest way to get started. + +## Installation + +```bash +npm install @tanstack/ai @tanstack/ai-react @tanstack/ai-openai +# or +pnpm add @tanstack/ai @tanstack/ai-react @tanstack/ai-openai +#or +yarn add @tanstack/ai @tanstack/ai-react @tanstack/ai-openai +``` + +## Server Setup + +First, create an API route that handles chat requests. Here's a simplified example: + +### TanStack Start + +```typescript ignore +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import { createFileRoute } from "@tanstack/react-router"; + +export const Route = createFileRoute("/api/chat")({ + server: { + handlers: { + POST: async ({ request }) => { + // Check for API key + if (!process.env.OPENAI_API_KEY) { + return new Response( + JSON.stringify({ + error: "OPENAI_API_KEY not configured", + }), + { + status: 500, + headers: { "Content-Type": "application/json" }, + }, + ); + } + + const body = await request.json(); + + try { + // Create a streaming chat response. `chat()` reads the AG-UI + // `threadId` for devtools correlation when available. + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: body.messages, + }); + + // Convert stream to HTTP response + return toServerSentEventsResponse(stream); + } catch (error) { + return new Response( + JSON.stringify({ + error: + error instanceof Error ? error.message : "An error occurred", + }), + { + status: 500, + headers: { "Content-Type": "application/json" }, + }, + ); + } + }, + }, + }, +}); +``` + +### Next.js + +```typescript +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; + +export async function POST(request: Request) { + // Check for API key + if (!process.env.OPENAI_API_KEY) { + return new Response( + JSON.stringify({ + error: "OPENAI_API_KEY not configured", + }), + { + status: 500, + headers: { "Content-Type": "application/json" }, + } + ); + } + + const body = await request.json(); + + try { + // Create a streaming chat response. `chat()` reads the AG-UI + // `threadId` for devtools correlation when available. + const stream = chat({ + adapter: openaiText("gpt-5.5"), + messages: body.messages, + }); + + // Convert stream to HTTP response + return toServerSentEventsResponse(stream); + } catch (error) { + return new Response( + JSON.stringify({ + error: error instanceof Error ? error.message : "An error occurred", + }), + { + status: 500, + headers: { "Content-Type": "application/json" }, + } + ); + } +} +``` + +## Client Setup + +To use the chat API from your React frontend, create a `Chat` component: + +```tsx +// components/Chat.tsx +import { useState } from "react"; +import { useChat, fetchServerSentEvents } from "@tanstack/ai-react"; + +export function Chat() { + const [input, setInput] = useState(""); + + const { messages, sendMessage, isLoading, error } = useChat({ + connection: fetchServerSentEvents("/api/chat"), + }); + + const handleSubmit = (e: React.FormEvent) => { + e.preventDefault(); + if (input.trim() && !isLoading) { + sendMessage(input); + setInput(""); + } + }; + + return ( +
+ {/* Messages */} +
+ {messages.map((message) => ( +
+
+ {message.role === "assistant" ? "Assistant" : "You"} +
+
+ {message.parts.map((part, idx) => { + if (part.type === "thinking") { + return ( +
+ 💭 Thinking: {part.content} +
+ ); + } + if (part.type === "text") { + return
{part.content}
; + } + return null; + })} +
+
+ ))} +
+ + {/* Error */} + {error && ( +

+ {error.message} +

+ )} + + {/* Input */} +
+
+ setInput(e.target.value)} + placeholder="Type a message..." + className="flex-1 px-4 py-2 border rounded-lg" + disabled={isLoading} + /> + +
+
+
+ ); +} +``` + +## Environment Variables + +To connect to AI providers, set your API keys in your environment variables. Create a `.env.local` file (or `.env` depending on your setup): +```bash +# OpenRouter (recommended — access 300+ models with one key) +OPENROUTER_API_KEY=sk-or-... + +# OpenAI +OPENAI_API_KEY=your-openai-api-key + +# Anthropic +ANTHROPIC_API_KEY=your-anthropic-api-key + +# Google Gemini +GEMINI_API_KEY=your-gemini-api-key +``` + +## That's It! + +You now have a working chat application. The `useChat` hook handles: + +- Message state management +- Streaming responses +- Loading states +- Error handling + +## Using Tools + +Since TanStack AI is framework-agnostic, you can define and use tools in any environment. Here's a quick example of defining a tool and using it in a chat: + +```typescript +import { chat, toolDefinition } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { z } from 'zod' +import { db } from './db' + +const getProductsDef = toolDefinition({ + name: 'getProducts', + description: 'Search the product catalog', + inputSchema: z.object({ query: z.string() }), + outputSchema: z.array(z.object({ id: z.string(), name: z.string() })), +}) + +const getProducts = getProductsDef.server(async ({ query }) => { + return await db.products.search(query) +}) + +const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages: [{ role: 'user', content: 'Find products' }], + tools: [getProducts], +}) +``` + +## Next Steps + +- Learn about [Tools](/ai/tools/tools) to add function calling +- Check out [Client Tools](/ai/tools/client-tools) for frontend operations +- See the [API Reference](/ai/api/ai) for more options diff --git a/mintlify/ai/index.mdx b/mintlify/ai/index.mdx new file mode 100644 index 000000000..47de358a4 --- /dev/null +++ b/mintlify/ai/index.mdx @@ -0,0 +1,21 @@ +--- +title: "TanStack AI" +description: "Type-safe, provider-agnostic AI SDK. Community-maintained Mintlify port of the official TanStack documentation." +--- + +Type-safe, provider-agnostic AI SDK. + + + + Open the first section in this documentation set. + + + Visit the canonical TanStack AI website. + + + View the original source and report upstream issues. + + + Review source commits, licenses, and migration notes. + + diff --git a/mintlify/ai/mcp/apps.md b/mintlify/ai/mcp/apps.md new file mode 100644 index 000000000..80ea623b0 --- /dev/null +++ b/mintlify/ai/mcp/apps.md @@ -0,0 +1,473 @@ +--- +title: MCP Apps +id: mcp-apps +order: 12 +description: "Render interactive ui:// widget resources returned by MCP servers — static display via UIResourcePart and full interactivity via createMcpAppCallHandler and createMcpAppBridge." +keywords: + - tanstack ai + - mcp + - mcp apps + - ui resource + - UIResourcePart + - MCPAppResource + - createMcpAppCallHandler + - createMcpAppBridge + - useMcpAppBridge + - interactive widgets +--- + +**MCP Apps** is a ratified MCP extension (standardized 2026-01-26) that lets MCP servers return interactive `ui://` resource widgets alongside normal tool results. Instead of the model receiving raw JSON, the server embeds a resource URI that TanStack AI fetches and streams to the client as a `UIResourcePart` — ready to render as a full interactive iframe widget. + +There are two levels of MCP Apps support: + +- **Static** — the MCP tool result contains a `ui://` resource. TanStack AI reads it during the `chat()` run and surfaces it as a `UIResourcePart` on the assistant `UIMessage`. No extra routes needed; render it with `MCPAppResource`. +- **Interactive** — the widget's iframe posts tool-call or prompt actions back. You mount a server handler (`createMcpAppCallHandler`) at a route and wire a client bridge (`createMcpAppBridge`) so those actions reach the right MCP server. + +## Static Widgets + +When an MCP tool's result carries a `ui://` resource, TanStack AI emits a `UIResourcePart` on the assistant `UIMessage`. The part is added to the message's `parts` array **alongside** the normal `ToolResultPart` — it never enters model input. + +### The `UIResourcePart` shape + +```ts +import type { UIResourcePart } from '@tanstack/ai' + +// Arrives on the assistant UIMessage alongside ToolCallPart / ToolResultPart: +// { +// type: 'ui-resource' +// resource: { uri: string; mimeType: string; text?: string; blob?: string } +// serverId?: string // pool prefix / config key — routes interactive calls +// toolCallId: string // links to the originating tool call +// toolName: string // MCP tool name whose UI this resource renders +// meta?: Record // reserved — currently always undefined +// } +``` + +No server-side changes are needed beyond connecting an MCP server that returns `ui://` resources. The resource is read eagerly during the chat run. If the read fails (network error, missing resource), the tool result still flows to the model — the widget is simply absent (**fail-soft**). + +### Server route + +```ts ignore +// src/routes/api.chat.ts (TanStack Start) +import { createFileRoute } from '@tanstack/react-router' +import { chat, toServerSentEventsResponse } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { createMCPClient } from '@tanstack/ai-mcp' + +export const Route = createFileRoute('/api/chat')({ + server: { + handlers: { + POST: async ({ request }) => { + const { messages } = await request.json() + + const mcp = await createMCPClient({ + transport: { + type: 'http', + url: process.env.MCP_URL!, + }, + }) + + const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages, + mcp: { clients: [mcp] }, + }) + + return toServerSentEventsResponse(stream) + }, + }, + }, +}) +``` + +### React client — rendering a static widget + +Install the optional peer dependency: + +```bash +pnpm add @mcp-ui/client +``` + +Then render each `ui-resource` part from the assistant message. + +> **What is `sandbox`?** `sandbox.url` points to a small static **sandbox-proxy HTML page that you host** (e.g. `mcp-sandbox.html` on your own origin). `AppRenderer` loads that page in an isolated iframe and renders the widget inside it — it's the security boundary, so it's a **deploy-time constant, the same for every widget**. It is *not* the widget's address: the widget's identity and HTML come from the message part (`part.resource`, a `ui://…` resource). See [`@mcp-ui/client`](https://mcpui.dev) for the proxy page. + +```tsx +// src/components/Chat.tsx +import { useChat } from '@tanstack/ai-react' +import { fetchServerSentEvents } from '@tanstack/ai-client' +import { MCPAppResource } from '@tanstack/ai-react/mcp-apps' +import type { UIResourcePart } from '@tanstack/ai' + +export function Chat() { + const { messages, sendMessage, status } = useChat({ + connection: fetchServerSentEvents('/api/chat'), + }) + + return ( +
+ {messages.map((m) => ( +
+ {m.parts.map((part, i) => { + if (part.type === 'text') { + return

{part.content}

+ } + if (part.type === 'ui-resource') { + return ( + + ) + } + return null + })} +
+ ))} + +
+ ) +} +``` + +`MCPAppResource` is powered by `@mcp-ui/client`'s `AppRenderer` under the hood. Without a `bridge` prop the widget renders in display-only mode — user interactions inside the iframe that trigger tool calls or prompts are ignored. + +> **Framework support:** React and Preact are shipped (`@tanstack/ai-react/mcp-apps` and `@tanstack/ai-preact/mcp-apps`). Preact requires a `preact/compat` alias. Solid, Vue, Svelte, and Angular wrappers are deferred — `@mcp-ui/client` v7's `AppRenderer` is React-only and a framework-agnostic renderer SDK is future work. + +## Interactive Widgets + +For widgets that need to call tools or send prompts back to the model, you wire two extra pieces: + +1. **Server** — mount `createMcpAppCallHandler` at a POST route. The widget's iframe calls this route. +2. **Client** — create a `createMcpAppBridge` and pass it to `MCPAppResource`. The bridge routes the iframe's actions (tool calls, prompts, links) to the correct handler. + +### Installation + +```bash +pnpm add @tanstack/ai-mcp @tanstack/ai-client @mcp-ui/client +``` + +### Server — the call handler route + +`createMcpAppCallHandler` from `@tanstack/ai-mcp/apps` accepts the MCP client(s) you already created (a single `MCPClient`, an `MCPClients` pool, or an array of either) and returns a request handler that: + +- Resolves each client's transport descriptor via `client.getInfo()` / `pool.getServers()` (pure config — no live socket needed). +- Reconnects to the MCP server per call using that descriptor (stateless; serverless-safe by default). +- Checks that the requested `toolName` is actually exposed by that server (same-server allowlist). +- Calls the tool and returns `{ ok: true, result }` or `{ ok: false, error }`. + +```ts ignore +// src/routes/api.mcp-apps-call.ts (TanStack Start) +import { createFileRoute } from '@tanstack/react-router' +import { createMCPClients } from '@tanstack/ai-mcp' +import { createMcpAppCallHandler } from '@tanstack/ai-mcp/apps' + +// Reuse the same pool you pass to chat({ mcp: { clients: [mcp] } }). +const mcp = await createMCPClients({ + weather: { + transport: { + type: 'http', + url: process.env.WEATHER_MCP_URL!, + headers: { Authorization: `Bearer ${process.env.WEATHER_MCP_TOKEN ?? ''}` }, + }, + }, +}) + +// clients: a single MCPClient, an MCPClients pool, or an array of either. +// The handler reads each client's transport descriptor via getInfo()/getServers() +// and reconnects per call — works in long-lived servers and serverless alike. +const handler = createMcpAppCallHandler({ clients: mcp }) + +export const Route = createFileRoute('/api/mcp-apps/call')({ + server: { + handlers: { + POST: async ({ request }) => { + const body = await request.json() + // body: { threadId, serverId, toolName, args?, messageId? } + const result = await handler(body) + return new Response(JSON.stringify(result), { + headers: { 'Content-Type': 'application/json' }, + }) + }, + }, + }, +}) +``` + +> **`link` actions need an `onLink` handler.** If the widget emits a `link` action and no `onLink` handler is wired in the bridge, the bridge drops the link (logging a warning) and `openLink` returns `{ isError: true }` — the call does not hang, and the widget cannot open arbitrary URLs in the host page. Pass `onLink` explicitly to opt in. +> +> Even with an `onLink` handler, the bridge only forwards `http:`, `https:`, and `mailto:` URLs. Unsafe schemes (`javascript:`, `data:`, `file:`, …) are **always rejected** before your handler runs, so a sandboxed widget can't smuggle a script-executing or local-resource URL through. + +#### Same-server allowlist + +`createMcpAppCallHandler` always verifies that `toolName` is in the list of tools the target server actually exposes. A request for a tool the server does not know about returns `{ ok: false, error: "Tool not allowed: " }` without ever executing it. This server-exposure check is unconditional and cannot be bypassed. + +Use the `allowTool` option to add a further restriction on top. A request must satisfy **both** the server-exposure check and `allowTool` — it is AND-ed, not a replacement for the server check: + +```ts +import { createMCPClients } from '@tanstack/ai-mcp' +import { createMcpAppCallHandler } from '@tanstack/ai-mcp/apps' + +const mcp = await createMCPClients({ + weather: { transport: { type: 'http', url: process.env.MCP_URL ?? '' } }, +}) + +const handler = createMcpAppCallHandler({ + clients: mcp, + // Additional restriction: even if the server exposes more tools, + // only allow this specific one through the call handler. + allowTool: (req) => req.toolName === 'place_order', +}) +``` + +### Chat route — wire the `serverId` + +The `serverId` on a `UIResourcePart` comes from the `prefix` you gave the MCP client. Use the same key in both places: + +> **Multi-server routing:** interactive calls route by `serverId`, which is each client's `prefix`. `createMCPClients` defaults every server's prefix to its config key, so routing works out of the box. If you pass multiple servers and disable prefixing on one (`prefix: ''`), that server has no `serverId` and its widgets can't make interactive calls — give each interactive server a distinct prefix (the default is fine). + + +```ts ignore +// src/routes/api.chat.ts +import { createFileRoute } from '@tanstack/react-router' +import { chat, toServerSentEventsResponse } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { createMCPClients } from '@tanstack/ai-mcp' + +export const Route = createFileRoute('/api/chat')({ + server: { + handlers: { + POST: async ({ request }) => { + const body = await request.json() + + // The pool key "weather" becomes the serverId on every UIResourcePart + // emitted by this server — must match the key used when constructing + // the pool passed to createMcpAppCallHandler. + const pool = await createMCPClients({ + weather: { + transport: { type: 'http', url: process.env.WEATHER_MCP_URL! }, + }, + }) + + const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages: body.messages, + mcp: { clients: [pool] }, + }) + + return toServerSentEventsResponse(stream) + }, + }, + }, +}) +``` + +### Client — bridge + interactive render + +`createMcpAppBridge` from `@tanstack/ai-client` returns an action handler you pass to `MCPAppResource`. It routes: + +- `tool` actions → POST to `callEndpoint` with the tool call payload. +- `prompt` actions → `chat.sendMessage(prompt)`. +- `link` actions → `onLink(url)` if provided; dropped (with a warning) otherwise. + +In React (or Preact), use the `useMcpAppBridge` hook — it returns a **stable** +bridge for the given `threadId`/`callEndpoint` and always calls your latest +`sendMessage`/`onLink`, so you don't hand-write `useMemo` or fight +`exhaustive-deps`. (The underlying `createMcpAppBridge` from +`@tanstack/ai-client` is framework-agnostic if you need it directly.) + +```tsx +// src/components/Chat.tsx +import { useChat, useMcpAppBridge } from '@tanstack/ai-react' +import { fetchServerSentEvents } from '@tanstack/ai-client' +import { MCPAppResource } from '@tanstack/ai-react/mcp-apps' + +export function Chat() { + // A stable id correlating widget calls back to this conversation. + const threadId = 'weather-chat' + const { messages, sendMessage, status } = useChat({ + connection: fetchServerSentEvents('/api/chat'), + }) + + const bridge = useMcpAppBridge({ + threadId, + callEndpoint: '/api/mcp-apps/call', + chat: { sendMessage: async (content) => void sendMessage({ content }) }, + // Opt in to link navigation — absent means links are blocked. + onLink: (url) => window.open(url, '_blank', 'noopener'), + }) + + return ( +
+ {messages.map((m) => ( +
+ {m.parts.map((part, i) => { + if (part.type === 'text') { + return

{part.content}

+ } + if (part.type === 'ui-resource') { + return ( + + ) + } + return null + })} +
+ ))} + +
+ ) +} +``` + +> **Writeback is client-side.** Widget tool calls do **not** append to the thread's chat history by default. The conversation state writeback path is out of scope for the current release. Each widget interaction is self-contained. + +## Session Persistence + +The call handler reconnects to the MCP server on every widget action using the transport descriptor it reads from `client.getInfo()` / `pool.getServers()` (**reconnect-per-call** — stateless, serverless-safe). For stateful MCP transports that require a persistent session, opt in to an in-memory session store: + +```ts +import { createMCPClients } from '@tanstack/ai-mcp' +import { + createMcpAppCallHandler, + inMemoryMcpSessionStore, +} from '@tanstack/ai-mcp/apps' + +const mcp = await createMCPClients({ + weather: { transport: { type: 'http', url: process.env.MCP_URL ?? '' } }, +}) + +// In-memory store: one Node.js process, no cross-instance sharing. +// Shape matches the McpSessionStore interface — SQL / KV stores +// can be dropped in later with no API change. +const store = inMemoryMcpSessionStore({ ttlMs: 30 * 60_000 }) + +const handler = createMcpAppCallHandler({ clients: mcp, store }) +``` + +> **Current limitation:** `inMemoryMcpSessionStore` is single-instance (one Node.js process). It does not survive serverless restarts or scale across replicas. The `McpSessionStore` interface is the persistence extension point — persistent backends (database, KV store) can be dropped in without any API changes. + +## API Reference + +### `createMcpAppCallHandler` (`@tanstack/ai-mcp/apps`) + +```ts +import { createMCPClients } from '@tanstack/ai-mcp' +import { createMcpAppCallHandler } from '@tanstack/ai-mcp/apps' +import type { McpAppCallHandlerOptions } from '@tanstack/ai-mcp/apps' + +const mcp = await createMCPClients({ + weather: { transport: { type: 'http', url: process.env.MCP_URL ?? '' } }, +}) + +const options: McpAppCallHandlerOptions = { + // Pass the MCP client(s) you already created: + // - a single MCPClient + // - an MCPClients pool (pool key = serverId on UIResourcePart) + // - an array of either + // The handler reads each client's transport descriptor via + // client.getInfo() / pool.getServers() (pure config, no live socket) + // and reconnects per call — serverless-safe by default. + clients: mcp, + + // Dynamic session store (opt-in for stateful transports) + // store: inMemoryMcpSessionStore(), + + // Custom tool allowlist — default: server's own exposed tools only + // AND-ed on top of the always-on same-server exposure check. + allowTool: (req) => req.toolName === 'get_weather', +} + +// Returns: (req) => Promise<{ ok: true; result: unknown } | { ok: false; error: string }> +const handler = createMcpAppCallHandler(options) +``` + +### `inMemoryMcpSessionStore` (`@tanstack/ai-mcp/apps`) + +```ts +import { inMemoryMcpSessionStore } from '@tanstack/ai-mcp/apps' + +const store = inMemoryMcpSessionStore({ + ttlMs: 30 * 60_000, // optional; default: 30 minutes +}) +``` + +### `createMcpAppBridge` (`@tanstack/ai-client`) + +```ts +import { createMcpAppBridge } from '@tanstack/ai-client' +import type { CreateMcpAppBridgeOptions } from '@tanstack/ai-client' + +const options: CreateMcpAppBridgeOptions = { + threadId: 'weather-chat', // identifies the thread for the call handler + callEndpoint: '/api/mcp-apps/call', // POST route mounting createMcpAppCallHandler + chat: { sendMessage: async (text) => console.log(text) }, // prompt-intent path + fetchImpl: fetch, // optional; injectable for testing + onLink: (url) => window.open(url, '_blank'), // absent → link is dropped (warned), openLink returns { isError: true } +} + +// Returns an McpAppBridge with callTool / sendPrompt / openLink methods. +const bridge = createMcpAppBridge(options) +``` + +### `useMcpAppBridge` (`@tanstack/ai-react` / `@tanstack/ai-preact`) + +The React/Preact wrapper around `createMcpAppBridge`. Returns a bridge that is +**stable** for a given `threadId`/`callEndpoint` (so it won't churn `MCPAppResource` +on every render) while always invoking the latest `chat.sendMessage`/`onLink`. +Takes the same options as `createMcpAppBridge`. + +```tsx +import { useChat, useMcpAppBridge } from '@tanstack/ai-react' +import { fetchServerSentEvents } from '@tanstack/ai-client' + +function useBridge(threadId: string) { + const { sendMessage } = useChat({ + connection: fetchServerSentEvents('/api/chat'), + }) + return useMcpAppBridge({ + threadId, + callEndpoint: '/api/mcp-apps/call', + chat: { sendMessage: async (content) => void sendMessage({ content }) }, + onLink: (url) => window.open(url, '_blank', 'noopener'), + }) +} +``` + +### `MCPAppResource` (`@tanstack/ai-react/mcp-apps`) + +```tsx +import { MCPAppResource } from '@tanstack/ai-react/mcp-apps' +// `part` is a UIResourcePart from the assistant message; `bridge` is a +// createMcpAppBridge result — both supplied by your component (see examples above). +import { part, bridge } from './chat-context' + +const widget = ( + +) +``` + +Preact: identical API from `@tanstack/ai-preact/mcp-apps` (requires `preact/compat` alias). diff --git a/mintlify/ai/media/audio-generation.md b/mintlify/ai/media/audio-generation.md new file mode 100644 index 000000000..d49b8e7c5 --- /dev/null +++ b/mintlify/ai/media/audio-generation.md @@ -0,0 +1,245 @@ +--- +title: Audio Generation +id: audio-generation +order: 15 +--- + +# Audio Generation + +TanStack AI's `generateAudio()` activity produces audio content — music, soundscapes, or sound effects — from a text prompt. It's distinct from [Text-to-Speech](/ai/media/text-to-speech), which is optimized for spoken-word synthesis. + +## Overview + +Audio generation is handled by audio adapters that follow the same tree-shakeable architecture as other adapters in TanStack AI. + +Currently supported: + +- **Google Gemini**: Lyria 3 Pro and Lyria 3 Clip music generation +- **fal.ai**: MiniMax Music, DiffRhythm, Google Lyria 2, Stable Audio 2.5, MMAudio, ElevenLabs sound effects, Thinksound, and more + +## Basic Usage + +### Google Lyria (Music) + +Google's Lyria models generate full-length songs with vocals and instrumentation. `lyria-3-pro-preview` handles multi-verse compositions, while `lyria-3-clip-preview` produces 30-second clips. + +```typescript +import { generateAudio } from '@tanstack/ai' +import { geminiAudio } from '@tanstack/ai-gemini' + +const result = await generateAudio({ + adapter: geminiAudio('lyria-3-pro-preview'), + prompt: 'Uplifting indie pop with layered vocals and jangly guitars', +}) + +console.log(result.audio.b64Json) // Base64-encoded audio bytes (Gemini) +console.log(result.audio.contentType) // e.g. "audio/mpeg" +``` + +### fal.ai + +fal.ai gives access to a broad catalogue of music, SFX, and general audio models through a single `falAudio` adapter. + +#### Music Generation (MiniMax Music 2.6) + +MiniMax's latest music model creates full compositions — vocals, backing music, and arrangements — from a single prompt. + +```typescript +import { generateAudio } from '@tanstack/ai' +import { falAudio } from '@tanstack/ai-fal' + +const result = await generateAudio({ + adapter: falAudio('fal-ai/minimax-music/v2.6'), + prompt: 'City Pop, 80s retro, groovy synth bass, warm female vocal, 104 BPM', +}) + +console.log(result.audio.url) // URL to the generated audio file +console.log(result.audio.contentType) // e.g. "audio/wav" +``` + +#### Music with Explicit Lyrics (DiffRhythm) + +```typescript +import { generateAudio } from '@tanstack/ai' +import { falAudio } from '@tanstack/ai-fal' + +const result = await generateAudio({ + adapter: falAudio('fal-ai/diffrhythm'), + prompt: 'An upbeat electronic track with synths', + modelOptions: { + lyrics: '[verse]\nHello world\n[chorus]\nLa la la', + }, +}) +``` + +#### Sound Effects + +```typescript +import { generateAudio } from '@tanstack/ai' +import { falAudio } from '@tanstack/ai-fal' + +const result = await generateAudio({ + adapter: falAudio('fal-ai/elevenlabs/sound-effects/v2'), + prompt: 'Thunderclap followed by heavy rain', + duration: 5, +}) +``` + +#### MiniMax Music v2 (lyrics_prompt) + +Earlier MiniMax variants use a `lyrics_prompt` field for lyric guidance. + +```typescript +import { generateAudio } from '@tanstack/ai' +import { falAudio } from '@tanstack/ai-fal' + +const result = await generateAudio({ + adapter: falAudio('fal-ai/minimax-music/v2'), + prompt: 'A dreamy pop ballad in the style of the 80s', + modelOptions: { + lyrics_prompt: '[instrumental]', + }, +}) +``` + +If a request doesn't return the audio you expected — a model silently truncates, a provider rejects a prompt, or the response shape looks off — pass `debug: true` to see every chunk the provider SDK emits. See [Debug Logging](/ai/advanced/debug-logging). + +## Options + +| Option | Type | Description | +|--------|------|-------------| +| `adapter` | `AudioAdapter` | The adapter created via `falAudio()` (required) | +| `prompt` | `string` | Text description of the audio to generate (required) | +| `duration` | `number` | Desired duration in seconds (model-dependent) | +| `modelOptions` | `object` | Provider-specific options (fully typed when the model ID is passed as a string literal) | +| `debug` | `DebugOption` | Enable per-category debug logging (`true`, `false`, or a `DebugConfig` — see [Debug Logging](/ai/advanced/debug-logging)) | + +## Result Shape + +```typescript +import type { TokenUsage } from '@tanstack/ai' + +interface AudioGenerationResult { + id: string + model: string + audio: { + url?: string + b64Json?: string + contentType?: string + duration?: number + } + // Canonical TokenUsage (same shape as chat), present when the provider + // reports it (e.g. Gemini Lyria via generateContent). Usage-billed providers + // (fal) instead surface `usage.unitsBilled` — the real billed quantity read + // from fal's `x-fal-billable-units` result header. Multiply by the endpoint's + // unit price (fal pricing API) for the exact cost. + usage?: TokenUsage +} +``` + +Gemini returns base64-encoded bytes in `result.audio.b64Json`. The fal adapter returns a URL in `result.audio.url` — if you need raw bytes, `fetch()` the URL yourself: + +```typescript +import { generateAudio } from '@tanstack/ai' +import { falAudio } from '@tanstack/ai-fal' + +const result = await generateAudio({ + adapter: falAudio('fal-ai/diffrhythm'), + prompt: 'An upbeat electronic track', +}) + +const bytes = new Uint8Array( + await (await fetch(result.audio.url!)).arrayBuffer() +) +``` + +## Client Hook (`useGenerateAudio`) + +For client-side usage, framework integrations expose a `useGenerateAudio` +hook (or `createGenerateAudio` in Svelte) that wraps the same generation +flow. It mirrors the API of `useGenerateSpeech`, `useGenerateImage`, and +other media hooks — see [Generation Hooks](/ai/media/generation-hooks) for the full +shape. + +### Server (streaming SSE route) + +```typescript +// routes/api/generate/audio.ts +import { generateAudio, toServerSentEventsResponse } from '@tanstack/ai' +import { falAudio } from '@tanstack/ai-fal' + +export async function POST(req: Request) { + const { prompt, duration } = await req.json() + + return toServerSentEventsResponse( + generateAudio({ + adapter: falAudio('fal-ai/diffrhythm'), + prompt, + duration, + stream: true, + }), + ) +} +``` + +### Client (React) + +```tsx +import { useGenerateAudio } from '@tanstack/ai-react' +import { fetchServerSentEvents } from '@tanstack/ai-client' + +function AudioGenerator() { + const { generate, result, isLoading, error, reset } = useGenerateAudio({ + connection: fetchServerSentEvents('/api/generate/audio'), + }) + + return ( +
+ + {error &&

Error: {error.message}

} + {result?.audio.url &&
+ ) +} +``` + +Use the `fetcher` option instead of `connection` when calling a TanStack +Start server function directly. + +## Differences vs Text-to-Speech + +| | `generateAudio()` | `generateSpeech()` | +|---|---|---| +| Purpose | Music, soundscapes, SFX | Spoken-word TTS | +| Result | `result.audio.url` or `result.audio.b64Json` | Base64 in `result.audio` | +| Primary input | `prompt` | `text` | +| Voice/speed controls | No | Yes (`voice`, `speed`) | + +Use `generateSpeech()` when you want a spoken voice, and `generateAudio()` when you want non-speech audio. + +## Environment Variables + +Each provider reads its own API key from the environment by default: + +```bash +GOOGLE_API_KEY=your-google-api-key +FAL_KEY=your-fal-api-key +``` + +Or pass it explicitly to the adapter: + +```typescript +import { createGeminiAudio } from '@tanstack/ai-gemini' +import { falAudio } from '@tanstack/ai-fal' + +createGeminiAudio('lyria-3-pro-preview', 'your-key') +falAudio('fal-ai/diffrhythm', { apiKey: 'your-key' }) +``` diff --git a/mintlify/ai/media/audio-recording.md b/mintlify/ai/media/audio-recording.md new file mode 100644 index 000000000..0836b40f2 --- /dev/null +++ b/mintlify/ai/media/audio-recording.md @@ -0,0 +1,271 @@ +--- +title: Audio Recording +id: audio-recording +description: "Record microphone audio in the browser with useAudioRecorder and send it to a chat or transcription as a ready-to-use content part, with an optional transform." +keywords: + - tanstack ai + - audio recording + - useAudioRecorder + - createAudioRecorder + - injectAudioRecorder + - voice input + - MediaRecorder +--- + +# Audio Recording + +You have a chat or generation UI and you want users to talk instead of type. By +the end of this guide you'll capture microphone audio in the browser with +`useAudioRecorder`, read the latest recording reactively, and send it straight +into a chat message or a transcription request — with no transcoding and no +extra dependencies. + +`useAudioRecorder` wraps the browser's `getUserMedia` / `MediaRecorder` and +returns the recorder's native output (`audio/webm` or `audio/mp4`). + +## Record audio + +Start with a button; end with a working recorder that toggles capture and hands +you the result. + +```tsx group=audio-recording +import { useAudioRecorder } from '@tanstack/ai-react' + +function RecordButton() { + const { isRecording, isSupported, start, stop } = useAudioRecorder({ + onError: (error) => console.error(error), + }) + + if (!isSupported) return

Recording is not supported in this browser.

+ + return ( + + ) +} +``` + +`stop()` resolves to an `AudioRecording`: + +| Field | Type | Description | +| ------------ | ----------- | ---------------------------------------------------------------------------- | +| `part` | `AudioPart` | Ready-to-use content part: `{ type: 'audio', source: { type: 'data', value, mimeType } }` | +| `base64` | `string` | Raw base64 of the recorded bytes | +| `blob` | `Blob` | The raw recorded blob | +| `mimeType` | `string` | Native recorder type, e.g. `audio/webm;codecs=opus` | +| `durationMs` | `number` | Recording length in milliseconds | + +## Handling errors + +Failures reach you through **two** channels — pick one, don't handle both: + +- `onError(error)` fires for permission denial and recorder errors. +- `start()` and `stop()` also **reject**. `start()` rejects on permission + denial; `stop()` rejects on a recorder error or with `Recording cancelled` if + the recording is cancelled while a stop is in flight (for example when the + component unmounts mid-recording). + +So if you `await start()` / `await stop()`, wrap them in `try`/`catch` rather +than discarding the promise with `void`. The recorder's native `mimeType` may +differ from a requested `mimeType` (browsers ignore unsupported types), so read +`recording.mimeType` if a downstream step requires a specific format. + +## Read the latest recording reactively + +The same value is also exposed as the reactive `recording` field, so you can +render a preview without capturing `stop()`'s return value yourself. It's `null` +until the first `stop()`: + +```tsx group=audio-recording +function Preview() { + const { recording, isRecording, start, stop } = useAudioRecorder() + // recording is AudioRecording | null +} +``` + +> Across frameworks `recording` follows the same shape as the other reactive +> fields: an accessor in Solid (`recording()`), a readonly ref in Vue +> (`recording.value`), a getter in Svelte (`recorder.recording`), and a +> `Signal` in Angular (`recording()`). + +## Transform the recording + +Pass `onComplete` to turn the raw recording into whatever your app needs — a URL +after upload, an encoded blob, or a custom object. Both `stop()` and the +reactive `recording` field then resolve to your transformed value, and the +transform can be `async`: + +```tsx group=audio-recording +function Uploader() { + const { recording, stop } = useAudioRecorder({ + onComplete: async (rec) => { + const res = await fetch('/api/upload', { method: 'POST', body: rec.blob }) + const { url } = await res.json() + return url // `recording` and `stop()` now resolve to string + }, + }) +} +``` + +Return nothing (`undefined`) to keep the raw `AudioRecording`; any returned +value — including `null` — is used as-is and re-types `stop()` and `recording`. +This is similar to the `onResult` transform on the +[generation hooks](/ai/media/generation-hooks), but is async-capable. (Unlike +`onResult`, where `null` means "keep the previous value," only `undefined` keeps +the raw recording here.) + +## Send a recording in chat + +The recording's `part` is already a chat content part, so it drops straight into +`sendMessage`: + +```tsx +import { + useAudioRecorder, + useChat, + fetchServerSentEvents, +} from '@tanstack/ai-react' + +function VoiceComposer() { + const { isRecording, start, stop } = useAudioRecorder() + const { sendMessage } = useChat({ + connection: fetchServerSentEvents('/api/chat'), + }) + + const toggle = async () => { + try { + if (!isRecording) { + await start() + return + } + const rec = await stop() + await sendMessage({ content: [rec.part] }) + } catch (error) { + // start()/stop() reject on permission denial, recorder error, or cancel. + console.error(error) + } + } + + return ( + + ) +} +``` + +## Transcribe a recording + +Wrap the recording as a `data:` URL so the provider receives the recorder's +native content type — passing raw `base64` makes the transcription adapter +assume `audio/mpeg` and mislabel the webm/mp4 bytes. See +[Transcription](/ai/media/transcription) for the matching server route. + +```tsx +import { + useAudioRecorder, + useTranscription, + fetchServerSentEvents, +} from '@tanstack/ai-react' + +function Transcriber() { + const { isRecording, start, stop } = useAudioRecorder() + const { generate, result } = useTranscription({ + connection: fetchServerSentEvents('/api/transcribe'), + }) + + const toggle = async () => { + try { + if (!isRecording) { + await start() + return + } + const rec = await stop() + // Wrap as a data URL so the provider gets the recorder's real content + // type. Passing raw base64 makes the transcription adapter assume + // `audio/mpeg`, which mislabels the native webm/mp4 bytes. Strip the + // `;codecs=...` parameter for a clean type. + const mimeType = rec.mimeType.split(';')[0] + await generate({ audio: `data:${mimeType};base64,${rec.base64}` }) + } catch (error) { + console.error(error) + } + } + + return ( +
+ + {result ?

{result.text}

: null} +
+ ) +} +``` + +## Other frameworks + +The same recorder ships for every framework with idiomatic reactivity. Svelte +uses the `createAudioRecorder` factory; because Svelte 5 runes can't register +automatic teardown, call `cancel()` from your component cleanup if a recording +may still be active: + +```svelte + + + +``` + +| Framework | Import | Function | Reactive fields | +| --------- | ----------------------- | -------------------- | ---------------------------------------------------- | +| React | `@tanstack/ai-react` | `useAudioRecorder` | `isRecording`, `recording` (values) | +| Solid | `@tanstack/ai-solid` | `useAudioRecorder` | `isRecording()`, `recording()` (accessors) | +| Vue | `@tanstack/ai-vue` | `useAudioRecorder` | `isRecording.value`, `recording.value` (readonly refs) | +| Svelte | `@tanstack/ai-svelte` | `createAudioRecorder`| `recorder.isRecording`, `recorder.recording` (getters) | +| Angular | `@tanstack/ai-angular` | `injectAudioRecorder`| `isRecording()`, `recording()` (signals; call in an injection context) | + +## Hook API + +`useAudioRecorder(options?)` — and the `createAudioRecorder` / +`injectAudioRecorder` equivalents — accept: + +| Option | Type | Description | +| ------------ | --------------------------------------------------- | --------------------------------------------------------------------------------------------------- | +| `onComplete` | `(recording: AudioRecording) => T \| Promise` | Optional transform. Its (awaited) return re-types `stop()` and `recording`. Return nothing to keep the raw recording | +| `onError` | `(error: Error) => void` | Called on permission denial or recorder error | +| `audio` | `MediaTrackConstraints \| boolean` | Passed to `getUserMedia({ audio })`. Defaults to `true` | +| `mimeType` | `string` | Preferred recorder mime type; falls back to the browser default if unsupported | + +And return: + +| Property | Type | Description | +| ------------- | --------------------- | ------------------------------------------------------------------- | +| `recording` | `T \| null` | Latest recording (transformed if `onComplete` provided), reactive | +| `isRecording` | `boolean` | Whether capture is currently active | +| `isSupported` | `boolean` | Whether the browser supports recording | +| `start` | `() => Promise` | Acquire the mic and begin recording | +| `stop` | `() => Promise` | Stop, and resolve with the recording (transformed if applicable) | +| `cancel` | `() => void` | Discard the in-progress recording and release the mic | + +> Reactive shapes (`recording`, `isRecording`) vary per framework — see the +> table in [Other frameworks](#other-frameworks). `T` is `AudioRecording` +> unless an `onComplete` transform changes it. diff --git a/mintlify/ai/media/generation-hooks.md b/mintlify/ai/media/generation-hooks.md new file mode 100644 index 000000000..860656fe9 --- /dev/null +++ b/mintlify/ai/media/generation-hooks.md @@ -0,0 +1,453 @@ +--- +title: Generation Hooks +id: generation-hooks +order: 7 +description: "Framework hooks for every TanStack AI media generation type — useGenerateImage, useGenerateAudio, useGenerateSpeech, useTranscription, useSummarize, useGenerateVideo." +keywords: + - tanstack ai + - generation hooks + - useGenerateImage + - useGenerateAudio + - useGenerateSpeech + - useTranscription + - useSummarize + - useGenerateVideo + - react hooks +--- + +# Generation Hooks + +TanStack AI provides framework hooks for every generation type: image, audio, speech, transcription, summarization, and video. Each hook connects to a server endpoint and manages loading, error, and result state for you. + +## Overview + +Generation hooks share a consistent API across all media types: + +| Hook | Input | Result Type | +|------|-------|-------------| +| `useGenerateImage` | `ImageGenerateInput` | `ImageGenerationResult` | +| `useGenerateAudio` | `AudioGenerateInput` | `AudioGenerationResult` | +| `useGenerateSpeech` | `SpeechGenerateInput` | `TTSResult` | +| `useTranscription` | `TranscriptionGenerateInput` | `TranscriptionResult` | +| `useSummarize` | `SummarizeGenerateInput` | `SummarizationResult` | +| `useGenerateVideo` | `VideoGenerateInput` | `VideoGenerateResult` | +| `useGeneration` | Generic `TInput` | Generic `TResult` | + +Every hook returns the same core shape: `generate`, `result`, `isLoading`, `error`, `status`, `stop`, and `reset`. You provide either a `connection` (streaming transport) or a `fetcher` (direct async call). + +## Server Setup + +Before using hooks on the client, you need a server endpoint that runs the generation and returns the result as SSE. Here's a minimal image generation endpoint: + +```typescript +// routes/api/generate/image.ts +import { generateImage, toServerSentEventsResponse } from '@tanstack/ai' +import { openaiImage } from '@tanstack/ai-openai' + +export async function POST(req: Request) { + const { prompt, size, numberOfImages } = await req.json() + + const stream = generateImage({ + adapter: openaiImage('dall-e-3'), + prompt, + size, + numberOfImages, + stream: true, + }) + + return toServerSentEventsResponse(stream) +} +``` + +The same pattern applies to all generation types -- swap `generateImage` for `generateSpeech`, `generateTranscription`, `summarize`, or `generateVideo`. See the individual media guides for server-side details. + +## useGenerateImage + +Trigger image generation and render the results. + +```tsx +import { useGenerateImage, fetchServerSentEvents } from '@tanstack/ai-react' +import { useState } from 'react' + +function ImageGenerator() { + const [prompt, setPrompt] = useState('') + const { generate, result, isLoading, error, reset } = useGenerateImage({ + connection: fetchServerSentEvents('/api/generate/image'), + }) + + return ( +
+ setPrompt(e.target.value)} + placeholder="Describe an image..." + /> + + + {error &&

Error: {error.message}

} + + {result?.images.map((img, i) => ( + {img.revisedPrompt + ))} + + {result && } +
+ ) +} +``` + +The `generate` function accepts an `ImageGenerateInput`: + +| Field | Type | Description | +|-------|------|-------------| +| `prompt` | `string` | Text description of the desired image (required) | +| `numberOfImages` | `number` | Number of images to generate | +| `size` | `string` | Image size in WIDTHxHEIGHT format (e.g., `"1024x1024"`) | +| `modelOptions` | `Record` | Model-specific options | + +## useGenerateSpeech + +Convert text to speech and play it back. + +```tsx +import { useGenerateSpeech, fetchServerSentEvents } from '@tanstack/ai-react' +import { useRef } from 'react' + +function SpeechGenerator() { + const audioRef = useRef(null) + const { generate, result, isLoading, error } = useGenerateSpeech({ + connection: fetchServerSentEvents('/api/generate/speech'), + }) + + return ( +
+ + + {error &&

Error: {error.message}

} + + {result && ( +
+ ) +} +``` + +The `generate` function accepts a `SpeechGenerateInput`: + +| Field | Type | Description | +|-------|------|-------------| +| `text` | `string` | The text to convert to speech (required) | +| `voice` | `string` | The voice to use (e.g., `"alloy"`, `"echo"`) | +| `format` | `'mp3' \| 'opus' \| 'aac' \| 'flac' \| 'wav' \| 'pcm'` | Output audio format | +| `speed` | `number` | Audio speed (0.25 to 4.0) | +| `modelOptions` | `Record` | Model-specific options | + +The `TTSResult` contains `audio` (base64-encoded), `format`, and optionally `duration` and `contentType`. + +## useTranscription + +Transcribe audio files to text. + +```tsx +import { useTranscription, fetchServerSentEvents } from '@tanstack/ai-react' + +function Transcriber() { + const { generate, result, isLoading, error } = useTranscription({ + connection: fetchServerSentEvents('/api/transcribe'), + }) + + const handleFile = (e: React.ChangeEvent) => { + const file = e.target.files?.[0] + if (file) { + const reader = new FileReader() + reader.onload = () => { + generate({ audio: reader.result as string, language: 'en' }) + } + reader.readAsDataURL(file) + } + } + + return ( +
+ + + {isLoading &&

Transcribing...

} + {error &&

Error: {error.message}

} + + {result && ( +
+

Transcription

+

{result.text}

+ {result.language &&

Language: {result.language}

} + {result.duration &&

Duration: {result.duration}s

} +
+ )} +
+ ) +} +``` + +The `generate` function accepts a `TranscriptionGenerateInput`: + +| Field | Type | Description | +|-------|------|-------------| +| `audio` | `string \| File \| Blob \| ArrayBuffer` | Audio data -- base64 string, File, Blob, or ArrayBuffer (required) | +| `language` | `string` | Language in ISO-639-1 format (e.g., `"en"`) | +| `prompt` | `string` | Optional prompt to guide the transcription | +| `responseFormat` | `'json' \| 'text' \| 'srt' \| 'verbose_json' \| 'vtt'` | Common output format | +| `modelOptions` | `Record` | Model-specific options | + +## useSummarize + +Summarize long text with configurable output styles. + +```tsx +import { useSummarize, fetchServerSentEvents } from '@tanstack/ai-react' +import { useState } from 'react' + +function Summarizer() { + const [text, setText] = useState('') + const { generate, result, isLoading, error } = useSummarize({ + connection: fetchServerSentEvents('/api/summarize'), + }) + + return ( +
+