feat(providers): add Sakana AI provider with Fugu models by waleedlatif1 · Pull Request #5169 · simstudioai/sim

waleedlatif1 · 2026-06-22T19:47:57Z

Summary

Add Sakana AI as a new LLM provider (OpenAI-compatible API at https://api.sakana.ai/v1, bearer auth)
Register two models: fugu (fast default) and fugu-ultra (reasoning flagship), both 1M context window
Full provider wiring: ProviderId, registry, PROVIDER_DEFINITIONS, providers metadata, SakanaIcon, tokenization config, attachment adapter
Provider executor supports streaming, the tool-execution loop, forced tool choice, and response_format — mirrors the deepseek OpenAI-compatible pattern
BYOK-only: not in getHostedModels(), so it is never served with Sim's hosted key or auto-billed (matches groq/cerebras/deepseek/xai)

Notes on pricing

fugu-ultra pricing ($5 input / $30 output / $0.50 cached per 1M) is taken directly from Sakana's pricing page (standard <272K tier)
Sakana does not publish a fixed per-token rate for the base fugu router ("a single rate based on the top-tier model involved"), so fugu is priced at the documented fugu-ultra ceiling so cost tracking never under-reports

Type of Change

New feature (provider integration)

Testing

bun run lint, typecheck, and bun run check:api-validation pass
Added provider definition tests (models.test.ts): model ids, 1M context, pricing, and fugu/fugu-ultra → sakana routing — 32 provider tests pass
Verified the API key authenticates against https://api.sakana.ai/v1 (live /v1/models returns fugu/fugu-ultra); a full chat completion is pending an account with active credit

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2026-06-22T19:48:03Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Jun 22, 2026 8:34pm

cursor · 2026-06-22T19:48:06Z

PR Summary

Medium Risk
New external inference path with tool/streaming/structured-output logic increases integration surface; Sakana is excluded from hosted models so billing stays BYOK-only.

Overview
Adds Sakana AI as a BYOK LLM provider targeting https://api.sakana.ai/v1, with fugu and fugu-ultra (1M context, /^fugu/ routing) registered in PROVIDER_DEFINITIONS, ProviderId, the provider registry, provider metadata, SakanaIcon, tokenization, and attachments (no file support, same as deepseek/cerebras).

The new sakanaProvider executor uses the OpenAI SDK against Sakana’s API: streaming, tool loops with forced tool choice, deferred response_format when tools are active, and cost/timing tracing—patterned after other OpenAI-compatible providers.

fugu pricing uses the documented fugu-ultra ceiling so usage cost is not under-reported. Tests in models.test.ts cover model registration and routing.

Separately, session-provider.test.tsx flush() now yields a macrotask tick so React Query observer updates settle reliably in tests.

^{Reviewed by Cursor Bugbot for commit ee72a69. Configure here.}

greptile-apps · 2026-06-22T19:53:52Z

Greptile Summary

This PR integrates Sakana AI as a new BYOK-only LLM provider, wiring up the fugu and fugu-ultra models end-to-end across the provider stack. The implementation closely follows the existing OpenAI-compatible provider pattern (deepseek/groq/cerebras) and touches all the required registration points consistently.

Provider executor (sakana/index.ts): Full implementation of streaming, the tool-execution loop, forced-tool-choice tracking, and deferred response_format — logic is structurally identical to peer providers and all edge cases (max iterations, tool message ordering, cost accumulation) are handled correctly.
Model definitions (models.ts): Two models registered with 1 M context windows; fugu priced at the fugu-ultra ceiling as documented in the PR description to avoid cost under-reporting.
Attachment/tokenization/registry wiring: sakana is added to every required data structure (UNSUPPORTED_FILE_PROVIDERS, ProviderId, provider registry, metadata map, tokenization config) with no gaps found.

Confidence Score: 5/5

Safe to merge — all registration points are consistent, the executor follows the established OpenAI-compatible pattern without deviation, and the BYOK-only constraint is correctly enforced.

The change is a self-contained provider addition that mirrors the deepseek/groq/cerebras integration pattern. Every required wiring point (ProviderId, registry, model definitions, tokenization, attachment adapter, metadata map, icon) has been updated in lockstep with no gaps. The tool-execution loop, streaming paths, forced-tool-choice tracking, and cost accumulation are structurally identical to peer providers. No regressions were identified in the existing test suite or the newly added provider tests.

No files require special attention.

Important Files Changed

Filename	Overview
apps/sim/providers/sakana/index.ts	New provider executor for Sakana AI — implements streaming, tool loop, forced-tool-choice, and deferred response_format following the existing OpenAI-compatible pattern; logic is correct and consistent with peer providers.
apps/sim/providers/models.ts	Adds sakana PROVIDER_DEFINITION with fugu and fugu-ultra models; pricing, context window, release date, and icon are all present and consistent.
apps/sim/providers/models.test.ts	Adds four focused tests covering registration, context window, pricing, and model-routing for the Sakana provider; all pass per the PR description.
apps/sim/providers/attachments.ts	Adds sakana to UNSUPPORTED_FILE_PROVIDERS, the AttachmentProvider type, PROVIDER_SUPPORTED_LABELS, getAttachmentProvider(), and isMimeTypeSupportedByProvider() — exhaustive and consistent with cerebras/deepseek precedent.
apps/sim/components/icons.tsx	SakanaIcon added as an inline SVG component matching Sakana's fish-logo path; no issues.
apps/sim/providers/sakana/utils.ts	Thin wrapper around the shared createOpenAICompatibleStream utility — no logic, no issues.
apps/sim/providers/types.ts	ProviderId union extended with 'sakana'.
apps/sim/providers/registry.ts	sakanaProvider imported and registered in the provider registry.
apps/sim/providers/utils.ts	sakana metadata entry added via buildProviderMetadata — no issues.
apps/sim/lib/tokenization/constants.ts	sakana tokenization config added with medium-confidence heuristic — matches the groq/cerebras/deepseek pattern.
apps/sim/app/_shell/providers/session-provider.test.tsx	flush() helper extended with a macrotask tick to drain React Query's notifyManager — unrelated to the Sakana integration but a correct test-infrastructure fix.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Client
    participant SakanaProvider
    participant OpenAI_SDK as OpenAI SDK (baseURL: api.sakana.ai/v1)
    participant Tools

    Client->>SakanaProvider: executeRequest(request)

    alt No tools / streaming only
        SakanaProvider->>OpenAI_SDK: "chat.completions.create({stream:true})"
        OpenAI_SDK-->>SakanaProvider: AsyncIterable ChatCompletionChunk
        SakanaProvider-->>Client: StreamingExecution
    else Tools active (non-streaming tool loop)
        SakanaProvider->>OpenAI_SDK: chat.completions.create(payload)
        OpenAI_SDK-->>SakanaProvider: response (tool_calls)
        loop up to MAX_TOOL_ITERATIONS
            SakanaProvider->>Tools: executeTool(name, params)
            Tools-->>SakanaProvider: result
            SakanaProvider->>OpenAI_SDK: chat.completions.create(nextPayload + tool messages)
            OpenAI_SDK-->>SakanaProvider: response
        end
        alt request.stream
            SakanaProvider->>OpenAI_SDK: "chat.completions.create({tool_choice:none, stream:true})"
            OpenAI_SDK-->>SakanaProvider: AsyncIterable ChatCompletionChunk
            SakanaProvider-->>Client: StreamingExecution (accumulated cost)
        else deferResponseFormat
            SakanaProvider->>OpenAI_SDK: "chat.completions.create({response_format, tool_choice:none})"
            OpenAI_SDK-->>SakanaProvider: structured JSON response
            SakanaProvider-->>Client: ProviderResponse
        else plain
            SakanaProvider-->>Client: ProviderResponse
        end
    end

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Client
    participant SakanaProvider
    participant OpenAI_SDK as OpenAI SDK (baseURL: api.sakana.ai/v1)
    participant Tools

    Client->>SakanaProvider: executeRequest(request)

    alt No tools / streaming only
        SakanaProvider->>OpenAI_SDK: "chat.completions.create({stream:true})"
        OpenAI_SDK-->>SakanaProvider: AsyncIterable ChatCompletionChunk
        SakanaProvider-->>Client: StreamingExecution
    else Tools active (non-streaming tool loop)
        SakanaProvider->>OpenAI_SDK: chat.completions.create(payload)
        OpenAI_SDK-->>SakanaProvider: response (tool_calls)
        loop up to MAX_TOOL_ITERATIONS
            SakanaProvider->>Tools: executeTool(name, params)
            Tools-->>SakanaProvider: result
            SakanaProvider->>OpenAI_SDK: chat.completions.create(nextPayload + tool messages)
            OpenAI_SDK-->>SakanaProvider: response
        end
        alt request.stream
            SakanaProvider->>OpenAI_SDK: "chat.completions.create({tool_choice:none, stream:true})"
            OpenAI_SDK-->>SakanaProvider: AsyncIterable ChatCompletionChunk
            SakanaProvider-->>Client: StreamingExecution (accumulated cost)
        else deferResponseFormat
            SakanaProvider->>OpenAI_SDK: "chat.completions.create({response_format, tool_choice:none})"
            OpenAI_SDK-->>SakanaProvider: structured JSON response
            SakanaProvider-->>Client: ProviderResponse
        else plain
            SakanaProvider-->>Client: ProviderResponse
        end
    end

_{Reviews (3): Last reviewed commit: "test(session): de-flake SessionProvider ..." | Re-trigger Greptile}

waleedlatif1 · 2026-06-22T20:00:22Z

@greptile review

waleedlatif1 · 2026-06-22T20:00:23Z

@cursor review

waleedlatif1 · 2026-06-22T20:07:09Z

@greptile review

waleedlatif1 · 2026-06-22T20:07:10Z

@cursor review

waleedlatif1 · 2026-06-22T20:14:22Z

@greptile review

waleedlatif1 · 2026-06-22T20:14:23Z

@cursor review

waleedlatif1 · 2026-06-22T20:21:50Z

@greptile review

waleedlatif1 · 2026-06-22T20:21:51Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit de13c78. Configure here.}

waleedlatif1 · 2026-06-22T20:27:27Z

@greptile review

OpenAI-compatible provider at https://api.sakana.ai/v1 (bearer auth). Registers fugu (fast default) and fugu-ultra (reasoning flagship), both 1M context. BYOK-only, never hosted/auto-billed. Streaming, tool loop, and response_format supported; attachments mirror deepseek (unsupported in the current adapter).

OpenAI-compatible backends reject a request carrying both response_format and active tools/tool_choice. Mirror the LiteLLM pattern: withhold the JSON schema while tools are active and apply it on a final tool-free call (tool_choice: none) for both streaming and non-streaming paths.

- Rethrow tool-loop failures instead of swallowing them, so a failed run surfaces as a ProviderError rather than a partial success (matches LiteLLM). - Force tool_choice: 'none' on the post-tool streaming pass so the model cannot emit fresh tool calls that the text-only stream adapter would drop.

- Pass stream_options: { include_usage: true } on both streaming calls so token/cost data is captured (the shared OpenAI-compatible stream helper only fills usage from chunk usage, which the API omits without the flag). - Include !hasActiveTools in the early-stream guard so requests whose tools are all filtered out (e.g. usageControl 'none') still take the fast streaming path instead of the tool-loop path. Mirrors LiteLLM.

… valid An assistant message lists all tool_calls, so a call for an unconfigured tool must still get a matching `tool` response or the next request violates the OpenAI message contract. Emit an error tool-result for unknown tools instead of dropping them.

flush() only drained microtasks, so the query->render update occasionally lost the race and ctx.data was still null after the flush budget. Yield one macrotask tick per flush so React Query's notifyManager and deferred renders settle deterministically. Verified across repeated local runs.

waleedlatif1 · 2026-06-22T20:35:45Z

Rebased onto latest staging and stabilized the flaky SessionProvider normal-load test (app/_shell/providers/session-provider.test.tsx) that was intermittently failing CI on this PR. Root cause: the test's flush() helper only drained microtasks, so the React Query query → render update occasionally lost the race and ctx.data was still null after the flush budget. Fix: yield one macrotask tick per flush so React Query's notifyManager and deferred renders settle deterministically. Verified green across 13 consecutive local runs. Not a Sakana-provider change — purely a test-stability fix for the test gating this PR.

waleedlatif1 · 2026-06-22T20:35:46Z

@greptile review

waleedlatif1 · 2026-06-22T20:35:47Z

@cursor review

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit ee72a69. Configure here.}