Skip to content

feat: add AWS Bedrock mantle endpoint to AI Gateway#26745

Open
evgeniy-scherbina wants to merge 4 commits into
mainfrom
yevhenii/add-bedrock-provider
Open

feat: add AWS Bedrock mantle endpoint to AI Gateway#26745
evgeniy-scherbina wants to merge 4 commits into
mainfrom
yevhenii/add-bedrock-provider

Conversation

@evgeniy-scherbina

@evgeniy-scherbina evgeniy-scherbina commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

AWS Bedrock mantle support in AI Gateway

Summary

Add support for the AWS Bedrock mantle endpoint (bedrock-mantle.{region}.api.aws/anthropic/v1/messages) to AI Gateway. Mantle serves Claude through the native Anthropic Messages API. We model it as a protocol field on the existing Bedrock provider settings (invoke-model default, or mantle) rather than as a new provider type, and we treat mantle as a pure passthrough: SigV4-sign and forward, no body translation.

Background

Claude on AWS Bedrock is reachable through two endpoints, each speaking exactly one wire protocol:

  1. InvokeModel (existing): bedrock-runtime.{region}.amazonaws.com. Model id in the URL path, request translated into Bedrock's InvokeModel format, responses returned as a binary AWS eventstream. This is what AI Gateway already supported for Bedrock.
  2. Mantle (this doc): bedrock-mantle.{region}.api.aws/anthropic/v1/messages. Native Anthropic Messages API: model in the body, plain SSE streaming.

Why a protocol field, not a new provider type

The alternative is to model mantle as its own ai_provider_type (bedrock-mantle) alongside bedrock. I chose the protocol field instead for two reasons:

  1. Mantle reads more like a protocol of Bedrock than a separate provider. It is the same AWS account, credentials, region, and IAM, reached over a different wire protocol and host. One Bedrock provider with two protocols (invoke-model default and mantle) models that more organically than two provider types.
  2. It avoids a database migration. The protocol field lives in the settings JSON blob (empty resolves to invoke-model, so existing providers are unaffected), whereas a new type means an enum value and the ALTER TYPE ... ADD VALUE migration that goes with it.

Why passthrough, not translation

The client already emits Bedrock-legal requests in mantle mode:

export CLAUDE_CODE_USE_MANTLE=1
export CLAUDE_CODE_SKIP_MANTLE_AUTH=1
export ANTHROPIC_BEDROCK_MANTLE_BASE_URL=https://<coder>/api/v2/aibridge/<provider-name>

So the gateway just forwards the body and SigV4-signs it (service bedrock-mantle), and skips all the InvokeModel body-translation (model remap, thinking conversion, beta-flag allowlist, field stripping). This keeps the mantle path thin and avoids a second copy of translation logic to maintain.

Consequences

  • Protocol-dependent fields: model / small_fast_model are used by InvokeModel but ignored by mantle (the client sends the model), and base_url is required for mantle but optional for InvokeModel. Validation is protocol-aware.
  • No central model control on mantle: because it is a passthrough, the operator cannot pin the model.
  • region and the base_url host must name the same region (the SigV4 scope must match the endpoint); a mismatch surfaces as Credential should be scoped to a valid region.

Draft UI

image

Follow-up PRs:

@evgeniy-scherbina evgeniy-scherbina force-pushed the yevhenii/add-bedrock-provider branch 4 times, most recently from 994dd65 to 1bdf7a3 Compare July 9, 2026 19:00
@evgeniy-scherbina evgeniy-scherbina force-pushed the yevhenii/add-bedrock-provider branch from 5eec1ac to f953c8c Compare July 9, 2026 19:33
@evgeniy-scherbina evgeniy-scherbina marked this pull request as ready for review July 10, 2026 11:42
@coder-tasks

coder-tasks Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Documentation Check

Updates Needed

  • docs/ai-coder/ai-gateway/providers.md - The Amazon Bedrock section needs to document the new protocol field:
    • The two protocol options: invoke-model (default, legacy behavior) and mantle (passthrough with SigV4 signing)
    • For mantle: both region and base_url are required (unlike invoke-model which accepts either)
    • For mantle: no request/response translation occurs; the gateway only signs and forwards
    • The mantle endpoint format: bedrock-mantle.{region}.api.aws/anthropic/v1/messages
    • The base_url for mantle should be the Messages API prefix without /v1/messages
    • Note that model and small_fast_model are not required for mantle (the client controls the model)

Automated review via Coder Agents

@evgeniy-scherbina

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

@coder-agents-review

coder-agents-review Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Chat: Review posted | View chat
Requested: 2026-07-10 11:46 UTC by @evgeniy-scherbina
Spend: $40.81 / $100.00

Review history
  • R1 (2026-07-10): 16 reviewers, 5 Nit, 3 Note, 1 P2, 10 P3, COMMENT. Review

deep-review v0.9.0 | Round 1 | f84801e..989e298

Last posted: Round 1, 19 findings (1 P2, 10 P3, 5 Nit, 3 Note), COMMENT. Review

Finding inventory

Findings

# Sev Status Location Summary Round Reviewer Posted
CRF-1 P3 Open base.go:365 Missing nil guard on i.bedrock in withBedrockMantleOptions R1 Netero Yes
CRF-2 P3 Open aiproviders.go:287 Mantle-region and protocol validation missing from UpdateAIProviderRequest.Validate() R1 Netero, Hisoka, Knov, Meruem Yes
CRF-3 Nit Open aibridged.proto:239 Grammar: "fallbacks" should be "falls back" R1 Netero Yes
CRF-4 P2 Open aiproviders.go:287 Unknown protocol values pass API validation and silently degrade to unsigned requests R1 Hisoka P2, Mafuuu P2, Knov P2, Meruem P2, Ryosuke P2, Pariston P3, Kurapika P3, Melody P3, Chopper P3 Yes
CRF-5 P3 Open base_internal_test.go:1132 TestBedrockMantleIsPassthrough claims passthrough but only asserts Model() R1 Bisky Yes
CRF-6 P3 Open aiproviders_test.go:246 InvokeModelDoesNotRequireRegionField tests nil Bedrock, not InvokeModel protocol R1 Bisky Yes
CRF-7 P3 Open base.go:82 appendBedrockPRMUserAgent silently drops attribution when User-Agent is empty R1 Meruem P3, Leorio Note, Ryosuke Note Yes
CRF-8 P3 Open base.go:366 withBedrockMantleOptions returns bare Validate() errors without wrapping R1 Leorio Yes
CRF-9 P3 Open base.go:385 Credential and signing errors in mantle middleware lack diagnostic wrapping R1 Chopper Yes
CRF-10 P3 Open base.go:72 Five doc comments restate code structure (base.go:72, 151, 359, 370, 414) R1 Gon Yes
CRF-11 P3 Dropped by orchestrator (process feedback, not code) - Feature commit empty body R1 Leorio No
CRF-12 P3 Open base.go:269 Redundant WithBaseURL: generic path sets it then Bedrock paths override R1 Ryosuke Yes
CRF-13 P3 Open typesGenerated.ts:374 Generated TS comment diverges from Go source (extra "without a schema version bump") R1 Mafu-san Yes
CRF-14 Note Open base.go:174 Tracing does not distinguish between InvokeModel and mantle protocols R1 Mafuuu Yes
CRF-15 Note Open base.go:297 Mantle passthrough grants clients model selection authority R1 Kurapika Yes
CRF-16 Note Dropped by orchestrator (structural observation, no action needed) aiproviders_bedrock.go:13 Protocol enum as two separate types with raw cast bridging R1 Meruem No
CRF-17 Note Dropped by orchestrator (low stakes, predicates clearly disjoint) base.go:286 Protocol dispatch uses separate ifs not if/else R1 Hisoka No
CRF-18 Note Open typesGenerated.ts:378 Mantle providers will appear broken in admin edit form R1 Nami Yes
CRF-19 Nit Open aiproviders_test.go:219 Manual contains check; slices.ContainsFunc available since Go 1.21 R1 Ging-Go Yes
CRF-20 Nit Open base_internal_test.go:1190 context.Background() where t.Context() available since Go 1.24 R1 Ging-Go Yes
CRF-21 Nit Open base.go:72 BedrockPRMUserAgent doc doesn't expand PRM acronym R1 Leorio Yes
CRF-22 Nit Dropped by orchestrator (process feedback, not code) - ci: make fmt commit uses wrong scope R1 Leorio No
CRF-23 Nit Open base.go:298 Credential timeout is inline magic number (30s) R1 Gon Yes

Round log

Round 1

Panel. 1 P2, 9 P3, 2 Note, 4 Nit new. 4 dropped. Reviewed against f84801e..989e298.

About deep-review

CRF = Coder Review Finding (P0-P4, Nit, Note)

Reviewer Focus
Bisky tests
Chopper ops/errors
Churn-guard change verification
Ging language modernization
Gon naming
Hisoka edge cases
Killua perf
Kite change integrity
Knov contracts
Knuckle SQL
Komugi flake/determinism
Kurapika security
Law decomposition
Leorio docs
Luffy product
Mafu-san process
Mafuuu contracts
Melody dispatch/pairing
Meruem structural
Nami frontend
Netero mechanical checks
Pariston premise testing
Pen-botter product gaps
Razor verification
Robin duplication
Ryosuke Go arch
Takumi concurrency
Zoro shape

🤖 Managed by Coder Agents.

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The protocol discriminator is cleanly layered: codersdk types flow through proto to config.BedrockProtocol to the dispatch predicates, with the empty value resolving to InvokeModel at every hop. The mantle signing middleware correctly appends last so it signs after all header mutations, and the passthrough semantics are enforced structurally since the mantle path never calls augmentRequestForBedrockInvokeModel. Test density is healthy at 57.7%, and the integration test proves the real SigV4 signing with the correct service name.

Severity count: 1 P2, 9 P3, 2 Note, 4 Nit.

The P2 is convergent across 9 reviewers: unknown protocol values pass API validation and silently produce unsigned requests at runtime. The config.AWSBedrock.Validate() default case exists but is structurally unreachable for this failure mode. A protocol enum check at the API boundary would close it.

Process observations: the feature commit (f953c8c) has no body, which makes git log and git blame less useful for a 504-line change that introduces a new wire protocol. The follow-up commit (989e298) uses ci: make fmt but ci: denotes pipeline changes; style: or chore: fits better. Neither is blocking.

"If someone accidentally added augmentation to the mantle path, Model() would still return the client's model, and this test would still pass. The fields in the payload look like they were placed there to prove passthrough, then nobody wrote the assertions." (Bisky)


aibridge/intercept/messages/base.go:174

Note [CRF-14] Tracing records IsBedrock as a boolean (i.bedrock != nil), which is true for both InvokeModel and mantle. When debugging production issues, operators cannot tell from traces which wire protocol a request used. These are meaningfully different code paths (translate vs passthrough). A tracing.BedrockProtocol string attribute would make the distinction observable.

(Mafuuu)

🤖

🤖 This review was automatically generated with Coder Agents.

Comment thread codersdk/aiproviders.go
})
}
// The Mantle protocol signs requests with SigV4, which requires a region.
if req.Settings.Bedrock.ResolvedProtocol() == AIProviderBedrockProtocolMantle && req.Settings.Bedrock.Region == "" {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 [CRF-4] Unknown protocol values pass API validation and silently degrade to unsigned requests at runtime.

Validate() checks that mantle requires a region but never rejects an unknown Protocol string. An operator who sets protocol: "mnatle" (typo) passes validation. At runtime, isBedrockInvokeModel() matches only "" and "invoke-model", isBedrockMantle() matches only "mantle". The unknown value matches neither, so neither signing branch in newMessagesService fires. The request proceeds to the upstream unsigned.

config.AWSBedrock.Validate() has a default reject clause, but it lives inside withBedrockInvokeModelOptions and withBedrockMantleOptions, which are never entered because the dispatch predicates already excluded the unknown value. The validation is unreachable for this failure mode.

"An operator who sets protocol: "tyop" passes validation. The value is stored. At request time, the request falls through to the generic Anthropic path with no SigV4 signing. AWS returns 403, and the operator gets a confusing error with no mention of the invalid protocol." (Pariston P3)

The same gap applies to UpdateAIProviderRequest.Validate() (line 335), which does not inspect the protocol field at all.

Fix in both Create and Update validators:

switch req.Settings.Bedrock.Protocol {
case "", AIProviderBedrockProtocolInvokeModel, AIProviderBedrockProtocolMantle:
default:
    validations = append(validations, ValidationError{
        Field:  "settings.protocol",
        Detail: fmt.Sprintf("unsupported bedrock protocol %q", req.Settings.Bedrock.Protocol),
    })
}

(Hisoka P2, Mafuuu P2, Knov P2, Meruem P2, Ryosuke P2, Pariston P3, Kurapika P3, Melody P3, Chopper P3)

🤖

Comment thread codersdk/aiproviders.go
})
}
// The Mantle protocol signs requests with SigV4, which requires a region.
if req.Settings.Bedrock.ResolvedProtocol() == AIProviderBedrockProtocolMantle && req.Settings.Bedrock.Region == "" {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 [CRF-2] UpdateAIProviderRequest.Validate() (line 335) does not check the protocol field at all. An update that sets protocol=mantle without providing a region, or sets an unknown protocol value, passes validation. The same protocol-enum check from CRF-4 and the mantle-region check from line 287 should be replicated in the Update validator.

(Netero, Hisoka, Knov, Meruem)

🤖

// request (service "bedrock-mantle") and forwards it; the response is plain
// SSE.
func (i *interceptionBase) withBedrockMantleOptions(ctx context.Context) ([]option.RequestOption, error) {
cfg := i.bedrock.Cfg

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 [CRF-1] Missing nil guard on i.bedrock. The sibling function withBedrockInvokeModelOptions (line 323) guards with if i.bedrock == nil { return nil, xerrors.New("nil bedrock runtime") }. This function dereferences i.bedrock.Cfg directly. The caller guards via isBedrockMantle() which requires i.bedrock != nil, so this won't panic in production. The risk is a nil-dereference panic if called directly without the caller guard.

(Netero)

🤖


i := &interceptionBase{
reqPayload: mustMessagesPayload(t,
`{"model":"anthropic.claude-opus-4-8","max_tokens":10000,"thinking":{"type":"adaptive"},"metadata":{"user_id":"u123"},"context_management":{"type":"auto"}}`),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 [CRF-5] Test claims passthrough but only asserts Model().

The payload includes thinking, metadata, and context_management fields that augmentRequestForBedrockInvokeModel would strip or rewrite for InvokeModel. The point of mantle is that these fields survive untouched, but no assertion verifies it. If someone accidentally added augmentation to the mantle path, Model() would still return the client's model (it reads from reqPayload for mantle, from config for InvokeModel), and this test would still pass.

Add an assertion that i.reqPayload is unchanged after the mantle code path runs, or remove the fields that aren't being tested.

(Bisky)

🤖


create.Settings.Bedrock.Region = "us-east-1"
require.False(t, hasFieldError(create.Validate(), "settings.region"))
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 [CRF-6] This test is titled "InvokeModelDoesNotRequireRegionField" but Settings: codersdk.AIProviderSettings{} leaves Bedrock nil, so Validate() never enters the if req.Settings.Bedrock != nil block. The protocol check is never reached.

Set Bedrock: &codersdk.AIProviderBedrockSettings{} (zero value resolves to InvokeModel via ResolvedProtocol()) so the test enters the protocol-gated branch and proves the region check is scoped to mantle.

(Bisky)

🤖

if v.Field == field {
return true
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit [CRF-19] hasFieldError is a manual loop that slices.ContainsFunc (available since Go 1.21) replaces. The project already uses it in tests.

hasFieldError := func(vs []codersdk.ValidationError, field string) bool {
    return slices.ContainsFunc(vs, func(v codersdk.ValidationError) bool {
        return v.Field == field
    })
}

(Ging-Go)

🤖

Creds: credentials.NewStaticCredentialsProvider("test-key", "test-secret", ""),
},
}
opts, err := base.withBedrockMantleOptions(context.Background())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit [CRF-20] context.Background() used where t.Context() is available since Go 1.24. The same PR's integration test already uses t.Context() for the mantle path.

(Ging-Go)

🤖

string role_arn = 6;
string external_id = 7;
// protocol selects the Bedrock wire protocol ("invoke-model" or "mantle").
// Empty fallbacks to invoke-model.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit [CRF-3] Grammar: "Empty fallbacks to invoke-model" should be "Empty falls back to invoke-model." ("fallbacks" is a noun; the verb form is "falls back.")

(Netero)

🤖

}

if i.isBedrockMantle() {
ctx, cancel := context.WithTimeout(ctx, time.Second*30)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit [CRF-23] time.Second*30 is a bare literal in both the mantle block (here) and the InvokeModel block (line 287). A named constant like bedrockCredentialTimeout would make the intent clear and reduce the duplication.

(Gon)

🤖

if ua := req.Header.Get("User-Agent"); ua != "" {
req.Header.Set("User-Agent", ua+" "+BedrockPRMUserAgent)
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit [CRF-21] BedrockPRMUserAgent doc comment doesn't expand PRM. PRM is the ISV Partner Registration Marker (or Partner Registration Manager, depending on the AWS doc vintage). The value itself is a percent-encoded APN token, adding another unexplained acronym. A one-line expansion in the doc comment saves someone a search:

// BedrockPRMUserAgent is Coder's AWS Partner Registration Marker (PRM),
// appended to Bedrock User-Agent headers for AWS ISV attribution.

(Leorio)

🤖

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant