feat: add AWS Bedrock mantle endpoint to AI Gateway#26745
feat: add AWS Bedrock mantle endpoint to AI Gateway#26745evgeniy-scherbina wants to merge 4 commits into
Conversation
994dd65 to
1bdf7a3
Compare
5eec1ac to
f953c8c
Compare
Documentation CheckUpdates Needed
Automated review via Coder Agents |
|
/coder-agents-review |
|
Chat: Review posted | View chat Review history
deep-review v0.9.0 | Round 1 | Last posted: Round 1, 19 findings (1 P2, 10 P3, 5 Nit, 3 Note), COMMENT. Review Finding inventoryFindings
Round logRound 1Panel. 1 P2, 9 P3, 2 Note, 4 Nit new. 4 dropped. Reviewed against f84801e..989e298. About deep-reviewCRF = Coder Review Finding (P0-P4, Nit, Note)
|
There was a problem hiding this comment.
The protocol discriminator is cleanly layered: codersdk types flow through proto to config.BedrockProtocol to the dispatch predicates, with the empty value resolving to InvokeModel at every hop. The mantle signing middleware correctly appends last so it signs after all header mutations, and the passthrough semantics are enforced structurally since the mantle path never calls augmentRequestForBedrockInvokeModel. Test density is healthy at 57.7%, and the integration test proves the real SigV4 signing with the correct service name.
Severity count: 1 P2, 9 P3, 2 Note, 4 Nit.
The P2 is convergent across 9 reviewers: unknown protocol values pass API validation and silently produce unsigned requests at runtime. The config.AWSBedrock.Validate() default case exists but is structurally unreachable for this failure mode. A protocol enum check at the API boundary would close it.
Process observations: the feature commit (f953c8c) has no body, which makes git log and git blame less useful for a 504-line change that introduces a new wire protocol. The follow-up commit (989e298) uses ci: make fmt but ci: denotes pipeline changes; style: or chore: fits better. Neither is blocking.
"If someone accidentally added augmentation to the mantle path, Model() would still return the client's model, and this test would still pass. The fields in the payload look like they were placed there to prove passthrough, then nobody wrote the assertions." (Bisky)
aibridge/intercept/messages/base.go:174
Note [CRF-14] Tracing records IsBedrock as a boolean (i.bedrock != nil), which is true for both InvokeModel and mantle. When debugging production issues, operators cannot tell from traces which wire protocol a request used. These are meaningfully different code paths (translate vs passthrough). A tracing.BedrockProtocol string attribute would make the distinction observable.
(Mafuuu)
🤖
🤖 This review was automatically generated with Coder Agents.
| }) | ||
| } | ||
| // The Mantle protocol signs requests with SigV4, which requires a region. | ||
| if req.Settings.Bedrock.ResolvedProtocol() == AIProviderBedrockProtocolMantle && req.Settings.Bedrock.Region == "" { |
There was a problem hiding this comment.
P2 [CRF-4] Unknown protocol values pass API validation and silently degrade to unsigned requests at runtime.
Validate() checks that mantle requires a region but never rejects an unknown Protocol string. An operator who sets protocol: "mnatle" (typo) passes validation. At runtime, isBedrockInvokeModel() matches only "" and "invoke-model", isBedrockMantle() matches only "mantle". The unknown value matches neither, so neither signing branch in newMessagesService fires. The request proceeds to the upstream unsigned.
config.AWSBedrock.Validate() has a default reject clause, but it lives inside withBedrockInvokeModelOptions and withBedrockMantleOptions, which are never entered because the dispatch predicates already excluded the unknown value. The validation is unreachable for this failure mode.
"An operator who sets
protocol: "tyop"passes validation. The value is stored. At request time, the request falls through to the generic Anthropic path with no SigV4 signing. AWS returns 403, and the operator gets a confusing error with no mention of the invalid protocol." (Pariston P3)
The same gap applies to UpdateAIProviderRequest.Validate() (line 335), which does not inspect the protocol field at all.
Fix in both Create and Update validators:
switch req.Settings.Bedrock.Protocol {
case "", AIProviderBedrockProtocolInvokeModel, AIProviderBedrockProtocolMantle:
default:
validations = append(validations, ValidationError{
Field: "settings.protocol",
Detail: fmt.Sprintf("unsupported bedrock protocol %q", req.Settings.Bedrock.Protocol),
})
}(Hisoka P2, Mafuuu P2, Knov P2, Meruem P2, Ryosuke P2, Pariston P3, Kurapika P3, Melody P3, Chopper P3)
🤖
| }) | ||
| } | ||
| // The Mantle protocol signs requests with SigV4, which requires a region. | ||
| if req.Settings.Bedrock.ResolvedProtocol() == AIProviderBedrockProtocolMantle && req.Settings.Bedrock.Region == "" { |
There was a problem hiding this comment.
P3 [CRF-2] UpdateAIProviderRequest.Validate() (line 335) does not check the protocol field at all. An update that sets protocol=mantle without providing a region, or sets an unknown protocol value, passes validation. The same protocol-enum check from CRF-4 and the mantle-region check from line 287 should be replicated in the Update validator.
(Netero, Hisoka, Knov, Meruem)
🤖
| // request (service "bedrock-mantle") and forwards it; the response is plain | ||
| // SSE. | ||
| func (i *interceptionBase) withBedrockMantleOptions(ctx context.Context) ([]option.RequestOption, error) { | ||
| cfg := i.bedrock.Cfg |
There was a problem hiding this comment.
P3 [CRF-1] Missing nil guard on i.bedrock. The sibling function withBedrockInvokeModelOptions (line 323) guards with if i.bedrock == nil { return nil, xerrors.New("nil bedrock runtime") }. This function dereferences i.bedrock.Cfg directly. The caller guards via isBedrockMantle() which requires i.bedrock != nil, so this won't panic in production. The risk is a nil-dereference panic if called directly without the caller guard.
(Netero)
🤖
|
|
||
| i := &interceptionBase{ | ||
| reqPayload: mustMessagesPayload(t, | ||
| `{"model":"anthropic.claude-opus-4-8","max_tokens":10000,"thinking":{"type":"adaptive"},"metadata":{"user_id":"u123"},"context_management":{"type":"auto"}}`), |
There was a problem hiding this comment.
P3 [CRF-5] Test claims passthrough but only asserts Model().
The payload includes thinking, metadata, and context_management fields that augmentRequestForBedrockInvokeModel would strip or rewrite for InvokeModel. The point of mantle is that these fields survive untouched, but no assertion verifies it. If someone accidentally added augmentation to the mantle path, Model() would still return the client's model (it reads from reqPayload for mantle, from config for InvokeModel), and this test would still pass.
Add an assertion that i.reqPayload is unchanged after the mantle code path runs, or remove the fields that aren't being tested.
(Bisky)
🤖
|
|
||
| create.Settings.Bedrock.Region = "us-east-1" | ||
| require.False(t, hasFieldError(create.Validate(), "settings.region")) | ||
| }) |
There was a problem hiding this comment.
P3 [CRF-6] This test is titled "InvokeModelDoesNotRequireRegionField" but Settings: codersdk.AIProviderSettings{} leaves Bedrock nil, so Validate() never enters the if req.Settings.Bedrock != nil block. The protocol check is never reached.
Set Bedrock: &codersdk.AIProviderBedrockSettings{} (zero value resolves to InvokeModel via ResolvedProtocol()) so the test enters the protocol-gated branch and proves the region check is scoped to mantle.
(Bisky)
🤖
| if v.Field == field { | ||
| return true | ||
| } | ||
| } |
There was a problem hiding this comment.
Nit [CRF-19] hasFieldError is a manual loop that slices.ContainsFunc (available since Go 1.21) replaces. The project already uses it in tests.
hasFieldError := func(vs []codersdk.ValidationError, field string) bool {
return slices.ContainsFunc(vs, func(v codersdk.ValidationError) bool {
return v.Field == field
})
}(Ging-Go)
🤖
| Creds: credentials.NewStaticCredentialsProvider("test-key", "test-secret", ""), | ||
| }, | ||
| } | ||
| opts, err := base.withBedrockMantleOptions(context.Background()) |
There was a problem hiding this comment.
Nit [CRF-20] context.Background() used where t.Context() is available since Go 1.24. The same PR's integration test already uses t.Context() for the mantle path.
(Ging-Go)
🤖
| string role_arn = 6; | ||
| string external_id = 7; | ||
| // protocol selects the Bedrock wire protocol ("invoke-model" or "mantle"). | ||
| // Empty fallbacks to invoke-model. |
There was a problem hiding this comment.
Nit [CRF-3] Grammar: "Empty fallbacks to invoke-model" should be "Empty falls back to invoke-model." ("fallbacks" is a noun; the verb form is "falls back.")
(Netero)
🤖
| } | ||
|
|
||
| if i.isBedrockMantle() { | ||
| ctx, cancel := context.WithTimeout(ctx, time.Second*30) |
There was a problem hiding this comment.
Nit [CRF-23] time.Second*30 is a bare literal in both the mantle block (here) and the InvokeModel block (line 287). A named constant like bedrockCredentialTimeout would make the intent clear and reduce the duplication.
(Gon)
🤖
| if ua := req.Header.Get("User-Agent"); ua != "" { | ||
| req.Header.Set("User-Agent", ua+" "+BedrockPRMUserAgent) | ||
| } | ||
| } |
There was a problem hiding this comment.
Nit [CRF-21] BedrockPRMUserAgent doc comment doesn't expand PRM. PRM is the ISV Partner Registration Marker (or Partner Registration Manager, depending on the AWS doc vintage). The value itself is a percent-encoded APN token, adding another unexplained acronym. A one-line expansion in the doc comment saves someone a search:
// BedrockPRMUserAgent is Coder's AWS Partner Registration Marker (PRM),
// appended to Bedrock User-Agent headers for AWS ISV attribution.(Leorio)
🤖
AWS Bedrock mantle support in AI Gateway
Summary
Add support for the AWS Bedrock mantle endpoint (
bedrock-mantle.{region}.api.aws/anthropic/v1/messages) to AI Gateway. Mantle serves Claude through the native Anthropic Messages API. We model it as aprotocolfield on the existing Bedrock provider settings (invoke-modeldefault, ormantle) rather than as a new provider type, and we treat mantle as a pure passthrough: SigV4-sign and forward, no body translation.Background
Claude on AWS Bedrock is reachable through two endpoints, each speaking exactly one wire protocol:
bedrock-runtime.{region}.amazonaws.com. Model id in the URL path, request translated into Bedrock's InvokeModel format, responses returned as a binary AWS eventstream. This is what AI Gateway already supported for Bedrock.bedrock-mantle.{region}.api.aws/anthropic/v1/messages. Native Anthropic Messages API: model in the body, plain SSE streaming.Why a
protocolfield, not a new provider typeThe alternative is to model mantle as its own
ai_provider_type(bedrock-mantle) alongsidebedrock. I chose theprotocolfield instead for two reasons:invoke-modeldefault andmantle) models that more organically than two provider types.protocolfield lives in the settings JSON blob (empty resolves toinvoke-model, so existing providers are unaffected), whereas a new type means an enum value and theALTER TYPE ... ADD VALUEmigration that goes with it.Why passthrough, not translation
The client already emits Bedrock-legal requests in mantle mode:
So the gateway just forwards the body and SigV4-signs it (service
bedrock-mantle), and skips all the InvokeModel body-translation (model remap, thinking conversion, beta-flag allowlist, field stripping). This keeps the mantle path thin and avoids a second copy of translation logic to maintain.Consequences
model/small_fast_modelare used by InvokeModel but ignored by mantle (the client sends the model), andbase_urlis required for mantle but optional for InvokeModel. Validation is protocol-aware.regionand thebase_urlhost must name the same region (the SigV4 scope must match the endpoint); a mismatch surfaces asCredential should be scoped to a valid region.Draft UI
Follow-up PRs: