feat: add automatic key failover for AI Bridge passthrough#24920
Merged
Conversation
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
b282407 to
ae98c2d
Compare
56a2653 to
f0af485
Compare
ae98c2d to
541a917
Compare
f0af485 to
aec65b9
Compare
541a917 to
337ee29
Compare
aec65b9 to
3c4acae
Compare
2f7c02d to
c38a2a8
Compare
c5042db to
95dd77d
Compare
c38a2a8 to
8ac3606
Compare
7091a0f to
4ccc07d
Compare
38ed74c to
866efb9
Compare
4ccc07d to
8fe651e
Compare
866efb9 to
3ddd9a2
Compare
8fe651e to
894d769
Compare
3ddd9a2 to
8590bdf
Compare
3657928 to
05a86ef
Compare
6f3b4f4 to
02cc359
Compare
926ca53 to
2fe5298
Compare
a432358 to
d143231
Compare
3e888ea to
3b185bb
Compare
d143231 to
4c78db8
Compare
3b185bb to
dc0fd49
Compare
64844ef to
99452e8
Compare
5442499 to
88e1d65
Compare
2dc9986 to
554ca68
Compare
a035e39 to
4d17599
Compare
554ca68 to
a6072ef
Compare
64f24d6 to
f621fe5
Compare
db0c7d2 to
36154dd
Compare
52acd55 to
8b61f0c
Compare
pawbana
reviewed
May 7, 2026
|
|
||
| // KeyFailoverConfig is the per-provider configuration consumed by | ||
| // NewKeyFailoverTransport. | ||
| type KeyFailoverConfig struct { |
Contributor
There was a problem hiding this comment.
nit: is this Config? Looks maybe like KeyManager or something?
pawbana
approved these changes
May 7, 2026
|
|
||
| // Mock upstream: counts requests and returns | ||
| // scripted responses keyed by API key. An unmapped | ||
| // key falls through to 500 so misconfigured cases |
Contributor
There was a problem hiding this comment.
maybe directly fail test in that case?
Contributor
Author
Merge activity
|
This was referenced May 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds automatic key failover for passthrough routes for the Anthropic and OpenAI providers. A new
keyFailoverTransportwraps the reverse-proxy transport: centralized requests walk the configured key pool and retry with the next key on key-specific failures (401/403/429), reusing the same key-marking semantics as the bridged routes.BYOK passthrough requests run as a single attempt with no failover.
Changes
keypool.KeyFailoverConfigcarrying thePoolto walk and the provider-specific closures (IsBYOK,InjectAuthKey,MarkKey,BuildExhaustedResponse).keypool.NewKeyFailoverTransport: wraps an innerhttp.RoundTripper. Returnsinnerunchanged whenPoolis nil, otherwise produces a transport that buffers the request body once, walks the pool per request, and replays each attempt with the next key.Provider.KeyFailoverConfig(logger)interface method. Anthropic injectsX-Api-Key; OpenAI injectsAuthorization: Bearer ...; Copilot returns an empty config.passthrough.gowiresNewKeyFailoverTransportaround the existing apidump middleware, so every retry attempt is recorded.Related Issues
Related to: coder/internal#1446
Related to: https://linear.app/codercom/issue/AIGOV-197/aibridge-automatic-key-failover-for-bridged-and-passthrough-routes
Follow-up PRs
Provider.InjectAuthHeadermethod now that all auth is applied per-attempt byKeyFailoverTransport.Note
Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira