test: migrate chatd tests to AI Gateway routing#26658
Draft
johnstcn wants to merge 4 commits into
Draft
Conversation
Migrate all chatd tests from AIGatewayRoutingEnabled=false (direct routing) to AI Gateway routing using the test helpers extracted in PR #26639. Full-server tests now call aibridgedtest.StartTestAIBridgeDaemon to wire a real in-process aibridged daemon. The coderdtest and coderdenttest helpers that return *coderd.API are switched to NewWithAPI so the daemon can be started after server creation. Bare-chatd tests already use chattest.NewMockAIBridgeTransport from PR #26639. The chatd.Config.AIGatewayRoutingEnabled=true assignment is kept (plain bool, zero value false). Two direct-branch-only internal tests are removed: - TestDirectModelBuildDoesNotRequireActiveAPIKeyID - TestResolveComputerUseModel_OpenAIMissingCredentials No production code changes. The AIGatewayRoutingEnabled flag and direct routing code remain untouched until PR 3.
Replace the deleted TestResolveComputerUseModel_OpenAIMissingCredentials (direct routing error path) with TestResolveComputerUseModel_AIGatewayMissingAPIKeyID which tests the same computer use error wrapping through the AI Gateway routing path.
johnstcn
commented
Jun 24, 2026
Comment on lines
-144
to
-170
| func TestResolveComputerUseModel_OpenAIMissingCredentials(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| server := &Server{} | ||
| provider := chattool.ComputerUseProviderOpenAI | ||
| modelProvider, modelName, ok := chattool.DefaultComputerUseModel(provider) | ||
| require.True(t, ok) | ||
|
|
||
| model, debugEnabled, resolvedProvider, resolvedModel, err := server.resolveComputerUseModel( | ||
| context.Background(), | ||
| database.Chat{ID: uuid.New(), OwnerID: uuid.New()}, | ||
| newDirectModelRoute(modelProvider, chatprovider.ProviderAPIKeys{}), | ||
| provider, | ||
| modelProvider, | ||
| modelName, | ||
| modelBuildOptions{}, | ||
| ) | ||
| require.Error(t, err) | ||
| require.Nil(t, model) | ||
| require.False(t, debugEnabled) | ||
| require.Empty(t, resolvedProvider) | ||
| require.Empty(t, resolvedModel) | ||
| require.Contains(t, err.Error(), `provider "openai" model "gpt-5.5"`) | ||
| require.Contains(t, err.Error(), "OPENAI_API_KEY is not set") | ||
| require.NotContains(t, err.Error(), "ANTHROPIC_API_KEY") | ||
| } | ||
|
|
Member
Author
There was a problem hiding this comment.
Replaced by TestResolveComputerUseModel_AIGatewayMissingAPIKeyID
- Change newChatClient* helper signatures from testing.TB to *testing.T to eliminate 5 t.(*testing.T) type assertions - Replace closer/closer2 variables with _ in enterprise TestChatStreamRelay subtests (the closer is handled by t.Cleanup inside NewWithAPI) - Reuse existing ctx where the daemon timeout matches the test's context timeout
The daemon context was created with testutil.Context(t, testutil.WaitLong) which has a 25s deadline. Under CI load, tests that take >25s would see the daemon shut down mid-test. Use t.Context() (no deadline, tied to test lifetime) instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Migrate all chatd tests from
AIGatewayRoutingEnabled = false(direct routing) to AI Gateway routing using the test helpers extracted in #26639.Full-server tests now call
aibridgedtest.StartTestAIBridgeDaemonto wire a real in-process aibridged daemon. Thecoderdtestandcoderdenttesthelpers are switched toNewWithAPIso the daemon can be started after server creation.Bare-chatd tests already use
chattest.NewMockAIBridgeTransportfrom #26639. Thechatd.Config.AIGatewayRoutingEnabled = trueassignment is kept (plain bool, zero value false).Two direct-branch-only internal tests are removed:
TestDirectModelBuildDoesNotRequireActiveAPIKeyIDTestResolveComputerUseModel_OpenAIMissingCredentialsNo production code changes. The
AIGatewayRoutingEnabledflag and direct routing code remain untouched until PR 3.Depends on #26639
Refs https://linear.app/codercom/issue/CODAGT-681/migrate-chatd-tests-to-ai-gateway-routing
Implementation plan
Three layers of defaults with different semantics:
codersdk.DeploymentValues.AI.Chat.AIGatewayRoutingEnabledserpent.Bool"true"(already)Set("false")callschatd.Config.AIGatewayRoutingEnabledboolfalse= true(plain bool, zero value false)chatd.Server.aiGatewayRoutingEnabledboolfalse: true(defer to PR 3)Internal test
= truecleanup is deferred to PR 3 when the field is removed entirely.Files changed (8 test files, no production code):
coderd/x/chatd/chatd_test.go— 6 full-server tests migrated toNewWithAPI+ daemon,directChatRoutingDeploymentValueshelper deleted, 3 bare-chatd tests renamedcoderd/x/chatd/context_integration_test.go— 2 tests migratedcoderd/exp_chats_test.go—chatDeploymentValueshelper deleted, all 5 helper functions now useNewWithAPI+ daemon internally (no call site changes)coderd/exp_chats_acl_test.go— stalechatDeploymentValuesreference replacedenterprise/coderd/exp_chats_test.go— 9 sites across 5TestChatStreamRelaysubtests migratedcli/exp_scaletest_chat_test.go— 1 test migratedcoderd/x/chatd/model_routing_internal_test.go— 1 direct-only test removedcoderd/x/chatd/chatd_internal_test.go— 1 direct-only test removed