Skip to content

feat: implement AI settings page with providers management UI#25328

Closed
jakehwll wants to merge 98 commits into
mainfrom
jakehwll/ai-settings
Closed

feat: implement AI settings page with providers management UI#25328
jakehwll wants to merge 98 commits into
mainfrom
jakehwll/ai-settings

Conversation

@jakehwll
Copy link
Copy Markdown
Contributor

@jakehwll jakehwll commented May 14, 2026

🤖 This PR was modified by Coder Agents on behalf of Jake Howell.

Scaffolds a new /ai/settings admin page for managing AI providers (formerly "AI Bridge providers"). Stacked on top of #24892 / #24893 / #24894: the page is now wired against the real /api/v2/ai/providers HTTP CRUD routes from the providers-handlers stack and the generated codersdk types.

  • New AISettingsLayout with breadcrumb + sidebar, mounted under /ai/settings
  • ProvidersPage list view + table of configured providers
  • AddProviderPage and UpdateProviderPage flows backed by a shared ProviderForm
  • Reusable ProviderIcon and ProviderRow components, each with Storybook stories
  • aiProviders query + create/update/delete mutation hooks against API.{create,update,delete,get}AIProvider(s)
  • Compound create flow: createAIProviderMutation posts the provider then, when an API key was supplied, chains POST /providers/{id}/keys and rolls the provider back via DELETE if the key fails
  • Bedrock remains a UI option but maps to a wire type:"anthropic" provider whose settings carries the discriminated {_type:"bedrock",_version:1,...} shape; isBedrockProvider detects this on load
  • Provider names validated against the API's kebab-case regex ^[a-z0-9]+(-[a-z0-9]+)*$
  • Page gated on a new viewAnyAIProvider permission against the ai_provider RBAC resource (instead of reusing viewAnyAIBridgeInterception, which belongs to the AI Bridge logs feature)
Recent review-pass cleanup
  • Collapse baseURL/baseUrl into a single baseUrl; OpenAI/Anthropic relabelled as "Custom endpoint" with matching validation
  • Rename providerFormValuesToCreateRequestproviderFormValuesToRequest (helper is used for both create and update)
  • Drop the non-functional organization filter on the providers list
  • Credential UX: masked inputs are now editable; focusing (or clicking "Clear key/keys") clears the saved mask; the brittle isCredentialPlaceholder heuristic is gone
  • getProviderIcon / getProviderName now fall back to CircleQuestionMarkIcon + a sensible label for unknown provider types (matching the AI Bridge sessions page convention)
  • Drop the imperative providerFormKey remount; rely on formik's enableReinitialize after the query refetches the provider
  • Wire to real /api/v2/ai/providers (and /keys sub-resource); delete the in-page mock module, move MockAIProvider* to testHelpers/entities.ts
  • Add viewAnyAIProvider permission entry to permissions.json and switch the four call sites that previously reused the AI Bridge permission
  • Merge the rebased dk/aibridge-providers-handlers and adapt providerFormApiMap.ts + testHelpers/entities.ts to the new discriminated AIProviderSettings ({_type, _version, region, model, small_fast_model, access_key, access_key_secret})

@jakehwll jakehwll requested a review from jeremyruppel May 14, 2026 10:42
@jakehwll jakehwll changed the title feat(site): scaffold AI settings page with providers management UI feat(site): implement AI settings page with providers management UI May 14, 2026
@jakehwll jakehwll changed the title feat(site): implement AI settings page with providers management UI feat(site): scaffold AI settings page with providers management UI May 14, 2026
args: {
provider: "bedrock",
},
};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe a test for the fallback name? idk, too low value?

@jakehwll jakehwll force-pushed the jakehwll/ai-settings branch from 8895cc0 to e2cecfa Compare May 14, 2026 13:11
@jakehwll jakehwll changed the base branch from main to dk/aibridge-providers-db May 14, 2026 13:11
Copy link
Copy Markdown
Contributor

@jeremyruppel jeremyruppel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Generated by Coder Agents on behalf of @jeremyruppel

Clean structure: container/view separation, React Query key hierarchy, cache invalidation, and Storybook coverage for every new component are all solid. The credential-masking UX is thoughtfully designed.

A few things to look at: 1 P1, 3 P2, 2 P3 across 7 inline comments. This review contains findings that may need attention before merge.

<div className="flex flex-col gap-2">
<Label htmlFor={id}>{label}</Label>
{description && (
<div className="text-xs text-content-secondary">{description}</div>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 description not linked to aria-describedby; screen readers won't announce it (Structural Analyst P2, Frontend Reviewer P2)

Frontend Reviewer: "The new description prop renders a div between <Label> and <Input>. However, the existing aria-describedby on the <Input> only references the error or helper text IDs. The description div has no id and is not included in aria-describedby. Screen readers will not announce the description when the input receives focus."

This is a cross-cutting change in a shared component, so worth getting right. Fix: give the div id={\${id}-description`}and include it in the input'saria-describedby` chain.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Generated by Coder Agents on behalf of @jakehwll

Fixed in 5166702d. FormField now generates a stable ${id}-description id for the description <div> and adds it to the input's aria-describedby chain alongside the existing helper / error ids.

Comment thread site/src/pages/AISettingsPage/ProvidersPage/ProvidersPageView.tsx
key={provider.name}
hover
className="cursor-pointer"
onClick={() => onClick?.()}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Clickable table row is not keyboard-accessible (Frontend Reviewer P2)

Frontend Reviewer: "<TableRow> has onClick and cursor-pointer but no tabIndex, role, or onKeyDown handler. Keyboard-only users cannot navigate to or activate these rows. Per site/AGENTS.md, interactive elements need a semantic role and keyboard access."

Fix: add tabIndex={0}, role="link" (or wrap with <Link>), and handle Enter/Space.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Generated by Coder Agents on behalf of @jakehwll

Fixed in 5166702d. ProviderRow now uses the shared useClickableTableRow hook (same one TemplateVersions uses), which adds the button role, tab stop, Enter/Space activation, focus ring, and middle-click handling.

const newAccessKey = values.accessKey.trim();
const newAccessKeySecret = values.accessKeySecret.trim();
const hasNewCredentials = newAccessKey !== "" && newAccessKeySecret !== "";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Partial Bedrock credential update is silently discarded (Test Auditor P2, Edge Case Analyst P2, Contract Auditor P2)

Contract Auditor: "hasNewCredentials requires both newAccessKey !== "" AND newAccessKeySecret !== "". If the user clears both fields, then types a new access key but leaves the secret empty, hasNewCredentials is false. The code falls through to the existingProvider.settings branch and silently preserves the old credentials. The user believes they updated the access key; the API receives the old one."

The Yup schema for edit mode makes both fields optional, so form-level validation won't catch this either. Fix: add a Yup .test() that requires both fields when either is non-empty.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Generated by Coder Agents on behalf of @jakehwll

Fixed in 338be75d. The Bedrock Yup schema now has paired .test() checks on accessKey and accessKeySecret: when one is non-empty and not the saved-credential mask, the other is required. Submitting a partial Bedrock rotation now surfaces an inline validation error instead of being silently dropped by the mapper. The mapper itself still gates the credential fields on both values being filled (and mask-sanitized) so a bug bypassing the schema would still keep the existing values rather than write a half-update.

Comment thread site/src/modules/dashboard/Navbar/NavbarView.tsx
!canViewHealth
)
return null;
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Braceless if after multi-line condition (Edge Case Analyst P2, Contract Auditor P2, Structural Analyst P2; downgraded to P3)

Looks like an accidental side-effect of adding canViewAISettings to the condition list. The closing ) lost its matching { for the body. This is the only braceless multi-line if in this file. Restore the braces.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Generated by Coder Agents on behalf of @jakehwll

Fixed in 76cc7626. Restored the braces around the early return null; so the guard matches the file's other multi-line conditionals.

<PageHeaderTitle>Providers</PageHeaderTitle>
<PageHeaderSubtitle>
Connect third-party LLM services like OpenAI, Anthropic, or Google.
Each provider supplies models that users can select for their
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Subtitle mentions "Google" but the type selector only offers OpenAI, Anthropic, and Bedrock (Product Reviewer P2; downgraded to P3)

Both ProvidersPageView and AddProviderPageView say "Connect third-party LLM services like OpenAI, Anthropic, or Google." No Google option exists in the form. Quick copy fix.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Generated by Coder Agents on behalf of @jakehwll

Fixed in 76cc7626. Both subtitles now say "OpenAI, Anthropic, or Amazon Bedrock", matching the type selector.

@dannykopping dannykopping force-pushed the dk/aibridge-providers-db branch from 22dabfa to 62b664c Compare May 14, 2026 13:20
@jakehwll jakehwll force-pushed the jakehwll/ai-settings branch from e2cecfa to 8e17245 Compare May 14, 2026 13:40
@jakehwll jakehwll changed the base branch from dk/aibridge-providers-db to dk/aibridge-providers-handlers May 14, 2026 13:40
@jakehwll jakehwll force-pushed the jakehwll/ai-settings branch from 8e17245 to aa9f084 Compare May 14, 2026 13:41
@dannykopping dannykopping force-pushed the dk/aibridge-providers-handlers branch from 35194e7 to 893ed76 Compare May 14, 2026 14:12
@jakehwll jakehwll force-pushed the jakehwll/ai-settings branch from d602abb to 76cc762 Compare May 14, 2026 14:18
@jakehwll jakehwll changed the title feat(site): scaffold AI settings page with providers management UI feat(site): implement AI settings page with providers management UI May 15, 2026
@dannykopping dannykopping force-pushed the dk/aibridge-providers-handlers branch 5 times, most recently from ba81223 to 1bf6275 Compare May 15, 2026 16:07
@jakehwll jakehwll changed the title feat(site): implement AI settings page with providers management UI feat: implement AI settings page with providers management UI May 18, 2026
@dannykopping dannykopping force-pushed the dk/aibridge-providers-handlers branch 8 times, most recently from 08b59d3 to 31d4359 Compare May 18, 2026 14:46
jakehwll and others added 8 commits May 20, 2026 17:13
… CredentialField in ProviderForm and drop the unused CredentialFieldProps export
Rebases the frontend changes from #25328 onto current main.
All three dependency PRs (#24892, #24893, #24894) are already merged,
so this lands the UI layer cleanly.

- AISettingsLayout with sidebar, mounted under /ai/settings
- ProvidersPage list view + table of configured providers
- AddProviderPage and UpdateProviderPage with shared ProviderForm
- ProviderIcon and ProviderRow components with Storybook stories
- aiProviders query + CRUD mutation hooks
- viewAnyAIProvider permission against ai_provider RBAC resource
- Navbar deployment dropdown entry for AI settings
jakehwll and others added 12 commits May 21, 2026 06:05
Adds an unsaved-changes prompt to the AI provider add/update forms. The
prompt pairs a beforeunload listener (for tab close, refresh) with
useBlocker (for in-app navigation), reusing the ConfirmDialog
primitive. The form treats values as saved once the parent mutation
resolves without an error, so a successful submit does not re-trigger
the prompt.

Parents now await mutateAsync so the form's submitting state stays true
through the success redirect, suppressing the in-flight prompt.
- Create AISettingsSidebarView with AI Governance, Providers, and
  Manage Coder Agents entries
- Move AI routes to /ai/settings with dedicated AISettingsLayout
- Add AI entry to admin settings dropdown in navbar
- Remove AI items from deployment sidebar
- Switch provider pages to SettingsHeader for consistent alignment
- Streamline ProviderForm layout with side-by-side fields
- Move enable toggle from form to update page header
- Add model count display and disabled badge to update page
Add -ml-3 to offset the subtle button's px-3 padding so the arrow
icon sits flush with the text content below it.
Add validateOnMount so isValid reflects required-field state from the
start. Button stays disabled until the user has both changed something
and all required fields pass validation.
Remove provider-specific model counting logic and link to
/agents/settings/models instead.
Point the Providers entry in the agents settings sidebar to the new
AI settings page and add an ArrowUpRightIcon to indicate the
external navigation.
…dirty form after a successful save

React Query reports a settled mutation's error as null, so the
post-success effect's submitError === undefined check never matched.
The dirty baseline was never cleared and the unsaved-changes prompt
fired after a successful update. Switch to a truthy check that covers
both null and undefined, and add a storybook regression that drives
the full submit-then-navigate path.
Port from PR #25328: warn before leaving a dirty provider form via
useBlocker (in-app) and beforeunload (tab close). Parents switch to
mutateAsync so the submitting state stays true through navigation.
Also fixes the dirty-reset after save (null vs undefined check).
Brings Tracey's UI improvements from #25551 onto this branch:

- New top-level AI section at /ai/settings with its own layout and
  sidebar (AISettingsLayout, AISettingsSidebarView): AI Governance,
  Providers, and Manage Coder Agents entries.
- Navbar: 'AI' entry under the admin settings dropdown, gated on
  viewAnyAIProvider.
- Provider pages use SettingsHeader for consistent alignment with the
  AI Governance page; provider form fields use a side-by-side layout
  and the enable toggle moved from the form into the update page
  header alongside a disabled badge and model count.
- Deployment sidebar: AI Governance and Manage Coder Agents items
  removed (moved to the new AI sidebar).
- Submit button disabled until the form is dirty and valid.
- Status column with an Enabled badge.

Also restores getChatACL / updateChatACL on api.ts which were dropped
in Tracey's rebase, and removes a brittle storybook timing test that
no longer survives the new validateOnMount config.
Copy link
Copy Markdown
Contributor Author

🤖 This PR was modified by Coder Agents on behalf of Jake Howell.

Split into a 5-PR Graphite stack to keep reviews focused. Each layer is reviewable on its own and the stack ends with the same UX as this PR:

  1. feat(site): add UI primitives for the AI settings stack #25579 — primitives (FormField, PageHeader, AvatarData, icons, vite.config.mts)
  2. feat(site): add AI provider API client and query layer #25580 — API client and React Query layer + viewAnyAIProvider permission
  3. feat(site): add AI settings provider form components #25581 — provider form components and stories (ProviderForm, CredentialField, ProviderIcon, ProviderRow)
  4. feat(site): add AI settings providers pages and routes #25583 — pages and /ai/settings routes (Providers / Add / Update + AISettingsLayout)
  5. feat(site): promote AI settings to a top-level section #25582 — dashboard navigation reshuffle and /deployment/ai-governance/ai/settings/governance move

This PR is being kept open as a reference until the stack lands; Jake will close it manually once the stack merges.

@jakehwll
Copy link
Copy Markdown
Contributor Author

Closing as this has been stacked in multiple PRs 🙂

@jakehwll jakehwll closed this May 21, 2026
@github-actions github-actions Bot locked and limited conversation to collaborators May 21, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants