Skip to content

feat: add bulk user secret import endpoint and SDK client (PLAT-240)#26724

Merged
dylanhuff-at-coder merged 10 commits into
mainfrom
dylan/plat-240-secrets-import-endpoint
Jul 23, 2026
Merged

feat: add bulk user secret import endpoint and SDK client (PLAT-240)#26724
dylanhuff-at-coder merged 10 commits into
mainfrom
dylan/plat-240-secrets-import-endpoint

Conversation

@dylanhuff-at-coder

@dylanhuff-at-coder dylanhuff-at-coder commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Adds POST /api/v2/users/{user}/secrets/batch and codersdk.Client.ImportUserSecrets to import env, JSON, or YAML secrets atomically. The endpoint validates each entry, rolls back the full batch on conflicts or limits, omits secret values from responses and audit logs, and imports keys that cannot be injected as environment variables with an empty env_name.

Part of the PLAT-240 bulk secret import stack. Reviewed and updated by Coder Agents on behalf of @dylanhuff-at-coder.

@linear-code

linear-code Bot commented Jun 25, 2026

Copy link
Copy Markdown

PLAT-240

@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown

Docs preview

Check off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here.

dylanhuff-at-coder commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

@github-actions github-actions Bot added the stale This issue is like stale bread. label Jul 3, 2026
@github-actions github-actions Bot closed this Jul 7, 2026
@dylanhuff-at-coder
dylanhuff-at-coder force-pushed the dylan/plat-240-secrets-import-endpoint branch from 7e85c27 to a0e3321 Compare July 21, 2026 15:31
@dylanhuff-at-coder dylanhuff-at-coder removed the stale This issue is like stale bread. label Jul 21, 2026
@dylanhuff-at-coder
dylanhuff-at-coder force-pushed the dylan/plat-240-secrets-import-endpoint branch 2 times, most recently from 9f5cad9 to df4f2be Compare July 21, 2026 21:42
Base automatically changed from dylan/plat-240-secrets-import-codersdk to main July 22, 2026 13:23
Add POST /api/v2/users/{user}/secrets/batch to create many user secrets
from an uploaded env/json/yaml file. The batch is inserted in one
transaction, so any validation, uniqueness, or per-user-limit failure
rolls back everything and emits zero audit logs. Adds the codersdk
ImportUserSecrets client and the regenerated API docs.
Previously ParseSecretsFile set EnvName unconditionally to the file key
for every parsed entry. This caused keys that are not valid POSIX env
names (e.g. MY-TOKEN, my.key) or that are reserved (e.g. PATH) to fail
ValidateCreateUserSecretRequest on the env_name field, rolling back the
entire atomic batch import. Common real-world .env files typically
contain such keys.

Change EnvName assignment to best-effort: only set it when the key
passes UserSecretEnvNameValid; otherwise leave EnvName empty. The secret
is still imported, just without env injection. The env_name uniqueness
index is a partial index (WHERE env_name != ''), so multiple empty
env_names are fine.

Update tests:
- Add TestParseSecretsFileEnvBestEffortEnvName in codersdk with cases
  for hyphenated key (MY-TOKEN), reserved name (PATH), dot key (my.key),
  and a valid key (MY_TOKEN).
- Remove ReservedEnvName (PATH) from the rollback failure cases in
  coderd/usersecretsimport_test.go; add
  TestImportUserSecretsReservedEnvNameBestEffort to assert the new
  success behavior (secret created, env_name empty).
@dylanhuff-at-coder
dylanhuff-at-coder force-pushed the dylan/plat-240-secrets-import-endpoint branch from c309123 to 9627726 Compare July 22, 2026 13:28
@dylanhuff-at-coder
dylanhuff-at-coder marked this pull request as ready for review July 22, 2026 14:47
@coder-tasks

coder-tasks Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Documentation Check

This PR adds a public API endpoint (POST /api/v2/users/{user}/secrets/batch) and the codersdk.Client.ImportUserSecrets client for bulk secret import. The auto-generated API reference (docs/reference/api/secrets.md, docs/reference/api/schemas.md) is regenerated and included, so no action is needed there.

New Documentation Needed

  • docs/user-guides/user-secrets.md - Add a "Bulk import" section covering the POST /secrets/batch endpoint and codersdk.Client.ImportUserSecrets: the env/json/yaml formats, the atomic all-or-nothing rollback, reserved/invalid env-name handling (empty env_name), and a link to the "Import user secrets from a file" operation in the Secrets API reference.

    ⚠️ A "Bulk import" section was added in an earlier push but has been reverted in the latest push (ea62f71). The endpoint and SDK still ship in this PR, so the prose is needed again. No changes to docs/user-guides/user-secrets.md are present in the current diff.


Automated review via Coder Agents

@dylanhuff-at-coder
dylanhuff-at-coder marked this pull request as draft July 22, 2026 14:57
@dylanhuff-at-coder
dylanhuff-at-coder marked this pull request as ready for review July 22, 2026 17:59
Comment thread docs/user-guides/user-secrets.md Outdated
Comment thread docs/user-guides/user-secrets.md Outdated

@nickvigilante nickvigilante left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs LGTM! Please get another review on the Go code, but that also looked good to me, too.

Comment thread docs/user-guides/user-secrets.md Outdated
For full command details, see [`coder secret`](../reference/cli/secret.md) and
the [Secrets API reference](../reference/api/secrets.md).

## Bulk import

@nickvigilante nickvigilante Jul 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Content/audience question (picking up @matifali's open thread on this file): this section lives in a user guide but reads in reference-doc register — it leads with a raw POST /api/v2/users/{user}/secrets/batch route and the codersdk.Client.ImportUserSecrets Go method. A user-guide reader is more likely to expect a CLI or UI entry point, with the wire/SDK details left to the API reference (which already documents this endpoint).

Given #26725 sits on top of this in the stack and may add the user-facing surface, a couple of options:

  • Reframe this around the user-facing entry point (CLI/UI) and link the API reference for the wire format, or
  • Hold this user-guide section until that surface lands, keeping only the API-reference coverage for now.

The technical content itself is accurate — formats, atomic rollback, and the empty-env_name behavior for reserved/invalid keys all match the implementation. This is purely about placement/audience.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Content note (following up on @matifali's thread): with the upload UI in #26725 closed/stale and no CLI in the stack, this API/SDK path is currently the only way to bulk-import — so keeping it documented here is the right call, and I'd keep the section. Placement isn't a concern either, since docs IA is being restructured separately.

One optional polish for the user-guide audience: a short task-oriented lead-in (when/why you'd reach for bulk import) plus a small example payload — a few lines of env content and the matching request body — would make it read a bit less like a route-and-method pointer. Not blocking. The technical content (formats, atomic rollback, empty env_name for reserved/invalid keys) is accurate as-is.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I will be reviving #26725, it was just opened a while ago. Once this is merged that will be reopened :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up pulling the section from this page entirely per Atif's request to keep reference docs off the user guide. I'll bring the task-oriented version (lead-in + example) back with the UI PR.

Comment thread coderd/usersecrets.go
Comment thread coderd/httpapi/httpapi.go

err := json.NewDecoder(r.Body).Decode(value)
if err != nil {
if _, ok := errors.AsType[*http.MaxBytesError](err); ok {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From Coder Agents:

FYI for reviewers (not a change request): this teaches the shared Read helper to return 413 on *http.MaxBytesError, which is a strict improvement over the old behavior (an oversized body previously surfaced as 400 "Request body must be valid JSON."). Flagging only that it's cross-cutting — the new status now applies to every endpoint that wraps its body in http.MaxBytesReader, not just the new batch import. That looks intentional and it's covered by the new TestRead/BodyTooLarge case; noting it so the shared-behavior change is a conscious sign-off rather than an incidental side effect.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, intentional. 413 is the right status for an oversized body, and it only affects endpoints already using http.MaxBytesReader 👍

Comment thread coderd/httpapi/httpapi.go

err := json.NewDecoder(r.Body).Decode(value)
if err != nil {
if _, ok := errors.AsType[*http.MaxBytesError](err); ok {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL of this syntax vs errors.As

// ValidateCreateUserSecretRequest.
// ValidateCreateUserSecretRequest. EnvName is set only when the key passes
// env-name validation (best-effort; keys like MY-TOKEN or PATH get an empty
// EnvName so they are still imported without env injection).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It won't surprise the user, right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we enforce that validation across the User Secrets interfaces

return count%2 == 1
}

func quotedInner(v string, quote byte) (string, bool) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not part of this PR, but agent flagged a bug:

Bug: single-quoted values silently swallow trailing data

quotedInner (codersdk/usersecretsimport.go:225) doesn't find the first closing quote like the double-quote path does. It just trims trailing whitespace and checks
whether the last character of the line is a quote, then strips first-and-last:

func quotedInner(v string, quote byte) (string, bool) {
      trimmed := strings.TrimRight(v, " \t")
      if len(trimmed) < 2 || trimmed[len(trimmed)-1] != quote {
              return "", false
      }
      return trimmed[1 : len(trimmed)-1], true
}

Actual behavior:

┌────────────────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│       Input        │                                                  Result                                                   │
├────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ KEY='abc' # 'note' │ value = abc' # 'note ⚠️  silent                                                                            │
├────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ KEY='a'b'          │ value = a'b ⚠️  silent                                                                                     │
├────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ KEY='a' 'b'        │ value = a' 'b ⚠️  silent                                                                                   │
├────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ KEY='abc' extra    │ error, but message says "missing closing single quote" (wrong; the quote is closed, the data is trailing) │
└────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll create a follow up PR for this shortly

@dylanhuff-at-coder
dylanhuff-at-coder merged commit d5a3963 into main Jul 23, 2026
33 checks passed
@dylanhuff-at-coder
dylanhuff-at-coder deleted the dylan/plat-240-secrets-import-endpoint branch July 23, 2026 21:55
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 23, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants