Skip to content

feat: bulk import user secrets from file#26640

Draft
dylanhuff-at-coder wants to merge 9 commits into
mainfrom
dylan/plat-240-add-bulk-uploading-of-secrets-via-file
Draft

feat: bulk import user secrets from file#26640
dylanhuff-at-coder wants to merge 9 commits into
mainfrom
dylan/plat-240-add-bulk-uploading-of-secrets-via-file

Conversation

@dylanhuff-at-coder

Copy link
Copy Markdown
Contributor

Adds bulk secret import so a user can create many secrets at once from a .env, .json, or .yaml/.yml file instead of one at a time.

Parsing and validation run server-side behind POST /api/v2/users/{user}/secrets/batch. The whole batch is inserted in a single transaction, so if any entry is invalid, conflicts, or trips a per-user limit, nothing is created and the response reports every problem with its entry index (and line, for .env). The file parser and per-entry validators are exported from codersdk so a future coder secret CLI import can reuse them; no CLI is added here.

The Add secret dialog gains an upload dropzone on create (edit is unchanged), wired to the new endpoint, with Storybook coverage for success, partial-failure, and unsupported-file cases.

Closes https://linear.app/codercom/issue/PLAT-240

Implementation context

Endpoint and shape

  • POST /api/v2/users/{user}/secrets/batch with body { "format": "env" | "json" | "yaml", "content": "<raw file text>" }. The client sends raw bytes and an explicit format discriminator (the UI derives it from the file extension); the backend owns all parsing so the same endpoint can serve a future CLI. Registered in the existing /secrets group, so it inherits identical auth/middleware as single-create (user-scoped, me resolution, dbauthz RBAC). Responds 201 with the created []UserSecret (metadata only, never values).

Parser semantics (codersdk/usersecretsimport.go)

  • Flat mapping for all formats: each KEY becomes a secret with name = env_name = KEY, value = VALUE (env-injected, matching the primary use case). file_path/description are empty.
  • .env: full-line # comments, blank lines, optional export prefix, single and double quotes (double-quote escapes \n \t \r \\ \"), split on the first = (= inside values kept), CRLF normalization, BOM stripping, surrounding-whitespace trim for unquoted values, literal # preserved in unquoted values, and unicode values.
  • .json / .yaml: flat { "KEY": "VALUE" } string maps only; source order is preserved; non-object/non-string/nested values are rejected.
  • Structural errors return 400: empty/all-comment file, unknown/empty format, malformed JSON/YAML, intra-file duplicate keys (since name == env_name == KEY and file_path is empty, one duplicate-key scan covers duplicate names/env_names/file_paths), and content larger than a 1 MiB cap (DoS guard).

Atomicity, errors, audit

  • All rows are inserted inside one database.Store.InTx. The per-user-limits Postgres trigger fires per row; any uniqueness/validation/limit failure aborts the transaction so nothing is created.
  • Validation runs up front and returns every problem at once (400), each tagged with per-entry context (secrets[i].field). Uniqueness conflicts map to 409 and per-user-limit trigger violations to 400, reusing the single-create helpers.
  • A successful batch emits one create audit log per secret (after commit); a rolled-back batch produces zero rows and zero audit logs. Secret values are never logged or returned.

Reuse for a future CLI (no CLI here)

  • codersdk exports ParseSecretsFile, ValidateCreateUserSecretRequest, SecretsFileFormat (+ consts), ImportUserSecretsRequest, MaxSecretsFileBytes, and the Client.ImportUserSecrets method. The package stays dependency-light (stdlib + xerrors + yaml.v3), so a CLI can reuse the parser and validators unchanged.

Frontend (create only)

  • A FileUpload dropzone plus an "or add individually" divider render at the top of the create branch only; the edit branch is untouched. The browser reads the file text, derives the format from the extension, calls a new importUserSecrets mutation, invalidates the secrets query, toasts success, and closes. Backend per-entry errors are surfaced in an alert (field path + detail) so the user sees which entry failed.

Test matrix

  • Parser table tests: .env edge cases (comments, blanks, export/non-export, quotes + escapes, = in value, CRLF, BOM, whitespace/tabs, unicode, literal #, missing =/unterminated quote with line, duplicate-cites-line); .json (flat map, malformed, non-object, number/bool/null/nested rejected, duplicate, trailing data); .yaml (flat map, malformed, non-mapping, non-string scalar, nested, duplicate, multi-document rejection, alias-bomb rejection); unknown/empty format, oversized, empty/all-comments, and mapping equivalence across all three formats.
  • Endpoint tests against real Postgres: happy path creates all + exactly N audit logs; reserved env name / empty value / oversized value / name with / reject the whole batch (zero rows, zero audit logs); conflict with an existing secret returns 409; count, total-bytes, and env-bytes caps each roll back fully (400, zero rows, zero audit logs); duplicate-within-file rejected before insert; values absent from responses.
  • Frontend: pure-logic vitest for the filename-to-format helper, plus Storybook play stories for successful import, partial-failure error surfacing, and unsupported-file.

Decisions and assumptions

  • The Figma was not reachable from the implementation environment, so the create-form layout follows the issue's screenshot description (upload dropzone, divider "or add individually", then the existing fields).
  • JSON/YAML are kept to the flat { "KEY": "VALUE" } map (no richer {name,value,env_name,file_path,description} object form), for parity with .env and the env-injection use case. If the design intends the richer object form, it is a small follow-up since the parser already produces CreateUserSecretRequest values.
  • Inline # in unquoted .env values is kept literally (no inline-comment stripping) to avoid silently truncating secret values; quote the value to include trailing spaces.
  • YAML scalar values must be strings, so PORT: "8080" (numbers/bools/null are rejected rather than coerced). Multi-document YAML is rejected rather than silently importing only the first document.
  • Per-entry error context uses the entry index (secrets[i].field); structural .env parse errors also cite the source line.

This PR was generated by Coder Agents on behalf of @dylanhuff-at-coder.

Add POST /api/v2/users/{user}/secrets/batch to create many user
secrets from an uploaded env/json/yaml file. The import is atomic: any
validation, uniqueness, or per-user-limit failure rolls back the whole
batch and emits zero audit logs.

The file parser (codersdk.ParseSecretsFile) and the shared per-entry
validator (codersdk.ValidateCreateUserSecretRequest) live in codersdk so
a future `coder secret` CLI can reuse them with no changes.
ParseSecretsFile parsed YAML with yaml.Unmarshal, which decodes only the
first document and silently discards everything after a '---' separator.
A multi-document secrets file would import the first document's secrets
and drop the rest while still returning a success response, so secrets
could go missing without warning.

Decode with a yaml.Decoder and reject any subsequent content-bearing
document, mirroring the JSON parser's trailing-data rejection. A bare
trailing '---' (or comments-only tail) decodes to a null document and is
still allowed.
@linear-code

linear-code Bot commented Jun 23, 2026

Copy link
Copy Markdown

PLAT-240

@github-actions

Copy link
Copy Markdown

Docs preview

📖 View docs preview for docs/reference/api/schemas.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant