feat: bulk import user secrets from file#26640
Draft
dylanhuff-at-coder wants to merge 9 commits into
Draft
Conversation
Add POST /api/v2/users/{user}/secrets/batch to create many user
secrets from an uploaded env/json/yaml file. The import is atomic: any
validation, uniqueness, or per-user-limit failure rolls back the whole
batch and emits zero audit logs.
The file parser (codersdk.ParseSecretsFile) and the shared per-entry
validator (codersdk.ValidateCreateUserSecretRequest) live in codersdk so
a future `coder secret` CLI can reuse them with no changes.
ParseSecretsFile parsed YAML with yaml.Unmarshal, which decodes only the first document and silently discards everything after a '---' separator. A multi-document secrets file would import the first document's secrets and drop the rest while still returning a success response, so secrets could go missing without warning. Decode with a yaml.Decoder and reject any subsequent content-bearing document, mirroring the JSON parser's trailing-data rejection. A bare trailing '---' (or comments-only tail) decodes to a null document and is still allowed.
…on limit rollback (PLAT-240)
…ort ordering (PLAT-240)
Docs preview📖 View docs preview for |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds bulk secret import so a user can create many secrets at once from a
.env,.json, or.yaml/.ymlfile instead of one at a time.Parsing and validation run server-side behind
POST /api/v2/users/{user}/secrets/batch. The whole batch is inserted in a single transaction, so if any entry is invalid, conflicts, or trips a per-user limit, nothing is created and the response reports every problem with its entry index (and line, for.env). The file parser and per-entry validators are exported fromcodersdkso a futurecoder secretCLI import can reuse them; no CLI is added here.The Add secret dialog gains an upload dropzone on create (edit is unchanged), wired to the new endpoint, with Storybook coverage for success, partial-failure, and unsupported-file cases.
Closes https://linear.app/codercom/issue/PLAT-240
Implementation context
Endpoint and shape
POST /api/v2/users/{user}/secrets/batchwith body{ "format": "env" | "json" | "yaml", "content": "<raw file text>" }. The client sends raw bytes and an explicit format discriminator (the UI derives it from the file extension); the backend owns all parsing so the same endpoint can serve a future CLI. Registered in the existing/secretsgroup, so it inherits identical auth/middleware as single-create (user-scoped,meresolution, dbauthz RBAC). Responds201with the created[]UserSecret(metadata only, never values).Parser semantics (
codersdk/usersecretsimport.go)KEYbecomes a secret withname = env_name = KEY,value = VALUE(env-injected, matching the primary use case).file_path/descriptionare empty..env: full-line#comments, blank lines, optionalexportprefix, single and double quotes (double-quote escapes\n \t \r \\ \"), split on the first=(=inside values kept), CRLF normalization, BOM stripping, surrounding-whitespace trim for unquoted values, literal#preserved in unquoted values, and unicode values..json/.yaml: flat{ "KEY": "VALUE" }string maps only; source order is preserved; non-object/non-string/nested values are rejected.400: empty/all-comment file, unknown/empty format, malformed JSON/YAML, intra-file duplicate keys (sincename == env_name == KEYandfile_pathis empty, one duplicate-key scan covers duplicate names/env_names/file_paths), and content larger than a 1 MiB cap (DoS guard).Atomicity, errors, audit
database.Store.InTx. The per-user-limits Postgres trigger fires per row; any uniqueness/validation/limit failure aborts the transaction so nothing is created.400), each tagged with per-entry context (secrets[i].field). Uniqueness conflicts map to409and per-user-limit trigger violations to400, reusing the single-create helpers.createaudit log per secret (after commit); a rolled-back batch produces zero rows and zero audit logs. Secret values are never logged or returned.Reuse for a future CLI (no CLI here)
codersdkexportsParseSecretsFile,ValidateCreateUserSecretRequest,SecretsFileFormat(+ consts),ImportUserSecretsRequest,MaxSecretsFileBytes, and theClient.ImportUserSecretsmethod. The package stays dependency-light (stdlib +xerrors+yaml.v3), so a CLI can reuse the parser and validators unchanged.Frontend (create only)
FileUploaddropzone plus an "or add individually" divider render at the top of the create branch only; the edit branch is untouched. The browser reads the file text, derives the format from the extension, calls a newimportUserSecretsmutation, invalidates the secrets query, toasts success, and closes. Backend per-entry errors are surfaced in an alert (field path + detail) so the user sees which entry failed.Test matrix
.envedge cases (comments, blanks,export/non-export, quotes + escapes,=in value, CRLF, BOM, whitespace/tabs, unicode, literal#, missing=/unterminated quote with line, duplicate-cites-line);.json(flat map, malformed, non-object, number/bool/null/nested rejected, duplicate, trailing data);.yaml(flat map, malformed, non-mapping, non-string scalar, nested, duplicate, multi-document rejection, alias-bomb rejection); unknown/empty format, oversized, empty/all-comments, and mapping equivalence across all three formats./reject the whole batch (zero rows, zero audit logs); conflict with an existing secret returns409; count, total-bytes, and env-bytes caps each roll back fully (400, zero rows, zero audit logs); duplicate-within-file rejected before insert; values absent from responses.Decisions and assumptions
{ "KEY": "VALUE" }map (no richer{name,value,env_name,file_path,description}object form), for parity with.envand the env-injection use case. If the design intends the richer object form, it is a small follow-up since the parser already producesCreateUserSecretRequestvalues.#in unquoted.envvalues is kept literally (no inline-comment stripping) to avoid silently truncating secret values; quote the value to include trailing spaces.PORT: "8080"(numbers/bools/null are rejected rather than coerced). Multi-document YAML is rejected rather than silently importing only the first document.secrets[i].field); structural.envparse errors also cite the source line.