fix(database-integrations): throw typed error for malformed integrations YAML - #425

Fixes #424

Problem

mergeApiIntegrationsIntoYaml(existingContent, apiIntegrations) failed with the cryptic Error: Document with errors cannot be stringified when existingContent was readable-but-malformed YAML — most commonly a file left with unresolved git merge conflict markers.

yaml.parseDocument() never throws; it returns a Document with .errors populated. So the ?? createNewDocument() fallback never fired, the errored document flowed through the whole merge, and the failure only surfaced at serialize time — with no indication of which file or which line was broken.

The same defect reached the CLI through readIntegrationsDocument, affecting integrations pull, add and edit. pull and add hit the serialize-time crash; edit failed earlier with a misleading semantic error (Integration with ID "…" not found), because its isSeq-guarded helpers silently returned empty on the errored document.

Approach

Fail fast at parse time behind a typed, exported error. No silent self-healing — the user is shown the path and the parse detail, and recovery is manual.

• parseIntegrationsDocument throws IntegrationsYamlParseError when doc.errors is non-empty. Contract is now: null for blank, Document for valid, throws for malformed.
• The CLI wraps it as MalformedIntegrationsFileError, carrying the file path plus the parse detail (line, column, code frame), and maps it to exit code 2 (invalid usage) in all three commands.
• pull now reads the local file before the empty-response early return, so a malformed file is caught on every run — not just when the workspace has integrations to pull.

Both error classes are exported so downstream consumers (webapp git-sync, the VS Code extension) can catch them and rebuild via mergeApiIntegrationsIntoYaml(null, apiIntegrations) if they'd rather discard a corrupt file than surface the error.

Before / after

$ deepnote integrations edit some-id --file broken.yaml

# before
Error: Document with errors cannot be stringified          (exit 1)

# after                                                     (exit 2)
Invalid YAML in integrations file: broken.yaml

Implicit keys need to be on a single line at line 2, column 1:

integrations:
<<<<<<< HEAD
^

This usually comes from unresolved merge conflict markers (<<<<<<<, =======, >>>>>>>) or a manual-edit typo.
Open the file, fix the reported line, and re-run the command — or delete the file manually if you no longer need its contents.

Why doc.errors is the right gate

This moves a failure earlier on a shared code path, so it's worth being precise about the blast radius.

Document.toString() refuses on if (this.errors.length > 0) (yaml/dist/doc/Document.js:322) — the same predicate as the new gate. The set of documents that now throw at parse time is therefore identical by construction to the set that already threw at serialize time. Nothing that works today starts failing; a cryptic late failure becomes an actionable early one.

Verified empirically across the boundary cases: duplicate keys, tab-as-indent and bad scalar starts all already threw at serialize time (so they improve), while CRLF, trailing tabs and postgres://h:5432/db produce zero errors and are untouched. Warnings are deliberately not gated on — TAG_RESOLVE_FAILED (e.g. a: !custom v) lands in doc.warnings and stringifies fine; gating on those would have been a real regression.

One intentional behavior change: pull against a zero-integration workspace with a malformed local file previously exited 0 without ever reading the file. It now exits 2. This is deliberate, tested, and documented.

Bonus fix: updateDotEnv previously ran before the failing serialize, so a malformed file could leave a half-written .env behind. The relocated read closes that window — the new tests assert .env is never created.

Tests

26 new tests. mergeApiIntegrationsIntoYaml had zero coverage before this, so it gets baseline coverage alongside the regression cases.

• integrations-document.test.ts (new) — blank → null, valid → round-trips, conflict markers and a: b: c → typed error exposing .errors and line info.
• merge-into-yaml.test.ts (new) — fresh-document and comment-preserving merge snapshots, both malformed fixtures, InvalidIntegrationsTypeError, and a test that executes the documented catch-and-rebuild recovery recipe. One test asserts the message does not contain Document with errors cannot be stringified, pinning the actual regression.
• CLI — readIntegrationsDocument unit coverage, plus end-to-end cases for all three commands asserting exit code 2, the YAML file byte-identical afterwards, and .env never created. The edit action test pins both branches of the new ternary (malformed → 2, missing file → 1).

Known gap (pre-existing, not addressed here)

An unresolved YAML alias (password: *missing) produces errors: 0, warnings: 0 yet still throws at toString(). Same shape as this issue — late crash, cryptic message, partial .env write — but a different trigger, and behavior is identical before and after this change. Worth a follow-up issue; out of scope here.

Verification

Docs updated per AGENTS.md: new skills/deepnote/references/cli-integrations.md (registered in SKILL.md) and a note under integrations pull in the CLI README.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PyawjygDS8h5qdX1ZWJoRH

Summary by CodeRabbit

• Bug Fixes

  • Improved handling of malformed local integrations YAML (including unresolved merge conflicts) with clearer recovery guidance.
  • integrations pull|add|edit now consistently exit with code 2 and preserve the integrations YAML and any secret .env file (no partial writes).

• Documentation

  • Updated integrations pull/add/edit docs with new defaults, shared options, interactive behavior notes, and explicit invalid-YAML guarantees.
  • Added a dedicated CLI “Integrations commands” reference and updated the CLI quick-reference table.

• Tests

  • Expanded coverage for invalid YAML parsing, error details, exit codes, and “leave files untouched” behavior.