feat(local-runner): add notebook orchestration pipelines - #435
feat(local-runner): add notebook orchestration pipelines#435jamesbhobbs wants to merge 20 commits into
Conversation
📝 WalkthroughWalkthroughThe change adds imperative and durable notebook orchestration APIs with local/cloud execution, dependency graphs, retries, recovery, output helpers, and streamed NDJSON responses. It adds regional sales pipeline examples with quality gates, provider reviews, arbitration, and Workflow SDK persistence. It also adds a static pipeline gallery with persisted manifests, notebook snapshots, interactive replay, routes, tests, and deployment documentation. Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant Browser
participant StaticServer
participant Orchestrator
participant DeepnoteCloud
participant WorkflowSDK
Browser->>StaticServer: Start orchestration request
StaticServer->>Orchestrator: Run regional pipeline
Orchestrator->>DeepnoteCloud: Execute regional notebooks
Orchestrator->>DeepnoteCloud: Execute provider reviews and arbiter
StaticServer-->>Browser: Stream events and result
Browser->>WorkflowSDK: Start durable workflow
WorkflowSDK->>DeepnoteCloud: Execute durable notebook steps
WorkflowSDK-->>Browser: Return run status and result
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (5 passed)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/local-runner/README.md (1)
183-191: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winSnippet calls
orchestratewithout importing it.📝 Fix
-import { serveStatic } from "`@deepnote/local-runner`"; +import { orchestrate, serveStatic } from "`@deepnote/local-runner`";🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/local-runner/README.md` around lines 183 - 191, Update the README TypeScript example using serveStatic to import orchestrate from the appropriate package or module before it is called in orchestrationRunner, while preserving the existing buildPipeline and onEvent wiring.
🧹 Nitpick comments (2)
packages/local-runner/src/orchestrate.ts (1)
300-306: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueIdentity check for status-callback dedup is subtle.
stepPoll?.onStatus !== inheritedPoll?.onStatusonly dedups when the same function reference is passed at both levels. Fine today, but a comment (or aSetof unique callbacks) would spare the next reader.♻️ Optional tidy
onStatus: (status, run) => { - inheritedPoll?.onStatus?.(status, run) - if (stepPoll?.onStatus !== inheritedPoll?.onStatus) { - stepPoll?.onStatus?.(status, run) - } + // A step may reuse the inherited callback reference; notify each distinct one once. + for (const onStatus of new Set([inheritedPoll?.onStatus, stepPoll?.onStatus].filter(Boolean))) { + onStatus?.(status, run) + } emit({ type: 'step_status', stepId: id, target: 'cloud', status }) },🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/local-runner/src/orchestrate.ts` around lines 300 - 306, Clarify the callback deduplication in the onStatus handler by documenting that the identity comparison prevents invoking the same function reference twice, or use a Set-based unique-callback approach if consistent with the surrounding code. Preserve inherited callback invocation, step-specific callback invocation for distinct references, and the step_status emission.examples/local-runner/workflow-orchestration/api/run.post.ts (1)
5-14: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winUnvalidated request body cast to
SalesDecisionRequest.
req.json().catch(() => ({}))is cast directly without runtime validation. Malformed field types (e.g., a stringifieddemandShockPct) will pass straight intosalesDecisionWorkflowand silently corrupt downstream numeric calculations instead of failing fast with a 400.As per coding guidelines, "**/*.{ts,tsx}: Use strict TypeScript type checking, prefer type safety over convenience... avoid `any` in favor of proper type definitions."♻️ Suggested validation before starting the workflow
+import { z } from 'zod' + +const requestSchema = z.object({ + demandShockPct: z.number().optional(), + qualityThreshold: z.number().optional(), + simulateFailureRegion: z.string().nullable().optional(), +}) + export default defineEventHandler(async ({ req }) => { - const body = (await req.json().catch(() => ({}))) as SalesDecisionRequest + const body = requestSchema.parse(await req.json().catch(() => ({}))) const run = await start(salesDecisionWorkflow, [body])🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@examples/local-runner/workflow-orchestration/api/run.post.ts` around lines 5 - 14, Validate the parsed body at runtime before passing it to start in the request handler, rather than directly casting it to SalesDecisionRequest. Reject malformed or incorrectly typed fields, including numeric fields such as demandShockPct, with an HTTP 400 response, and only invoke salesDecisionWorkflow after validation succeeds.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/local-runner/orchestration/README.md`:
- Around line 3-4: Update the pipeline description in the README to clarify that
the two source notebooks fan out in parallel during local execution, while cloud
execution runs them sequentially to avoid create-if-missing races. Preserve the
existing description of their output feeding the third notebook and its agent
block.
In `@examples/local-runner/orchestration/run.mjs`:
- Around line 9-11: Update the process.loadEnvFile error handling to ignore only
missing-file (ENOENT) errors; rethrow all other errors so unexpected .env
loading failures are not swallowed and do not silently switch execution to local
mode.
In `@examples/local-runner/README.md`:
- Around line 3-22: Update the stale sentence immediately following the examples
table to reflect all four listed examples, specifically describing that the run
app and snapshot viewer use two committed artifacts; preserve the surrounding
guidance and avoid implying that only two examples exist.
In `@examples/local-runner/run-app/README.md`:
- Around line 67-93: Update the “Run the pipeline” description and live-graph
diagram to show fan-out to GPT-5.5 and Claude Sonnet 5, followed by the Deepnote
Auto arbiter, instead of a single agent decision or final memo. Revise the
provider/arbiter execution paragraph to match serve.mjs: without DEEPNOTE_TOKEN,
those notebooks are skipped and report that they run in Deepnote Cloud; do not
claim a local OPENAI_API_KEY-driven agent step. Preserve the existing
descriptions of regional recovery, aggregation, and notebook links.
In `@examples/local-runner/workflow-orchestration/package.json`:
- Around line 11-17: Regenerate the workspace lockfile to include the
dependencies declared in the workflow-orchestration package manifest. Run pnpm
install from the repository root, then commit the updated pnpm-lock.yaml so pnpm
install --frozen-lockfile succeeds in CI.
In `@examples/local-runner/workflow-orchestration/workflows/sales-report.ts`:
- Around line 191-237: Update regionalResult and its caller runRegion so missing
RESULT_MARKER or invalid JSON produces a null RegionalResult instead of
throwing, allowing successful steps with malformed output to reach
needsRecovery. Preserve valid-output parsing and add coverage in
sales-report.test.ts for a success: true step with bad or missing output.
In `@packages/local-runner/src/serve-static.ts`:
- Around line 196-207: Update the local request handler’s sendFrame function to
return without writing when the response has ended, finished, or been destroyed.
Keep all event, result, and error frame behavior unchanged while ensuring late
orchestrationRunner emissions cannot write to a closed response or throw outside
the try block.
---
Outside diff comments:
In `@packages/local-runner/README.md`:
- Around line 183-191: Update the README TypeScript example using serveStatic to
import orchestrate from the appropriate package or module before it is called in
orchestrationRunner, while preserving the existing buildPipeline and onEvent
wiring.
---
Nitpick comments:
In `@examples/local-runner/workflow-orchestration/api/run.post.ts`:
- Around line 5-14: Validate the parsed body at runtime before passing it to
start in the request handler, rather than directly casting it to
SalesDecisionRequest. Reject malformed or incorrectly typed fields, including
numeric fields such as demandShockPct, with an HTTP 400 response, and only
invoke salesDecisionWorkflow after validation succeeds.
In `@packages/local-runner/src/orchestrate.ts`:
- Around line 300-306: Clarify the callback deduplication in the onStatus
handler by documenting that the identity comparison prevents invoking the same
function reference twice, or use a Set-based unique-callback approach if
consistent with the surrounding code. Preserve inherited callback invocation,
step-specific callback invocation for distinct references, and the step_status
emission.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: c65f1346-5285-40df-a4e5-385cdfada93f
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (36)
examples/local-runner/README.mdexamples/local-runner/orchestration/README.mdexamples/local-runner/orchestration/run.mjsexamples/local-runner/run-app/README.mdexamples/local-runner/run-app/decision-arbiter.deepnoteexamples/local-runner/run-app/decision-claude.deepnoteexamples/local-runner/run-app/decision-gpt.deepnoteexamples/local-runner/run-app/index.htmlexamples/local-runner/run-app/regional-analysis.deepnoteexamples/local-runner/run-app/serve.mjsexamples/local-runner/workflow-orchestration/.gitignoreexamples/local-runner/workflow-orchestration/README.mdexamples/local-runner/workflow-orchestration/api/run.post.tsexamples/local-runner/workflow-orchestration/api/runs/[runId].get.tsexamples/local-runner/workflow-orchestration/notebooks/executive-decision.deepnoteexamples/local-runner/workflow-orchestration/notebooks/regional-sales-analysis.deepnoteexamples/local-runner/workflow-orchestration/package.jsonexamples/local-runner/workflow-orchestration/run-demo.mjsexamples/local-runner/workflow-orchestration/tsconfig.jsonexamples/local-runner/workflow-orchestration/vite.config.tsexamples/local-runner/workflow-orchestration/workflows/deepnote.tsexamples/local-runner/workflow-orchestration/workflows/sales-report.test.tsexamples/local-runner/workflow-orchestration/workflows/sales-report.tspackage.jsonpackages/local-runner/README.mdpackages/local-runner/package.jsonpackages/local-runner/src/index.tspackages/local-runner/src/orchestrate.test.tspackages/local-runner/src/orchestrate.tspackages/local-runner/src/serve-static.test.tspackages/local-runner/src/serve-static.tspackages/local-runner/src/workflows/index.tspackages/local-runner/src/workflows/run-notebook-step.test.tspackages/local-runner/src/workflows/run-notebook-step.tspackages/local-runner/tsdown.config.tspnpm-workspace.yaml
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #435 +/- ##
==========================================
+ Coverage 87.78% 87.81% +0.03%
==========================================
Files 187 189 +2
Lines 9927 10194 +267
Branches 2768 2912 +144
==========================================
+ Hits 8714 8952 +238
- Misses 1212 1241 +29
Partials 1 1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Addressed all review feedback in
All CI checks are green across Node 22–25 and the repo default. All seven inline threads are resolved. @coderabbitai review |
|
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
examples/local-runner/workflow-orchestration/run-api.test.ts (1)
7-7: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueCo-locate the endpoint test.
Move this to
examples/local-runner/workflow-orchestration/api/run.post.test.tsand import./run.post.As per coding guidelines,
**/*.test.{ts,tsx}must place tests next to their source files.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@examples/local-runner/workflow-orchestration/run-api.test.ts` at line 7, Move the test file to the api directory alongside the run.post endpoint, and update its import to reference ./run.post from the new location. Preserve the existing test behavior while complying with the required test co-location convention.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/local-runner/workflow-orchestration/run-api.test.ts`:
- Line 26: Update the assertion for startMock in the relevant test to verify it
was called once with salesDecisionWorkflow and the validated request argument,
rather than only checking invocation count. Ensure the test fully covers the
start invocation contract for the new workflow orchestration behavior.
In `@examples/local-runner/workflow-orchestration/workflows/sales-report.test.ts`:
- Around line 151-159: Add a negative qualityThreshold case to the existing
parseSalesDecisionRequest rejection table, using a value below 0, so the
parser’s lower-bound validation is covered alongside the existing out-of-range
upper-bound case.
---
Nitpick comments:
In `@examples/local-runner/workflow-orchestration/run-api.test.ts`:
- Line 7: Move the test file to the api directory alongside the run.post
endpoint, and update its import to reference ./run.post from the new location.
Preserve the existing test behavior while complying with the required test
co-location convention.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 3b1d215b-2e3d-4001-a272-98232db1777a
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (13)
examples/local-runner/README.mdexamples/local-runner/orchestration/README.mdexamples/local-runner/orchestration/run.mjsexamples/local-runner/run-app/README.mdexamples/local-runner/workflow-orchestration/api/run.post.tsexamples/local-runner/workflow-orchestration/run-api.test.tsexamples/local-runner/workflow-orchestration/workflows/sales-report.test.tsexamples/local-runner/workflow-orchestration/workflows/sales-report.tspackages/local-runner/README.mdpackages/local-runner/src/orchestrate.test.tspackages/local-runner/src/orchestrate.tspackages/local-runner/src/serve-static.test.tspackages/local-runner/src/serve-static.ts
🚧 Files skipped from review as they are similar to previous changes (9)
- examples/local-runner/orchestration/README.md
- examples/local-runner/orchestration/run.mjs
- packages/local-runner/src/serve-static.test.ts
- packages/local-runner/src/serve-static.ts
- packages/local-runner/README.md
- packages/local-runner/src/orchestrate.test.ts
- examples/local-runner/workflow-orchestration/workflows/sales-report.ts
- packages/local-runner/src/orchestrate.ts
- examples/local-runner/run-app/README.md
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
examples/local-runner/gallery/pipeline.test.ts (1)
27-27: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winValidate the manifest instead of asserting its type.
JSON.parse()is cast directly toPipelineManifest, bypassing the manifest boundary. Parse it asunknownand validate/narrow the required UI contract before the tests consume it.As per coding guidelines, “Use strict TypeScript type checking, prefer type safety over convenience, and avoid
anyin favor of proper type definitions.”🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@examples/local-runner/gallery/pipeline.test.ts` at line 27, Update the manifest loading in pipeline.test.ts to parse the JSON as unknown rather than casting it directly to PipelineManifest, then validate and narrow it against the required PipelineManifest UI contract before test consumption. Reuse an existing manifest validator if available; otherwise add a type-safe validation path without any.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/local-runner/gallery/index.html`:
- Around line 106-116: Make the pipeline destination portable across static
hosts: in examples/local-runner/gallery/index.html lines 106-116, change the
Pipeline card to target ./pipeline.html; in
examples/local-runner/gallery/README.md lines 122-128, document the same .html
URL requirement and any /pipeline rewrite alternative if supported.
In `@examples/local-runner/gallery/pipeline.html`:
- Around line 484-499: Introduce a safe hash-decoding helper near the pipeline
initialization logic that catches decodeURIComponent failures and returns no
selection for malformed fragments. Use it for both the initial hash used to
compute initial and the hashchange handler before matching manifest.nodes, so
invalid fragments neither misreport artifact-loading failures nor throw uncaught
errors. Add a regression test covering malformed fragments such as “#%”.
In `@examples/local-runner/gallery/pipeline.json`:
- Around line 40-51: Update the detail field for the analyze-europe entry to
report the snapshot’s recorded quality score of approximately 83% instead of
92%. Keep the existing execution metadata and downstream recovery-threshold
behavior unchanged.
---
Nitpick comments:
In `@examples/local-runner/gallery/pipeline.test.ts`:
- Line 27: Update the manifest loading in pipeline.test.ts to parse the JSON as
unknown rather than casting it directly to PipelineManifest, then validate and
narrow it against the required PipelineManifest UI contract before test
consumption. Reuse an existing manifest validator if available; otherwise add a
type-safe validation path without any.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: a809df66-2b2a-481a-b844-eb6ab8254665
📒 Files selected for processing (14)
examples/local-runner/gallery/README.mdexamples/local-runner/gallery/gallery.jsexamples/local-runner/gallery/index.htmlexamples/local-runner/gallery/pipeline-snapshots/analyze-asia-pacific.snapshot.deepnoteexamples/local-runner/gallery/pipeline-snapshots/analyze-europe.snapshot.deepnoteexamples/local-runner/gallery/pipeline-snapshots/analyze-north-america.snapshot.deepnoteexamples/local-runner/gallery/pipeline-snapshots/decision-claude.snapshot.deepnoteexamples/local-runner/gallery/pipeline-snapshots/decision-gpt.snapshot.deepnoteexamples/local-runner/gallery/pipeline-snapshots/final-arbiter.snapshot.deepnoteexamples/local-runner/gallery/pipeline-snapshots/recover-europe.snapshot.deepnoteexamples/local-runner/gallery/pipeline.htmlexamples/local-runner/gallery/pipeline.jsonexamples/local-runner/gallery/pipeline.test.tsexamples/local-runner/gallery/serve.mjs
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
examples/local-runner/run-app/index.html (1)
305-311: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winAlign the Claude model name.
The accessible label says “Claude Sonnet 5,” while the graph visibly says “Claude 5.” Use one name so the graph does not misidentify the reviewed model.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@examples/local-runner/run-app/index.html` around lines 305 - 311, Update the Claude node labels in the graph-node-link for data-step-id “decision-claude” so the accessible data-label and visible node-name use the same model name, specifically “Anthropic Claude Sonnet 5,” while preserving the existing node structure and state text.
🧹 Nitpick comments (3)
packages/local-runner/src/orchestrate.ts (1)
531-536: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueTautology:
value === undefined ? undefined : value.Reduces to
value.♻️ Tidy-up
persistedState.checkpoints[id] = { kind: 'control', fingerprint, - value: value === undefined ? undefined : value, + value, valueIsUndefined: value === undefined || undefined, }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/local-runner/src/orchestrate.ts` around lines 531 - 536, In the checkpoint assignment within the persisted state update, simplify the `value` field expression to directly use `value` instead of the redundant `value === undefined ? undefined : value` conditional; leave `valueIsUndefined` and the surrounding checkpoint structure unchanged.packages/local-runner/src/orchestrate.test.ts (1)
435-476: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConsider covering
persistence.resume: falseand a cached failed step.Both branches are live in
orchestrate(fresh-start path, and the cachedstep_failedemit for anallowFailurestep) but untested here.Also applies to: 492-496
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/local-runner/src/orchestrate.test.ts` around lines 435 - 476, Add tests in the orchestration test suite covering persistence.resume set to false and resuming with a cached step_failed result for an allowFailure step. Exercise the fresh-start behavior when resume is disabled and verify the cached failed step emits the expected result without rerunning it, using the existing orchestrate and runnerMock test patterns.packages/local-runner/src/orchestration-state.ts (1)
73-80: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueTemp file survives a failed write.
If
writeFileorrenamethrows,<file>.<pid>.<uuid>.tmpis left behind next to the state file. Atry/finallyunlink keeps the directory clean.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/local-runner/src/orchestration-state.ts` around lines 73 - 80, Update writePersistedState to wrap the temporary-file write and rename operations in a try/finally block, and unlink the generated temporary path in the finally cleanup. Preserve the existing successful atomic rename behavior while ensuring cleanup runs when either writeFile or rename fails.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/local-runner/gallery/pipeline-data.js`:
- Around line 56-62: Update layoutNodes to assign columns using a topological
traversal rather than the input order: validate that every edge endpoint exists,
process zero-indegree nodes through a ready queue, and assign each node’s column
after its parents are resolved. Reject cyclic graphs when not all nodes are
processed, while preserving the existing root and parent-based column
calculation.
In `@examples/local-runner/gallery/pipeline.test.ts`:
- Around line 32-95: Extend the tests for normalizePipelineManifest with a
malformed manifest case that asserts the documented TypeError is thrown. Keep
the existing successful persisted-result test unchanged and use the invalid
input shape expected by normalizePipelineManifest to exercise its validation
path.
In `@packages/local-runner/src/orchestrate.ts`:
- Around line 577-584: Update the catch block in the orchestration flow so
failures from the failure-path saveState call cannot replace the original
orchestration error. Preserve the persisted failed status and error message when
possible, but contain or otherwise ignore saveState/persistenceWrites failures
before rethrowing the original error.
- Around line 302-303: Update the resumed-state timestamp initialization around
orchestrationStartedMs to validate the Date.parse result and fall back to
Date.now() when it is not finite, while preserving valid persisted timestamps
and the existing orchestrationStartedAt behavior.
In `@packages/local-runner/src/orchestration-state.ts`:
- Around line 131-135: Update the key sorting in the normalization logic used by
the fingerprint generation to use a deterministic code-unit comparison instead
of localeCompare. Preserve the existing Object.fromEntries and normalize
behavior while ensuring identical candidate keys produce the same order across
locales, machines, and Node builds.
- Around line 145-159: Extend isPersistedState to validate every checkpoints
entry before accepting persisted state. Add a per-entry guard requiring the
expected kind and fingerprint fields, and for notebook entries verify the
minimally required result shape, so invalid or truncated payloads are rejected
at the file-read trust boundary before orchestrate uses saved.result.
---
Outside diff comments:
In `@examples/local-runner/run-app/index.html`:
- Around line 305-311: Update the Claude node labels in the graph-node-link for
data-step-id “decision-claude” so the accessible data-label and visible
node-name use the same model name, specifically “Anthropic Claude Sonnet 5,”
while preserving the existing node structure and state text.
---
Nitpick comments:
In `@packages/local-runner/src/orchestrate.test.ts`:
- Around line 435-476: Add tests in the orchestration test suite covering
persistence.resume set to false and resuming with a cached step_failed result
for an allowFailure step. Exercise the fresh-start behavior when resume is
disabled and verify the cached failed step emits the expected result without
rerunning it, using the existing orchestrate and runnerMock test patterns.
In `@packages/local-runner/src/orchestrate.ts`:
- Around line 531-536: In the checkpoint assignment within the persisted state
update, simplify the `value` field expression to directly use `value` instead of
the redundant `value === undefined ? undefined : value` conditional; leave
`valueIsUndefined` and the surrounding checkpoint structure unchanged.
In `@packages/local-runner/src/orchestration-state.ts`:
- Around line 73-80: Update writePersistedState to wrap the temporary-file write
and rename operations in a try/finally block, and unlink the generated temporary
path in the finally cleanup. Preserve the existing successful atomic rename
behavior while ensuring cleanup runs when either writeFile or rename fails.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 5b6bbb0f-31d3-4f4f-a957-fb754dc97d65
📒 Files selected for processing (16)
.gitignoreexamples/local-runner/gallery/README.mdexamples/local-runner/gallery/pipeline-data.d.tsexamples/local-runner/gallery/pipeline-data.jsexamples/local-runner/gallery/pipeline.htmlexamples/local-runner/gallery/pipeline.test.tsexamples/local-runner/gallery/serve.mjsexamples/local-runner/orchestration/README.mdexamples/local-runner/orchestration/run.mjsexamples/local-runner/run-app/index.htmlexamples/local-runner/run-app/serve.mjspackages/local-runner/README.mdpackages/local-runner/src/index.tspackages/local-runner/src/orchestrate.test.tspackages/local-runner/src/orchestrate.tspackages/local-runner/src/orchestration-state.ts
🚧 Files skipped from review as they are similar to previous changes (6)
- examples/local-runner/gallery/serve.mjs
- examples/local-runner/orchestration/run.mjs
- packages/local-runner/src/index.ts
- examples/local-runner/gallery/README.md
- examples/local-runner/gallery/pipeline.html
- examples/local-runner/run-app/serve.mjs
| it('renders a persisted orchestrate result without a hand-authored graph manifest', () => { | ||
| const normalized = normalizePipelineManifest({ | ||
| status: 'completed', | ||
| result: { | ||
| value: { title: 'Generated pipeline', finalDecision: 'proceed' }, | ||
| steps: [ | ||
| { id: 'north', target: 'cloud', durationMs: 10, snapshotYaml: 'north snapshot', runId: 'run-north' }, | ||
| { id: 'final', target: 'cloud', durationMs: 20, snapshotYaml: 'final snapshot', runId: 'run-final' }, | ||
| ], | ||
| graph: { | ||
| concludingNodeId: 'final', | ||
| nodes: [ | ||
| { id: 'inputs', label: 'Inputs', kind: 'control', status: 'success', startedAt: '2026-01-01' }, | ||
| { | ||
| id: 'north', | ||
| label: 'North', | ||
| kind: 'notebook', | ||
| target: 'cloud', | ||
| status: 'success', | ||
| startedAt: '2026-01-01', | ||
| }, | ||
| { | ||
| id: 'gate', | ||
| label: 'Quality gate', | ||
| kind: 'gate', | ||
| status: 'success', | ||
| startedAt: '2026-01-01', | ||
| }, | ||
| { | ||
| id: 'final', | ||
| label: 'Final', | ||
| kind: 'notebook', | ||
| target: 'cloud', | ||
| status: 'success', | ||
| concluding: true, | ||
| startedAt: '2026-01-01', | ||
| }, | ||
| ], | ||
| edges: [ | ||
| { from: 'inputs', to: 'north' }, | ||
| { from: 'north', to: 'gate' }, | ||
| { from: 'gate', to: 'final', label: 'passed' }, | ||
| ], | ||
| }, | ||
| startedAt: '2026-01-01T00:00:00.000Z', | ||
| finishedAt: '2026-01-01T00:00:01.000Z', | ||
| durationMs: 1_000, | ||
| }, | ||
| }) | ||
|
|
||
| expect(normalized).toMatchObject({ | ||
| schemaVersion: 2, | ||
| title: 'Generated pipeline', | ||
| concludingStepId: 'final', | ||
| summary: { notebookRuns: 2, finalDecision: 'proceed' }, | ||
| }) | ||
| expect(normalized.nodes).toEqual([ | ||
| expect.objectContaining({ id: 'inputs', kind: 'local', column: 0, lane: 1 }), | ||
| expect.objectContaining({ id: 'north', kind: 'notebook', column: 1, snapshotYaml: 'north snapshot' }), | ||
| expect.objectContaining({ id: 'gate', kind: 'local', column: 2 }), | ||
| expect.objectContaining({ id: 'final', kind: 'notebook', column: 3, snapshotYaml: 'final snapshot' }), | ||
| ]) | ||
| expect(normalized.stageLabels?.at(-1)).toBe('CONCLUSION') | ||
| }) |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Cover the invalid-record path.
Add a regression asserting malformed input throws the documented TypeError; this new feature otherwise only tests its happy path. As per coding guidelines, “Write comprehensive tests covering new features, edge cases, error handling.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@examples/local-runner/gallery/pipeline.test.ts` around lines 32 - 95, Extend
the tests for normalizePipelineManifest with a malformed manifest case that
asserts the documented TypeError is thrown. Keep the existing successful
persisted-result test unchanged and use the invalid input shape expected by
normalizePipelineManifest to exercise its validation path.
Source: Coding guidelines
| const orchestrationStartedMs = resumedState ? Date.parse(resumedState.startedAt) : Date.now() | ||
| const orchestrationStartedAt = resumedState?.startedAt ?? new Date(orchestrationStartedMs).toISOString() |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Date.parse can yield NaN for a resumed state.
isPersistedState only checks typeof startedAt === 'string', so a hand-edited or truncated file gives NaN here, and every durationMs derived from it becomes NaN. Worse, assertJsonSerializable(result, …) then rejects the completed run on the non-finite check. Fall back to Date.now() when the parse fails.
🛡️ Proposed fix
- const orchestrationStartedMs = resumedState ? Date.parse(resumedState.startedAt) : Date.now()
- const orchestrationStartedAt = resumedState?.startedAt ?? new Date(orchestrationStartedMs).toISOString()
+ const resumedStartedMs = resumedState ? Date.parse(resumedState.startedAt) : Number.NaN
+ const orchestrationStartedMs = Number.isFinite(resumedStartedMs) ? resumedStartedMs : Date.now()
+ const orchestrationStartedAt = Number.isFinite(resumedStartedMs)
+ ? (resumedState?.startedAt as string)
+ : new Date(orchestrationStartedMs).toISOString()📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const orchestrationStartedMs = resumedState ? Date.parse(resumedState.startedAt) : Date.now() | |
| const orchestrationStartedAt = resumedState?.startedAt ?? new Date(orchestrationStartedMs).toISOString() | |
| const resumedStartedMs = resumedState ? Date.parse(resumedState.startedAt) : Number.NaN | |
| const orchestrationStartedMs = Number.isFinite(resumedStartedMs) ? resumedStartedMs : Date.now() | |
| const orchestrationStartedAt = Number.isFinite(resumedStartedMs) | |
| ? (resumedState?.startedAt as string) | |
| : new Date(orchestrationStartedMs).toISOString() |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/local-runner/src/orchestrate.ts` around lines 302 - 303, Update the
resumed-state timestamp initialization around orchestrationStartedMs to validate
the Date.parse result and fall back to Date.now() when it is not finite, while
preserving valid persisted timestamps and the existing orchestrationStartedAt
behavior.
| const normalized = Object.fromEntries( | ||
| Object.entries(candidate) | ||
| .sort(([left], [right]) => left.localeCompare(right)) | ||
| .map(([key, entry]) => [key, normalize(entry)]) | ||
| ) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Fingerprint key order depends on locale.
localeCompare ordering varies with ICU data/default locale, so the same node can fingerprint differently across machines or Node builds — resume then fails with "definition or inputs changed". Use a code-unit comparison for a stable hash.
🔒️ Proposed fix
const normalized = Object.fromEntries(
Object.entries(candidate)
- .sort(([left], [right]) => left.localeCompare(right))
+ .sort(([left], [right]) => (left < right ? -1 : left > right ? 1 : 0))
.map(([key, entry]) => [key, normalize(entry)])
)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const normalized = Object.fromEntries( | |
| Object.entries(candidate) | |
| .sort(([left], [right]) => left.localeCompare(right)) | |
| .map(([key, entry]) => [key, normalize(entry)]) | |
| ) | |
| const normalized = Object.fromEntries( | |
| Object.entries(candidate) | |
| .sort(([left], [right]) => (left < right ? -1 : left > right ? 1 : 0)) | |
| .map(([key, entry]) => [key, normalize(entry)]) | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/local-runner/src/orchestration-state.ts` around lines 131 - 135,
Update the key sorting in the normalization logic used by the fingerprint
generation to use a deterministic code-unit comparison instead of localeCompare.
Preserve the existing Object.fromEntries and normalize behavior while ensuring
identical candidate keys produce the same order across locales, machines, and
Node builds.
| function isPersistedState(value: unknown): value is PersistedOrchestrationState { | ||
| if (!isRecord(value)) { | ||
| return false | ||
| } | ||
| return ( | ||
| value.schemaVersion === PERSISTENCE_SCHEMA_VERSION && | ||
| (value.status === 'running' || value.status === 'completed' || value.status === 'failed') && | ||
| typeof value.startedAt === 'string' && | ||
| typeof value.updatedAt === 'string' && | ||
| isRecord(value.checkpoints) && | ||
| isRecord(value.graph) && | ||
| Array.isArray(value.graph.nodes) && | ||
| Array.isArray(value.graph.edges) | ||
| ) | ||
| } |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
Validation stops at the top level; checkpoint payloads are trusted.
checkpoints is only checked to be a record. A truncated or hand-edited file therefore feeds arbitrary objects into saved.result in orchestrate, which are pushed into results, re-emitted as step_completed/step_failed, and handed to workflow code as an OrchestrationStepResult. A per-entry guard (kind, fingerprint, and for notebooks a minimally shaped result) keeps the trust boundary at the file read. Similar to prior feedback in this package about not relying on a single schema check before values reach downstream APIs.
🛡️ Sketch
+function isCheckpoint(value: unknown): value is OrchestrationCheckpoint {
+ if (!isRecord(value) || typeof value.fingerprint !== 'string') {
+ return false
+ }
+ if (value.kind === 'control') {
+ return true
+ }
+ return value.kind === 'notebook' && isRecord(value.result) && typeof value.result.id === 'string'
+} isRecord(value.checkpoints) &&
+ Object.values(value.checkpoints).every(isCheckpoint) &&
isRecord(value.graph) &&Based on learnings that in packages/local-runner unvalidated structures should be explicitly handled before reaching downstream APIs rather than relying solely on the deserialization schema.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| function isPersistedState(value: unknown): value is PersistedOrchestrationState { | |
| if (!isRecord(value)) { | |
| return false | |
| } | |
| return ( | |
| value.schemaVersion === PERSISTENCE_SCHEMA_VERSION && | |
| (value.status === 'running' || value.status === 'completed' || value.status === 'failed') && | |
| typeof value.startedAt === 'string' && | |
| typeof value.updatedAt === 'string' && | |
| isRecord(value.checkpoints) && | |
| isRecord(value.graph) && | |
| Array.isArray(value.graph.nodes) && | |
| Array.isArray(value.graph.edges) | |
| ) | |
| } | |
| function isCheckpoint(value: unknown): value is OrchestrationCheckpoint { | |
| if (!isRecord(value) || typeof value.fingerprint !== 'string') { | |
| return false | |
| } | |
| if (value.kind === 'control') { | |
| return true | |
| } | |
| return value.kind === 'notebook' && isRecord(value.result) && typeof value.result.id === 'string' | |
| } | |
| function isPersistedState(value: unknown): value is PersistedOrchestrationState { | |
| if (!isRecord(value)) { | |
| return false | |
| } | |
| return ( | |
| value.schemaVersion === PERSISTENCE_SCHEMA_VERSION && | |
| (value.status === 'running' || value.status === 'completed' || value.status === 'failed') && | |
| typeof value.startedAt === 'string' && | |
| typeof value.updatedAt === 'string' && | |
| isRecord(value.checkpoints) && | |
| Object.values(value.checkpoints).every(isCheckpoint) && | |
| isRecord(value.graph) && | |
| Array.isArray(value.graph.nodes) && | |
| Array.isArray(value.graph.edges) | |
| ) | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/local-runner/src/orchestration-state.ts` around lines 145 - 159,
Extend isPersistedState to validate every checkpoints entry before accepting
persisted state. Add a per-entry guard requiring the expected kind and
fingerprint fields, and for notebook entries verify the minimally required
result shape, so invalid or truncated payloads are rejected at the file-read
trust boundary before orchestrate uses saved.result.
Source: Learnings
The orchestration example was the only consumer of runWithPolicy, defineRunPolicy, definePipeline and invoke. Rewrite it on run/control/ outputs alone, ahead of removing those APIs from the package. Retry becomes a loop and a sub-pipeline becomes a function taking the orchestration context. Both must handle two distinct failures: a notebook that ran and failed returns success: false, while infrastructure that never got the notebook running throws regardless of allowFailure. The local fan-out relies on this — two kernels race for a toolkit port, and the losing attempt throws rather than returning. Add ORCHESTRATION_TARGET so a local run stays reachable with a token in .env, and drop the persistence wiring. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
orchestrate() shipped an opt-in JSON checkpoint file that looked like durability without providing it. It was at-least-once by its own documentation, took no lock so two processes on one file clobbered each other, and rewrote the whole file — including every notebook's snapshot, stored both raw and parsed — on every node transition. Workflow SDK's event log already does this properly, and runNotebookStep is the seam to it. Delete the option, orchestration-state.ts, the fingerprinting, and the resume bookkeeping. The generated graph is untouched: it is derived from execution, not from the checkpoint file. The gallery consumes a flattened graph export and its tests still pass unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both re-encoded as configuration what durable execution gives you as ordinary control flow. runWithPolicy was a retry/backoff/fallback config object that synthesised graph nodes and leaked policyNodeId, forcing every caller into `dependsOn: [x.policyNodeId ?? x.id]`. definePipeline/invoke existed to prefix child node IDs, which a function taking the context does just as well. Under Workflow SDK a for-loop with try/catch and a durable sleep is a retry policy, and a nested "use workflow" function is a sub-pipeline — both already resumable and visible in the run history. Removes defineRunPolicy, definePipeline, runWithPolicy, invoke, their types, the pipeline_* events, the 'policy'/'pipeline' node kinds, and the parentId plumbing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replace the resume, retry-policy, and sub-pipeline sections with one that shows those as ordinary TypeScript under a "use workflow" function, and state plainly that orchestrate() is not durable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
examples/local-runner/run-app/index.html (1)
153-181: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winMake the pipeline graph keyboard accessible.
The SVG uses
role="img"while its notebook-run nodes are<a>elements without statichrefattributes. This exposes the graph as an image and does not provide reliable keyboard activation for the nodes. Use real links with accessible names, remove the image role from the interactive graph, and provide visible focus styling.Also applies to: 291-304, 305-322, 335-343, 354-371, 374-382
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@examples/local-runner/run-app/index.html` around lines 153 - 181, Make the pipeline graph interactive and keyboard accessible by removing the SVG image role, giving each notebook-run node link a real static href and accessible name, and preserving navigation when activated with the keyboard. Update the graph node link markup and related rendering paths around the notebook-run nodes, then add visible :focus-visible styling for the link/group without relying on hover-only behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/local-runner/orchestration/run.mjs`:
- Line 17: Validate the ORCHESTRATION_TARGET value before selecting the
execution mode: accept only 'local' and 'cloud', while preserving the existing
default based on DEEPNOTE_TOKEN when the variable is unset or empty. Reject any
other non-empty value immediately instead of treating it as cloud.
---
Outside diff comments:
In `@examples/local-runner/run-app/index.html`:
- Around line 153-181: Make the pipeline graph interactive and keyboard
accessible by removing the SVG image role, giving each notebook-run node link a
real static href and accessible name, and preserving navigation when activated
with the keyboard. Update the graph node link markup and related rendering paths
around the notebook-run nodes, then add visible :focus-visible styling for the
link/group without relying on hover-only behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 20b158d7-a604-4fe1-ad85-3865d51ea1cf
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (15)
examples/local-runner/README.mdexamples/local-runner/gallery/pipeline-data.jsexamples/local-runner/gallery/pipeline.test.tsexamples/local-runner/orchestration/README.mdexamples/local-runner/orchestration/run.mjsexamples/local-runner/run-app/README.mdexamples/local-runner/run-app/index.htmlexamples/local-runner/run-app/serve.mjspackage.jsonpackages/local-runner/README.mdpackages/local-runner/src/index.tspackages/local-runner/src/orchestrate.test.tspackages/local-runner/src/orchestrate.tspackages/local-runner/src/serve-static.test.tspackages/local-runner/src/serve-static.ts
💤 Files with no reviewable changes (1)
- packages/local-runner/src/orchestrate.test.ts
🚧 Files skipped from review as they are similar to previous changes (6)
- package.json
- examples/local-runner/gallery/pipeline-data.js
- examples/local-runner/README.md
- packages/local-runner/src/serve-static.ts
- examples/local-runner/run-app/README.md
- packages/local-runner/src/serve-static.test.ts
|
|
||
| // Cloud when a token is present, unless ORCHESTRATION_TARGET says otherwise. The override keeps a | ||
| // local run reachable on a machine that has a token in `.env`. | ||
| const target = process.env.ORCHESTRATION_TARGET || (process.env.DEEPNOTE_TOKEN ? 'cloud' : 'local') |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Validate ORCHESTRATION_TARGET.
Any non-empty value except 'local' selects cloud execution. For example, ORCHESTRATION_TARGET=locla can spend cloud credentials instead of failing fast. Accept only 'local' and 'cloud', or throw for other values.
Proposed fix
-const target = process.env.ORCHESTRATION_TARGET || (process.env.DEEPNOTE_TOKEN ? 'cloud' : 'local')
+const requestedTarget = process.env.ORCHESTRATION_TARGET
+if (requestedTarget && requestedTarget !== 'local' && requestedTarget !== 'cloud') {
+ throw new Error('ORCHESTRATION_TARGET must be "local" or "cloud"')
+}
+const target = requestedTarget ?? (process.env.DEEPNOTE_TOKEN ? 'cloud' : 'local')🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@examples/local-runner/orchestration/run.mjs` at line 17, Validate the
ORCHESTRATION_TARGET value before selecting the execution mode: accept only
'local' and 'cloud', while preserving the existing default based on
DEEPNOTE_TOKEN when the variable is unset or empty. Reject any other non-empty
value immediately instead of treating it as cloud.
Summary
End-to-end demo
Run pnpm example:local-runner and choose Run orchestrated pipeline. With DEEPNOTE_TOKEN set, the app executes the notebook pipeline in Deepnote Cloud while orchestration remains in the local Node process.
The decision demo shows:
Every notebook node and result card opens its exact cloud run.
Validation
Summary by CodeRabbit
New Features
Documentation