Workflow Execution Engine

Relevant source files

• apps/docs/content/docs/en/blocks/response.mdx
• apps/docs/content/docs/en/execution/basics.mdx
• apps/sim/app/api/billing/update-cost/route.ts
• apps/sim/app/api/mcp/tools/execute/route.ts
• apps/sim/app/api/workflows/[id]/execute/route.ts
• apps/sim/app/api/workflows/[id]/log/route.ts
• apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tag-dropdown/types.ts
• apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts
• apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-execution-utils.test.ts
• apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-execution-utils.ts
• apps/sim/app/workspace/[workspaceId]/w/components/preview/components/preview-workflow/components/subflow/subflow.tsx
• apps/sim/app/workspace/[workspaceId]/w/components/preview/components/preview-workflow/preview-workflow.tsx
• apps/sim/executor/dag/construction/edges.test.ts
• apps/sim/executor/dag/construction/edges.ts
• apps/sim/executor/dag/construction/nodes.ts
• apps/sim/executor/dag/types.ts
• apps/sim/executor/execution/block-executor.ts
• apps/sim/executor/execution/edge-manager.test.ts
• apps/sim/executor/execution/edge-manager.ts
• apps/sim/executor/execution/engine.test.ts
• apps/sim/executor/execution/engine.ts
• apps/sim/executor/execution/executor.ts
• apps/sim/executor/execution/snapshot-serializer.ts
• apps/sim/executor/execution/snapshot.test.ts
• apps/sim/executor/execution/snapshot.ts
• apps/sim/executor/execution/state.ts
• apps/sim/executor/execution/types.ts
• apps/sim/executor/handlers/workflow/workflow-handler.test.ts
• apps/sim/executor/handlers/workflow/workflow-handler.ts
• apps/sim/executor/index.ts
• apps/sim/executor/orchestrators/loop.ts
• apps/sim/executor/orchestrators/node.ts
• apps/sim/executor/orchestrators/parallel.test.ts
• apps/sim/executor/orchestrators/parallel.ts
• apps/sim/executor/types.ts
• apps/sim/executor/utils/iteration-context.test.ts
• apps/sim/executor/utils/parallel-expansion.ts
• apps/sim/executor/utils/subflow-utils.ts
• apps/sim/executor/variables/resolvers/loop.test.ts
• apps/sim/executor/variables/resolvers/loop.ts
• apps/sim/executor/variables/resolvers/parallel.test.ts
• apps/sim/executor/variables/resolvers/parallel.ts
• apps/sim/executor/variables/resolvers/workflow.ts
• apps/sim/hooks/use-execution-stream.ts
• apps/sim/lib/billing/calculations/usage-monitor.ts
• apps/sim/lib/billing/client/utils.ts
• apps/sim/lib/billing/core/billing.ts
• apps/sim/lib/billing/core/organization.ts
• apps/sim/lib/billing/core/usage.ts
• apps/sim/lib/billing/webhooks/invoices.ts
• apps/sim/lib/core/idempotency/service.ts
• apps/sim/lib/core/utils/arrays.test.ts
• apps/sim/lib/core/utils/arrays.ts
• apps/sim/lib/logs/execution/logger.test.ts
• apps/sim/lib/logs/execution/logger.ts
• apps/sim/lib/logs/execution/logging-factory.test.ts
• apps/sim/lib/logs/execution/logging-factory.ts
• apps/sim/lib/logs/execution/logging-session.test.ts
• apps/sim/lib/logs/execution/logging-session.ts
• apps/sim/lib/logs/execution/trace-spans/span-factory.ts
• apps/sim/lib/logs/execution/trace-spans/trace-spans.test.ts
• apps/sim/lib/logs/execution/trace-spans/trace-spans.ts
• apps/sim/lib/logs/types.ts
• apps/sim/lib/workflows/executor/execution-core.test.ts
• apps/sim/lib/workflows/executor/execution-core.ts
• apps/sim/lib/workflows/executor/execution-events.ts
• apps/sim/providers/index.test.ts
• apps/sim/stores/terminal/console/store.test.ts
• apps/sim/stores/terminal/console/store.ts
• apps/sim/stores/terminal/console/types.ts

The Workflow Execution Engine is the core system responsible for orchestrating and executing workflow definitions through a DAG-based (Directed Acyclic Graph) execution model. It manages the entire lifecycle from trigger reception through block-by-block execution to final output delivery, supporting multiple execution modes including streaming responses, background jobs, and human-in-the-loop pause/resume patterns.

Architecture Overview

The execution engine is structured around three primary layers: the entry layer (API routes and background jobs), the orchestration layer (preprocessing, snapshot management, and core execution), and the execution layer (DAG executor and block handlers).

Execution Flow: Natural Language to Code Entities

The following diagram maps high-level execution concepts to the specific classes and functions implemented in the codebase.

Sources: apps/sim/app/api/workflows/[id]/execute/route.ts:46-47, apps/sim/lib/workflows/executor/execution-core.ts22-31 apps/sim/executor/execution/block-executor.ts59-81 apps/sim/executor/index.ts1-7 apps/sim/app/api/workflows/[id]/execute/route.ts:56-56

Execution Pipeline

The execution pipeline processes workflow requests through distinct phases: trigger reception, preprocessing, serialization, DAG execution, and output delivery.

Trigger Sources

Workflow executions are initiated via several entry points defined in the API and background workers:

Trigger Type	Entry Point	Authentication	Execution Mode
Manual / API	POST /api/workflows/[id]/execute	checkHybridAuth	Inline/Async
Webhook	POST /api/webhooks/trigger/[path]	Provider-specific	Background (Trigger.dev)
Schedule	executeWorkflowJob	Cron Trigger	Background (Trigger.dev)
Child Workflow	WorkflowBlockHandler	Internal Context	Inline

Sources: apps/sim/app/api/workflows/[id]/execute/route.ts:10-10, apps/sim/app/api/workflows/[id]/execute/route.ts:157-210, apps/sim/executor/handlers/workflow/workflow-handler.ts46-68 apps/sim/app/api/workflows/[id]/execute/route.ts:56-56

Preprocessing Phase

All executions pass through preprocessExecution apps/sim/app/api/workflows/[id]/execute/route.ts:40-40 which validates requests and enforces constraints such as authentication, rate limits, and deployment status. In the API layer, the executeWorkflowBodySchema apps/sim/app/api/workflows/[id]/execute/route.ts:9-9 validates the incoming payload before passing it to executeWorkflowCore.

Execution Context

The execution engine maintains state through ExecutionSnapshot, LoggingSession, and the ExecutionContext.

ExecutionSnapshot

ExecutionSnapshot apps/sim/app/api/workflows/[id]/execute/route.ts:62-62 encapsulates all data required to execute or resume a workflow. It includes metadata, workflow state, and initial variables.

LoggingSession

LoggingSession apps/sim/lib/logs/execution/logging-session.ts132 manages execution log lifecycle and database persistence. It provides methods like safeComplete apps/sim/lib/logs/execution/logging-session.ts204-211 to ensure logs and costs are written even if execution fails. It also tracks real-time progress by persisting markers for the last started apps/sim/lib/logs/execution/logging-session.ts188-201 and completed apps/sim/lib/logs/execution/logging-session.ts203-216 blocks.

Core Execution Components

DAGExecutor

The DAGExecutor apps/sim/executor/execution/executor.ts6 (aliased as Executor in apps/sim/executor/index.ts6) is the central orchestrator. It builds a dependency graph and executes nodes in topological order.

BlockExecutor

The BlockExecutor apps/sim/executor/execution/block-executor.ts59 handles the lifecycle of an individual block's execution:

1. Input Resolution: Uses VariableResolver to resolve inputs apps/sim/executor/execution/block-executor.ts125-147
2. Handler Selection: Finds the appropriate BlockHandler via findHandler apps/sim/executor/execution/block-executor.ts82
3. Execution: Calls handler.executeWithNode or handler.execute apps/sim/executor/execution/block-executor.ts169-171
4. Output Normalization: Processes raw output into NormalizedBlockOutput apps/sim/executor/execution/block-executor.ts191-196
5. State Persistence: Updates BlockStateWriter with results apps/sim/executor/execution/block-executor.ts221

Loop & Parallel Orchestration

The engine supports complex control flow via specialized orchestrators:

• LoopOrchestrator: Manages iteration indexing, collection resolution, and exit conditions.
• ParallelOrchestrator: Manages concurrent execution branches. It expands the DAG dynamically to create virtual nodes for parallel branches.

Sources: apps/sim/executor/orchestrators/loop.ts1-10 apps/sim/executor/orchestrators/parallel.ts1-10

Streaming and Callbacks

The execution engine supports real-time output delivery through Server-Sent Events (SSE) and callback functions.

Callback System

ExecutionContext apps/sim/executor/types.ts330-348 includes hooks for execution events:

• onBlockStart: Fired when a block begins apps/sim/executor/execution/block-executor.ts104
• onBlockComplete: Fired when a block finishes apps/sim/executor/execution/block-executor.ts227-231
• onStream: Handles streaming content from blocks like Agents apps/sim/executor/execution/block-executor.ts181-190

Client-Side Execution Hook

The useWorkflowExecution hook apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts:141-141 manages the frontend execution state. It uses useExecutionStream apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts:46-46 to process SSE events and updates the useTerminalConsoleStore apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts:154-163 to reflect block status and logs in the UI.

Nested Workflow Execution

The WorkflowBlockHandler apps/sim/executor/handlers/workflow/workflow-handler.ts38 allows workflows to call other workflows.

1. Call Chain Validation: Prevents infinite recursion by checking depth via validateCallChain apps/sim/executor/handlers/workflow/workflow-handler.ts83-90
2. Input Mapping: Cleans up and maps parent variables to child inputs using lazyCleanupInputMapping apps/sim/executor/handlers/workflow/workflow-handler.ts123-129
3. Sub-Executor: Spawns a new Executor instance for the child workflow apps/sim/executor/handlers/workflow/workflow-handler.ts168-195

Pause and Resume System

The engine manages "Human-in-the-loop" blocks via the pause/resume infrastructure.

• Context IDs: Every pause point is assigned a unique contextId generated via generatePauseContextId apps/sim/executor/execution/block-executor.ts29
• Snapshots: The entire SerializableExecutionState is saved when a workflow pauses apps/sim/executor/execution/types.ts67-68
• Resumption: The runFromBlock logic in executeWorkflowCore loads the snapshot and restarts execution from the specified block apps/sim/lib/workflows/executor/execution-core.ts50-53

Execution Logs and Terminal

Logs are processed into TraceSpan objects apps/sim/lib/logs/execution/trace-spans/trace-spans.ts49-52 for hierarchical display in the Terminal.

Terminal Data Flow: Code to UI

This diagram shows how execution events travel from the backend handlers to the frontend Terminal UI.

Sources: apps/sim/stores/terminal/console/store.ts13-18 apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts:141-163, apps/sim/executor/types.ts225-247 apps/sim/lib/workflows/executor/execution-core.ts172-211

• Trace Spans: Group block executions, including nested child workflows apps/sim/lib/logs/execution/trace-spans/trace-spans.ts49-66
• Terminal Console: The useTerminalConsoleStore apps/sim/stores/terminal/console/store.ts16-18 maintains a list of ConsoleEntry items, which are updated in real-time as SSE events arrive via addConsole apps/sim/stores/terminal/console/store.ts207

Sources: apps/sim/executor/types.ts225-247 apps/sim/lib/logs/execution/trace-spans/trace-spans.ts49-66 apps/sim/stores/terminal/console/store.ts12-18