Skip to content

runtime

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History
 
 

runtime

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
src
src
 
 
tests
tests
 
 
worker
worker
 
 
README.md
README.md
 
 

README.md

Edge Python Runtime

JS half of Edge Python: hosts compiler_lib.wasm in a Web Worker, resolves and registers .py / .wasm modules, dispatches native calls. Drive it programmatically with createWorker, or declaratively with the <edge-python> HTML element.

Install

No install, the official CDN serves both the runtime and matching compiler_lib.wasm:

import { createWorker } from "https://runtime.edgepython.com/js/src/index.js";
// Local checkout: import { createWorker } from "../../runtime/src/index.js";

Usage

const worker = await createWorker({
    wasmUrl: "https://runtime.edgepython.com/js/compiler_lib.wasm",
    integrity: true, // default: IDB + lockfile CAS
    imports: { dom: "./dom.wasm" }, // bare-name shortcut, optional
    loaders: [], // opt-in module loaders, optional
});

worker.onOutput((line) => console.log(line));

const { out, ms } = await worker.run(`
    from dom import query, set_text
    set_text(query("#app"), "hello")
`);

worker.dispose();

HTML element (<edge-python>)

Declarative alternative to createWorker: include the script, drop a tag, and a .py file runs. The element wraps createWorker on the page's main thread.

<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <script type="module" src="https://runtime.edgepython.com/js/src/element.js"></script>
</head>
<body>
    <edge-python entry="./app/main.py" packages="./app/packages.json"></edge-python>
</body>
</html>

Importing element.js auto-registers the tag. On connect, the element reads its attributes and packages.json, spawns the worker, runs entry if present, then fires a ready event. compiler_lib.wasm loads from the CDN automatically. Modules load lazily: only what a run actually imports is fetched, host libraries included.

Attribute Description
entry Optional URL of a .py file to run on connect. Omit it to drive the worker via run(). Resolved against the document.
packages Optional packages.json URL. Its host and imports fields declare the modules to load (see below).

Programmatic use

The element keeps its worker on el.worker, so you can drive the same VM from JS after ready fires; run(src, opts?) and onOutput(cb) proxy the worker.

const el = document.querySelector("edge-python");
await new Promise((r) => el.addEventListener("ready", r, { once: true }));
el.onOutput((line) => console.log(line));
await el.run("print(1 + 1)"); // 2

Registration

Where customElements is absent (Cloudflare Workers, Deno, SSR), append ?setElement=false to the script URL to skip the auto-call, then register manually with the exported defineElement(tag = "edge-python"), where custom tags must contain a hyphen:

import { defineElement } from "https://runtime.edgepython.com/js/src/element.js?setElement=false";
defineElement("edge-py");

Importing host libraries

Host libraries (DOM, etc.) are plain-JS modules whose handlers run on the page's main thread, because they touch document / window, which the worker can't reach. Declare them in the host field of packages.json:

{
  "host": {
    "dom": "/edge-python-host/dom/src/index.js"
  }
}

Each host entry maps a name to an ESM URL (resolved against the packages.json location). The element passes these to createWorker as hostModules; the module is import()ed lazily the first time a run imports that name, never at connect, so an unused host library is never fetched. The ESM exports its handler factory under the host name (or as default), so export const dom answers from dom import ...:

# app/main.py
from dom import query, set_text
set_text(query("#app"), "hello")

The element reads the same packages.json for the standard imports field too: those bare-name .py / .wasm modules are passed to createWorker's imports. So one manifest drives both directions, host to the main thread and imports to the worker. Together they are the declarative form of the mainThreadModules and imports options.

API

createWorker(opts) -> Promise<Worker>

Spawns a Web Worker, loads compiler_lib.wasm inside it, returns a proxy.

Option Type Default Description
wasmUrl string , URL of compiler_lib.wasm.
integrity boolean true When true, use IDB + lockfile to cache and verify fetched module bytes. Falls back to in-memory cache (with console.warn) if IDB is unavailable.
imports Record<string, string> null Bare-name shortcut: maps Python bare names (from <name> import ...) to URLs of .py / .wasm modules. Replaces the need for a physical packages.json for simple projects.
loaders string[] [] URLs of module loader plugins. Each loader is a .js file with a default export { match, load }. See Writing a loader.
mainThreadModules Record<string, factory | object> {} Main-thread modules supplied as in-memory factories/objects, registered eagerly. Use hostModules instead when you have URLs and want lazy loading. See Main-thread modules.
hostModules Record<string, string> {} Main-thread host libraries by URL (name -> ESM url), import()ed lazily the first time a run imports the name. The <edge-python> element fills this from the host field.
defaults boolean true Seed the resolution table with the official packages so they resolve by bare name without a packages.json: std json / re (worker .wasm) and host dom / network / storage / time (main-thread ESM). Lazy, an unused default is never fetched. Set false to opt out. URLs live in src/defaults.js.
version string null Optional lockfile version key. When present, mismatches with the stored version invalidate the cache before run. Useful to pin cache to a deploy/commit.

Worker

The returned object exposes:

Member Type Description
integrityActive boolean true iff IDB cache opened successfully. Inspect after createWorker to detect silent fallback.
loadMs number Wall time to load + compile compiler_lib.wasm.
run(src, opts?) (string, {entryDir?, baseUrl?}) => Promise<{out, ms}> Execute a Python source string. The runtime does not auto-invoke main, scripts that define async def main() must drive it themselves with a trailing run(main()). Top-level scripts (no main) execute under the implicit module-body coroutine, so receive(), sleep(), etc. still work without wrapping. entryDir is a prefix joined to relative import specs; baseUrl overrides the base for URL resolution (defaults to the worker's location.href). Resolves with stdout (concatenated print() lines if no onOutput) and wall time.
onOutput(handler) (line: string) => void Streaming output callback fired once per print() line.
reset() () => Promise<void> Clear registered modules without rebooting the worker.
clearCache() () => Promise<void> Wipe IDB CAS + lockfile (or memory cache). Next run re-fetches everything.
pushEvent(message) (string) => void Wake a paused receive() in the running script with message. Fire-and-forget. Browser bridges fire CustomEvent("edge-python-event") on window, which createWorker routes through pushEvent automatically.
dispose() () => void Terminate the worker. Subsequent calls fail.

Writing a loader

A loader is a .js file with a default export:

export default {
    /** Inspect the compiled module; return true if this loader handles it. */
    match(module) {
        const names = WebAssembly.Module.exports(module).map(e => e.name);
        return names.includes('my_marker_export');
    },

    /** Load the module and return its callable surface. */
    async load(module, ctx) {
        // ctx.compilerExports, compiler_lib.wasm instance exports (wasm_alloc, host_edge_*, etc.)
        // ctx.rt, handle codec helpers (decodeStr, encodeInt, ...)
        // ctx.fetchedSources, Map of already-fetched spec -> bytes
        // ctx.loaders, full loader list (in order)

        return {
            kind: 'wasmpdk' | 'capability',
            names: ['fn1', 'fn2', ...],
            fns: [fn1Impl, fn2Impl, ...],
        };
    },
};

Two valid kind values:

  • wasmpdk, each fn is a wasm export with signature (g_argv, argc, g_out) -> i32 reading from its own linear memory. Each fn must be annotated with __edge_alloc and __edge_memory (the built-in loader does this automatically). The dispatcher stages argv in guest memory and copies the result handle back.

  • capability, each fn is a plain JS function (handles: number[]) => number taking u32 handles in compiler_lib's memory and returning a u32 result handle. The dispatcher calls it directly without staging.

The built-in Path A wasm-pdk loader is always tried last as fallback; custom loaders run first in order.

Main-thread modules

Engine runs in a Web Worker, so handlers can't reach document / window. mainThreadModules: a pure-JS module declares its handlers, the runtime synthesises the native registration so Python can from <name> import ..., each call defers to main transparently. Python sees a regular synchronous call.

Factory (ctx) => handlers or {name: handler}. Factory form receives { pushEvent } so async callbacks (events, observers, file reads) can wake a paused receive().

const dom = ({ pushEvent }) => {
    const nodes = [];
    const alloc = (n) => { nodes.push(n); return nodes.length - 1; };
    const node = (h) => nodes[h];

    return {
        query: (sel) => alloc(document.querySelector(sel)),
        set_text: (h, txt) => { node(h).textContent = txt; },
        bind_event: (h, type, msg) => {
            node(h).addEventListener(type, (e) => {
                pushEvent(JSON.stringify({ msg, type: e.type, target_id: e.target.id }));
            });
        },
    };
};

const worker = await createWorker({
    wasmUrl: "...",
    mainThreadModules: { dom },
});
from dom import query, set_text, bind_event

bind_event(query("#btn"), "click", "click")
async def main():
    while True:
        receive()
        set_text(query("#btn"), "clicked")

Supported tags: None, bool, int (i64, range-limited by JS Number), float, string bytes. Opaque references (DOM nodes, files, observers) -> integer IDs in a main-thread registry (the alloc / node pattern).

Per-call overhead: one postMessage round-trip (around 0.1 to 0.4 ms in modern browsers). Fine for UI-rate workloads. For tight per-frame loops over thousands of fine-grained ops, prefer a Worker-side capability (Path A .wasm).

Worker bootstrap

When the runtime is cross-origin (page on demo.edgepython.com, runtime on runtime.edgepython.com), Chromium rejects new Worker(crossOriginUrl) even with type: 'module'. createWorker spawns from a same-origin Blob URL that dynamically import()s the cross-origin module. Same-origin imports use the direct path; createWorker auto-selects from import.meta.url. The Blob bootstrap buffers any postMessage arriving before worker.js installs its handler.

Module fetch lifecycle

load runs once per Worker; run can be called many times. compiler_lib.wasm is compiled once at load; a fresh instance is created per run so VM state cannot leak. Resolution is lazy: the compiler classifies each import and only the modules a run actually uses get fetched. Bare names resolve against the manifest chain (built-in defaults < user packages.json); manifests are resolution tables, not download lists, so a declared-but-unused package is never downloaded. Module bytes (.py / .wasm / packages.json) are cached across runs in the same Worker, prefetch skips fetched specs, 404'd manifests are remembered. Use clearCache() to drop both caches.

A spec the prefetch can't fetch or register (wrong scheme, a .wasm served as HTML, a malformed binary) aborts the run before it starts with a clear error, with an https:// hint for http:// or schemeless URL specs, instead of letting the VM fail later with not registered.

Layout

├── README.md
├── src
│   ├── cache
│   │   ├── idb.js
│   │   └── memory.js
│   ├── element.js
│   ├── env.js
│   ├── fetch.js
│   ├── index.js
│   ├── native.js
│   ├── prefetch.js
│   ├── rt.js
│   └── specs.js
└── worker
    ├── engine.js
    └── worker.js

Files

Path Purpose
src/index.js Public API. createWorker factory (main-thread).
src/element.js Public <edge-python> custom element. Wraps createWorker; reads host / imports from packages.json (host libraries load lazily on first import).
worker/engine.js Internal orchestrator (Worker only). load, run, pushEvent, reset, clearCache, dispose, setHostCallDelegate, setLoadHostDelegate.
src/env.js The 4 env.* imports compiler_lib declares: host_print, host_call_native, host_fetch_bytes, host_now_ns.
src/native.js Native module loader extension point + built-in Path A (wasm-pdk) loader + nativeTable.
src/prefetch.js Lazy BFS over the dependency graph; resolves each imported name and registers only the .py / .wasm / host modules a run uses.
src/defaults.js Built-in base manifest: official std (json, re) and host (dom, ...) packages, resolvable by bare name without packages.json.
src/fetch.js CAS-backed fetch with lockfile integrity check.
src/specs.js URL/spec helpers mirroring compiler_lib::modules::packages::manifest.
src/rt.js Handle codec wrappers (decodeStr, encodeInt, ...) for loaders.
src/cache/memory.js In-memory cache backend (per-Worker only).
src/cache/idb.js IndexedDB cache backend (persistent across sessions).
worker/worker.js Web Worker entry; postMessage protocol.

License

MIT OR Apache-2.0