diff --git a/docs/superpowers/specs/2026-07-19-runtime-design.md b/docs/superpowers/specs/2026-07-19-runtime-design.md new file mode 100644 index 0000000..df646fa --- /dev/null +++ b/docs/superpowers/specs/2026-07-19-runtime-design.md @@ -0,0 +1,278 @@ +# Moroutine Runtime Design + +**Date:** 2026-07-19 +**Status:** Draft + +## Motivation + +Moroutine today is a library: you hold a pool object (`workers()`) or you get implicit +one-thread-per-function dedicated workers. This design reframes moroutine as a **runtime**: +a process-wide worker topology that application code lives inside. The goal is writing full +applications in a multi-threaded way — work pinned to certain threads, servers scaled across +threads, background tasks — with all workers part of a single whole and work scheduled +cooperatively across them. + +## Summary of decisions + +- One **global runtime** per process; defined as a module (like moroutines themselves), + registered from main, booted lazily on first dispatch, torn down automatically via unref. +- The **default runtime is just another runtime module**, shipped inside moroutine. +- **Main is coordinator only**: it dispatches, consumes results, and brokers peer channels; + tasks never run on main. +- **Workers are peers**: any worker can dispatch to any other over lazily-established + direct channels (full mesh potential, realized on demand). +- **One protocol everywhere**: the existing TaskMsg/callId dispatch protocol runs identically + main→worker and worker→worker. +- **Dedicated workers are removed**; bare `await task` dispatches to the global runtime. +- **`workers()` remains** as the explicitly-scoped tool for scripts, sharing all mechanisms + with the runtime; using only `workers()` never boots the global runtime. +- This is a **semver-major**. + +## 1. Concept & API + +### Runtime definition + +A runtime is defined in a side-effect-free module with deterministic identity, the same +pattern as `mo()`: + +```ts +// app-runtime.ts +import { defineRuntime } from 'moroutine'; +import { keyAffinity } from './balancer.ts'; + +export default defineRuntime(import.meta, { + size: 8, + balance: keyAffinity(), +}); +``` + +Moroutine ships its own internal default runtime module (`size: availableParallelism()`, +round-robin) — "default vs custom" is not a code path, only a question of which module ID +is in play. + +### Registration and lifecycle + +```ts +// main.ts +import { register } from 'moroutine'; +import appRuntime from './app-runtime.ts'; + +register(appRuntime); // must precede first dispatch +``` + +- Boot is **lazy**: the first dispatch (bare await or `runtime.run()`) boots whichever + runtime is registered — the default if none. +- `register()` after the runtime has booted **throws**. No silent reconfiguration. +- Teardown is **automatic**: workers are `unref()`'d when idle and `ref()`'d while work is + in flight (the pattern dedicated workers use today). The process exits naturally when main + finishes and no tasks are pending. +- **Graceful shutdown is opt-in** via `runtime.shutdown()` (signal → drain in-flight up to + a timeout → terminate), for servers that need coordinated drain. Scripts never call it. +- `register()` is a plain call in main; runtime definition modules stay side-effect free + (every thread imports them). + +### The ambient runtime handle + +Task code imports the generic handle from moroutine — never a specific runtime module — so +worker code is not coupled to any particular runtime: + +```ts +// stats.ts +import { mo, assign, runtime } from 'moroutine'; +import { recordMetric } from './metrics.ts'; + +export const processItem = mo(import.meta, async (item: Item) => { + await runtime.run(recordMetric(item.size)); // policy placement + await runtime.run(assign(runtime.workers[0], recordMetric(...))); // pinned +}); +``` + +`runtime` is a lazy view of "the global runtime this thread belongs to": on main, the +registered (or default) runtime; on a worker, bound by the boot handshake. It is a `Runner` +(same shape `workers()` returns): `runtime.run(task | task[])`, `runtime.workers`, +`runtime.signal`. + +### Bare await + +`await add(3, 4)` is sugar for dispatching to the global runtime — on any thread. Replaces +the dedicated-worker behavior. The thenable/async-iterable task machinery is unchanged; +only its target changes. + +### `workers()` mode + +`workers()` is unchanged in spirit: an explicitly scoped pool for script-like usage, owned +by a `using` block. It shares dispatch, handles, balancer machinery, and teardown code with +the runtime. If a program only uses `workers()`/`run()`, the global runtime never boots and +no mesh exists. Tasks running inside a `workers()` pool have no ambient reference to their +own pool (the `runtime` handle refers to the global runtime), so pool scheduling decisions +only ever occur on main. + +## 2. Topology & mesh + +Star-plus-mesh: main at the center as coordinator and broker; n workers as peers with +direct lazily-created channels. + +### Boot handshake + +Each worker spawns with `workerData` carrying: the runtime module ID, its own worker index, +pool size, and the runtime's shared state block (one `SharedArrayBuffer`: per-worker active +counts, watchdog words, balancer state). The worker imports the runtime module by ID, +reconstructs config, and binds the ambient `runtime` handle. + +### Lazy peer connection (brokered by main) + +1. Worker A's first dispatch to worker B: check peer table (`Map>`) + — miss → send `{ctrl: 'connect', peer: B}` up parentPort, store pending promise. +2. Main keys pairs by `(min, max)`. Unplumbed → create one `MessageChannel`, push + `{ctrl: 'peer', index, port}` to each side. Already plumbed/in-flight → no-op. +3. Both sides cache the port and resolve pending promises. Queued dispatches flow. + +Race analysis (verified empirically): + +- **Simultaneous requests for the same pair**: main serializes; second request is a no-op. +- **A sends before B attaches a listener**: safe — MessagePorts buffer delivered messages + until a listener attaches. Discipline: the `peer` push handler must wire the protocol + listener in the same tick, and never close a port that may hold queued traffic. No ack + protocol is needed; port delivery through main is the rendezvous. +- **B dead/terminating when A requests**: main answers with an error frame; A's dispatch + rejects rather than hangs. + +Measured costs (Apple Silicon, extrapolated linearly): cold first contact ≈ 2 ms per pair, +once ever; warm round-trip ≈ 0.01 ms. Full 64-worker mesh if fully realized: 2,016 channels, +~19 MB RSS, ~14 ms creation — 2–3% of what 64 workers themselves cost. Lazy means apps pay +only for pairs that actually communicate; `workers()`-only scripts pay nothing. + +### Protocol unification + +- `execute()` / `dispatchStream()` generalize from `Worker` to any port-like endpoint. +- parentPort traffic gains a control-frame discriminator (`connect` / `peer` / shutdown + vs task messages) in `setupWorker()` and the worker entry handler. +- **Same-worker shortcut**: a dispatch resolving (by pin or policy) to the executing worker + invokes locally — no port, no serialization. Invariant: the shortcut is never observable; + semantics (including arg deserialization behavior) are identical to the remote path. + +### Worker handles + +One uniform `WorkerHandle` interface on every thread: `{ index, exec, activeCount }`. + +- On main, handle `i` fronts the real `Worker`; on worker W, it fronts the peer port to + worker `i` (handle W itself is the local shortcut). +- `assign(runtime.workers[3], task)` works identically everywhere. Pins serialize as + `workerIndex` on the wire and re-resolve against the receiving thread's table. +- Handles carry their runtime ID; using a handle against a different runner/runtime throws. +- **`thread` is dropped** from `WorkerHandle` (its only in-tree consumers were two test + assertions; `serverThreads` uses handles purely as identity tokens). If a main-only need + for the raw `Worker` arises, expose it on a main-only surface then. +- `activeCount` moves into the shared state block (`Int32Array` slot per worker, 64-byte + padded against false sharing), updated by the dispatching thread. Measured: atomics + inc/dec ≈ 15 ns uncontended vs ≈ 23 ns for today's Map-based counting — an improvement, + and it makes `leastBusy` correct from every thread. + +## 3. Scheduling & balancers + +Balancer code identity comes for free from the runtime module import: every thread evaluates +the same runtime module, hence the same `select` logic. No separate registration. Only +**state** needs a mechanism, because per-thread module evaluation would give each thread its +own closure state. + +```ts +// Balancer interface (breaking change: state field + third select arg) +interface Balancer { + state?: DescriptorSchema; // schema, not allocation + select(workers: readonly WorkerHandle[], task: Task, state: S): WorkerHandle; +} +``` + +```ts +export function keyAffinity(): Balancer<{ cursor: Uint32Atomic }> { + return { + state: { cursor: uint32atomic }, + select(workers, task, { cursor }) { + let key: string | undefined; + if (isTask(increment, task)) key = task.args[0]; + else if (isTask(read, task)) key = task.args[0]; + if (key === undefined) return workers[cursor.add(1) % workers.length]; + return workers[hash(key) % workers.length]; + }, + }; +} +``` + +- At boot, main allocates declared `state` via `shared()` into the runtime SAB block; + workers bind their evaluated `select` to the reconstructed state. +- `select` must be pure over `(workers, task, state)` in runtime mode; closure state is + structurally safe only where scheduling is single-threaded — which `workers()` mode + guarantees — so plain closure balancers remain valid there, untouched. +- Built-ins (`roundRobin`, `leastBusy`) are reworked onto declared state (`leastBusy` reads + the shared active counts) so they work in both modes with unchanged call-site syntax. +- Dispatch flow on any thread: pinned? → resolve index → local shortcut or peer port; + otherwise `balancer.select(...)` → same. `isTask()`-based routing works unchanged + everywhere since it only inspects task descriptors. + +## 4. Liveness & failure + +- **Invariant: runtime code never sync-blocks a worker's event loop.** All parking is + `waitAsync`/promise-based. Consequence: awaiting a nested task parks a promise, not the + thread, so plain fork-join nested dispatch cannot deadlock the runtime — worst case it + loses parallelism, not liveness (tested at pool size 1). +- **Watchdog (opt-in)** via runtime config `{ watchdog: ms }`: workers heartbeat a shared + word; main scans; "all workers parked + no in-flight traffic for T" → crash loudly with a + who-waits-on-whom dump. Crash over hang. +- **Documented footguns (not prevented mechanically):** sync `Atomics.wait` inside user + tasks; cyclic channel topologies saturated at high-water. The watchdog catches both at + runtime. +- **Worker death** (recorded in TODO for follow-through): fail in-flight tasks with the + worker error as cause; main answers pending/future `connect` requests for the dead peer + with error frames; task-arg caches on that worker are lost (documented semantics). + No auto-restart in v1 — supervision is future work. + +## 5. Migration & compatibility + +Removed: + +- **Dedicated workers** (`dedicated-runner.ts`). Bare `await task` now targets the global + runtime. Deltas: one shared pool instead of one thread per function; task-arg caches are + per-worker, so a cached task-arg may recompute on first touch per worker instead of once + globally; pipelines get placement from policy (or explicit pins) instead of + thread-per-stage. +- `WorkerHandle.thread`. + +Changed: + +- `Balancer` interface: optional `state` schema, `select` gains a state argument. Closure + balancers keep working in `workers()` mode. +- Task-thenable dispatch target (dedicated worker → global runtime). + +Unchanged: + +- `mo()`, `workers()`/`run()`, `assign()`, `channel()`, `map()`, `transfer()`, `inert()`, + shared memory API, `moroutine/serve` (main-side; `serverThreads` consumes handles as + identity tokens only). + +## 6. Testing + +- **Unit:** broker (pair dedup, simultaneous requests, dead-peer error frames); peer table + promise states; handle reconstruction and runtime-mismatch errors; declared-state balancer + allocation/reconstruction; control-frame multiplexing alongside task traffic. +- **Integration:** nested dispatch worker→worker and worker→self (shortcut equivalence, + including arg-deserialization semantics); pinning from workers; recursive fork-join at + pool size 1 completes (liveness invariant); watchdog fires on an intentional user + deadlock; unref-based natural process exit; `register()` after boot throws; two runtimes' + handles cross-used throws; `workers()`-only programs never boot the runtime. +- **Perf gates:** dispatch overhead unchanged in `workers()` mode; same-worker shortcut + comparable to a local call; first-contact latency benchmarked; shared-counter dispatch + accounting no slower than today's Map-based counting. + +## Resolved design questions (with rationale) + +| Question | Decision | Why | +|---|---|---| +| Nested moroutine calls | Dispatch to global pool, policy-routable, same-worker optimized | Full peer model; fork-join is safe given async-parking invariant | +| Main thread role | Coordinator only | Keeps event-loop-owning thread responsive; simplest topology | +| Pool creation | Registered from main, lazy boot, unref auto-teardown | Zero ceremony; no silent reconfiguration (`register` after boot throws) | +| Default runtime | Internal runtime module in moroutine | "Just another runtime"; no special-case code path | +| Config consistency | Runtime module identity + shared-state descriptors | Same determinism trick as `mo()`; state solved separately from code | +| Worker addressing | Indexed handles, uniform on all threads | Minimal new concepts; index is the portable truth | +| Mesh setup | Lazy, brokered by main | ~2 ms once per pair; zero cost for non-peer apps; ~25 extra lines over eager | +| Peer protocol | Same TaskMsg protocol as main→worker | One protocol to maintain; maximal mechanism sharing | +| Explicit dispatch from tasks | `runtime.run()` (ambient handle) | Keeps `assign()` contract identical; no bare-await semantic fork |