feat: durable workflows engine runtime

.changeset/engine-runtime.md

@@ -0,0 +1,13 @@
+---
+"durable-workflows": minor
+---
+
+Engine runtime: `durableWorkflows(options)`. The connector that ties the authoring surface and execution host to a persistence store, exposing the full instance lifecycle — `create`, `get`, `continueWorkflow`, `terminate`, `evict`, `restart`, `dispose`, and `pendingPromises`.
+
+Each lifecycle call resolves to one replay turn: load the instance record + boundary cache from the `WorkflowStore`, resolve the pinned definition version via the store's `getDefinition`, hydrate (and cache, per pinned version) a runner mounted with the plugin shims + any `alias` re-export modules, execute with per-run handlers, then persist the grown cache and new status and return a `RunOutcome`. Storage is minimal (instances + one opaque cache blob each); retry, scheduling and wake-ups stay the caller's job; resume is plain re-execution.
+
+The store is ONE persistent world: instances, caches, and READ-ONLY definition access (`getDefinition(name, version?)` — no version means "active/latest", the pinned version on replay). Writing definitions (upload, versioning, rollback) is deliberately the application's own deploy layer against the same backend; the contract is that a handed-out `(name, version)` stays fetchable and byte-identical while instances pin it.
+
+Plugin handlers receive the structured `DurableHandlerInput` (`{ instanceId, workflow, run, stepId, payload }`) — the `durable-workflows:internal` `operation` shim now forwards the boundary key alongside the args so the engine can recover `stepId` (the kernel never hands a handler its key), and `payload` is the full forwarded argument list. The host now forwards iso4 resource limits (engine defaults ← `options.limits` ← per-definition `limits`).
+
+Ships an in-memory `memoryStore()` reference adapter (with a `deploy` seeding helper playing the deploy layer's role for tests).

AGENTS.md

@@ -4,7 +4,7 @@
 
 This repo is a **pnpm monorepo** (modeled on the [iso4](https://github.com/schplitt/iso4) repo, releases via [changesets](https://github.com/changesets/changesets)) containing two published packages:
 
-- **`durable-workflows`** — a composable durable workflow library, actively inspired by [Cloudflare Workflows](https://developers.cloudflare.com/workflows/) and built on top of [`iso4`](https://github.com/schplitt/iso4). Workflows run inside a sandbox with a step-based API whose results are persisted, enabling suspend/resume, retries, and event-driven continuation. It consumes `durable-isolates` (`workspace:*`). The **authoring surface** is shipped so far: two preloaded virtual modules — `durable-workflows:workflow` (author-facing `defineWorkflow({ run })` + `step.do(id, fn)`) and `durable-workflows:internal` (the shim-facing wrapper over the kernel, for plugin authors) — plus `durableWorkflowHost`, the execution host that wraps a `durable-isolates` host: `hydrate({ workflow, plugins })` mounts a definition with the core auto-injected, and `runner.execute({ input, cache })` runs one replay turn (input baked into a generated entry as JSON). The higher-level engine runtime (store adapters, the `durableWorkflows()` lifecycle factory in `types.ts`) is not built yet.
+- **`durable-workflows`** — a composable durable workflow library, actively inspired by [Cloudflare Workflows](https://developers.cloudflare.com/workflows/) and built on top of [`iso4`](https://github.com/schplitt/iso4). Workflows run inside a sandbox with a step-based API whose results are persisted, enabling suspend/resume, retries, and event-driven continuation. It consumes `durable-isolates` (`workspace:*`). Three layers ship: the **authoring surface** — two preloaded virtual modules, `durable-workflows:workflow` (author-facing `defineWorkflow({ run })` + `step.do(id, fn)`) and `durable-workflows:internal` (the shim-facing wrapper over the kernel, for plugin authors) — the **execution host** `durableWorkflowHost` (`hydrate({ workflow, plugins, limits })` mounts a definition with the core auto-injected; `runner.execute({ input, cache, handlers, limits })` runs one replay turn, input baked into a generated entry as JSON), and the **engine runtime** `durableWorkflows({ store, plugins, alias, limits, onEvent })` — the full instance lifecycle (`create`/`get`/`continueWorkflow`/`terminate`/`evict`/`restart`/`dispose`/`pendingPromises`) over a `WorkflowStore` adapter. The store is ONE persistent world: instances, boundary-cache blobs, and READ-ONLY definition access (`getDefinition(name, version?)` — instances pin the version they were created on; a handed-out `(name, version)` must stay fetchable and byte-identical). Writing definitions (upload/versioning/rollback) is deliberately app-layer, not engine scope. Plugin handlers get `DurableHandlerInput` (`{ instanceId, workflow, run, stepId, payload }`); the `operation` shim forwards its boundary key as the leading arg so the engine can recover `stepId` (the kernel never passes handlers their key), and `payload` is the full forwarded argument list. `memoryStore()` is the reference store adapter (with a `deploy` seeding helper standing in for the app's deploy layer).
 - **`durable-isolates`** — the replay kernel `durable-workflows` builds on: durably execute one isolate program over a **keyed cache** of boundaries. Implemented as a **memoize-by-key router** — in-sandbox shims form a `key` and call the primitives from `durable-isolates:internal`: `durableCall(key, name, …args)` (host-handler work), `durableLookup`/`durableCommit` (sandbox-side checkpoints) and the nestable `boundary(key, fn)`/`nextKey(name)` sugar over them. The ambient boundary prefix is carried through iso4's `AsyncLocalStorage` (`@iso4/sandbox` ≥ 0.3.0), so nested boundaries key deterministically whether they run sequentially or in parallel (`Promise.all`). The host answers a boundary from the cache by `key` or dispatches the `name` handler. A handler throwing `SuspendIsolate` suspends the run (waiting record + abort); resume is always **re-execution** (the waiting boundary re-dispatches and the handler consults host state — there is no delivery API and no tokens); `handle.suspend()` suspends externally (drains in-flight handler IO into the cache, for server teardown). Determinism is a documented contract, not an enforced check (no divergence detection). The caller owns storage, retry/eviction (cache surgery), and reacting to pending operations.
 
 The project uses ESM modules, Vitest for testing, ESLint for code quality, and tsdown for builds. Node `>=26`.
@@ -17,13 +17,16 @@ The project uses ESM modules, Vitest for testing, ESLint for code quality, and t
 packages/
   durable-workflows/      # Published as `durable-workflows`
     src/
-      index.ts            # Public entry — types + durableWorkflowHost + specifier constants
-      types.ts            # Engine/host public type surface (the not-yet-built engine runtime)
-      host.ts             # durableWorkflowHost: hydrate({ workflow, plugins }) → runner.execute({ input, cache })
+      index.ts            # Public entry — types + durableWorkflows + durableWorkflowHost + memoryStore + specifier constants
+      types.ts            # Engine/store/plugin/outcome public type surface
+      engine.ts           # durableWorkflows(): instance lifecycle over a WorkflowStore, per-version runner cache, handler wrapping
+      memory-store.ts     # memoryStore(): in-memory WorkflowStore + `deploy` seeding helper (tests)
+      host.ts             # durableWorkflowHost: hydrate({ workflow, plugins, limits }) → runner.execute({ input, cache, handlers, limits })
       shim.ts             # Source of the two virtual modules + coreModules bundle (internal)
       internal.ts         # `./internal` export — shim-facing types (operation/boundary) for plugin authors
       workflow.d.ts        # `./workflow` export — ambient `declare module 'durable-workflows:workflow'` (copied to dist)
     tests/authoring-surface.test.ts  # Authoring surface + host, against the real kernel
+    tests/engine.test.ts  # Engine lifecycle end-to-end (memory store + real sandbox)
     __snapshots__/tsnapi/ # tsnapi public-API snapshots (index, internal)
     package.json          # workspace:* dep on durable-isolates
     tsconfig.json         # standalone (no root base)
@@ -50,7 +53,7 @@ pnpm-workspace.yaml       # `packages/*` + dependency catalog
 .changeset/               # changesets config + release docs
 ```
 
-`durable-workflows` public exports: `.` (types + `durableWorkflowHost` + `INTERNAL_SPECIFIER`/`WORKFLOW_SPECIFIER`), `./internal` (shim-facing types), and `./workflow` (author-facing ambient `.d.ts`). The core shim sources and `coreModules` are internal — the host mounts them; callers never do.
+`durable-workflows` public exports: `.` (types + `durableWorkflows` + `durableWorkflowHost` + `memoryStore` + `INTERNAL_SPECIFIER`/`WORKFLOW_SPECIFIER`), `./internal` (shim-facing types), and `./workflow` (author-facing ambient `.d.ts`). The core shim sources and `coreModules` are internal — the host mounts them; callers never do.
 
 Each package owns its own `src/` (public API exported from `src/index.ts`), a standalone `tsconfig.json` (no shared root base), `tsdown.config.ts`, and `vitest.config.ts`. Shared root config is just ESLint and the pnpm catalog.
 
@@ -85,7 +88,7 @@ Root scripts use `pnpm -r --filter="./packages/*"`; `lint`/`lint:fix` run ESLint
 - Each package has its own `vitest.config.ts`; put tests inside that package's `tests/` directory (or alongside source under `src/`) — both are covered by the package `tsconfig.json` `include`, so tests are type-checked against the same config as source
 - Use the `*.test.ts` file naming convention
 - Run `pnpm test:run` from the root for all packages (no watch), or run it inside a single package
-- Both packages have real suites (`durable-isolates/tests/kernel.test.ts`, `durable-workflows/tests/authoring-surface.test.ts`) and neither sets `passWithNoTests`. The `durable-workflows` suite mounts the real shims on a `durable-isolates` runner via `durableWorkflowHost`, so it exercises the whole chain (workflow → `:workflow` → `:internal` → `durable-isolates:internal` → host)
+- Both packages have real suites (`durable-isolates/tests/kernel.test.ts`, `durable-workflows/tests/authoring-surface.test.ts` + `tests/engine.test.ts`) and neither sets `passWithNoTests`. The `durable-workflows` suites mount the real shims on a `durable-isolates` runner (via `durableWorkflowHost` directly, and via the full engine with a `memoryStore`), so they exercise the whole chain (engine → workflow → `:workflow` → `:internal` → `durable-isolates:internal` → host → handlers)
 
 Example test structure:
 

packages/durable-workflows/README.md

@@ -4,7 +4,7 @@ Durable workflows for JavaScript, inspired by [Cloudflare Workflows](https://dev
 
 Write a workflow as a plain async function. Each step's result is saved, so a workflow can pause for days, survive restarts and deploys, and continue right where it left off. No state machines, no manual bookkeeping.
 
-> Status: work in progress. The authoring surface and runner are here; the full engine (persistence, scheduling) lands next.
+> Status: work in progress. The authoring surface, the execution host and the engine (instance lifecycle over a pluggable store) are here; first-party capability plugins land next.
 
 ## Features
 
@@ -38,6 +38,43 @@ export default defineWorkflow({
 
 Run one loads the device, asks for approval, and pauses. Days later the approval arrives, the workflow replays through the saved step in milliseconds, applies the fix, and finishes. A deploy in between changes nothing.
 
+## Running workflows: the engine
+
+`durableWorkflows` runs instances of deployed workflow code over a store you provide:
+
+```ts
+import { durableWorkflows, memoryStore } from 'durable-workflows'
+
+const store = memoryStore() // or your own WorkflowStore adapter
+store.deploy('device-fix', 'v1', workflowCode) // in production your own deploy layer writes these rows
+
+const engine = durableWorkflows({
+  store,
+  plugins: {
+    'my:approvals': approvalsPlugin(myBackend), // keys are the import specifiers workflows use
+  },
+})
+
+// your triggers start instances — resolves the active version, pins it, runs the first turn
+app.post('/devices/:id/fix', async (req) => {
+  const outcome = await engine.create('device-fix', { deviceId: req.params.id })
+  // outcome.status: 'completed' | 'waiting' | 'failed'
+})
+
+// and your own wiring (webhook, cron, queue) resumes waiting instances:
+app.post('/approvals/:instanceId/decide', async (req) => {
+  await engine.continueWorkflow(req.params.instanceId)
+})
+```
+
+How the pieces divide:
+
+- **The store** (`WorkflowStore`) is one persistent world: instance records, each instance's saved step history, and read access to deployed workflow code (`getDefinition`). Implement it once against your database.
+- **Definitions are read-only to the engine.** Uploading, versioning and rollback belong to your app — you write code rows into the same backend the store reads. Instances pin the version they started on and always replay exactly that code, so rollbacks never disturb running instances. The one rule: never mutate or delete a version that instances still reference.
+- **Resuming is re-running.** `continueWorkflow(id)` replays the workflow over its saved history; a step that was waiting asks its plugin handler again, and the handler checks your systems (the approval row, the clock, the job status) to answer, keep waiting, or fail. There are no callbacks to register and nothing to inject.
+- **Scheduling is yours.** The engine never wakes anything up — your cron/webhooks/queues decide when to call `continueWorkflow`.
+- **Remediation:** `evict(id, stepId)` deletes a step (and everything after it) from history and replays; `restart(id)` replays from scratch; `terminate(id)` ends an instance.
+
 ## License
 
 MIT

packages/durable-workflows/__snapshots__/tsnapi/index.snapshot.d.ts

@@ -36,7 +36,6 @@ export interface DurableWorkflowsEngine {
 export interface DurableWorkflowsOptions {
   sandbox?: SandboxOptions;
   store: WorkflowStore;
-  resolveDefinition: (_: string, _?: string) => Promise<ResolvedDefinition | null>;
   plugins?: Readonly<Record<string, DurableWorkflowsPlugin>>;
   alias?: Readonly<Record<string, string>>;
   limits?: Partial<ResourceLimits>;
@@ -51,6 +50,9 @@ export interface FailedInstanceRecord extends InstanceRecordBase {
   status: "failed";
   error: SerializedError;
 }
+export interface MemoryWorkflowStore extends WorkflowStore {
+  deploy: (_: string, _: string, _: string, _?: Partial<ResourceLimits>) => void;
+}
 export interface PendingOperation {
   stepId: string;
   operation: string;
@@ -75,10 +77,12 @@ export interface WorkflowExecuteOptions {
   input?: unknown;
   cache: BoundaryCache;
   handlers?: PerExecuteHandlers;
+  limits?: Partial<ResourceLimits>;
 }
 export interface WorkflowHydrateOptions {
   workflow: string;
   plugins?: Readonly<Record<string, ModuleDefinition>>;
+  limits?: Partial<ResourceLimits>;
 }
 export interface WorkflowInstanceHandle {
   readonly id: string;
@@ -96,6 +100,7 @@ export interface WorkflowStore {
   deleteInstance: (_: string) => Promise<void>;
   getCache: (_: string) => Promise<BoundaryCache | null>;
   putCache: (_: string, _: BoundaryCache) => Promise<void>;
+  getDefinition: (_: string, _?: string) => Promise<ResolvedDefinition | null>;
 }
 // #endregion
 
@@ -146,6 +151,8 @@ export type RunOutcome = (RunOutcomeBase & {
 
 // #region Functions
 export declare function durableWorkflowHost(_?: DurableIsolatesOptions): DurableWorkflowHost;
+export declare function durableWorkflows(_: DurableWorkflowsOptions): DurableWorkflowsEngine;
+export declare function memoryStore(): MemoryWorkflowStore;
 // #endregion
 
 // #region Variables

packages/durable-workflows/__snapshots__/tsnapi/index.snapshot.js

@@ -3,6 +3,8 @@
  */
 // #region Functions
 export function durableWorkflowHost(_) {}
+export function durableWorkflows(_) {}
+export function memoryStore() {}
 // #endregion
 
 // #region Variables

packages/durable-workflows/package.json

@@ -56,6 +56,7 @@
   },
   "devDependencies": {
     "@schplitt/eslint-config": "catalog:",
+    "@types/node": "catalog:",
     "eslint": "catalog:",
     "tsdown": "catalog:",
     "tsnapi": "catalog:",