fix(runtime,spec): the resume door asks the engine whether a status-less exit is repairable

.changeset/17541-resume-door-consults-inspection.md

@@ -0,0 +1,64 @@
+---
+'@objectstack/spec': minor
+'@objectstack/runtime': minor
+---
+
+the resume door's `repairable` is answered by the engine on the exits that stamp no status — `IAutomationService` declares the read-only `inspectConsumedSuspension` (#17541)
+
+Clause-②: yes (widening)
+
+The resume route's `400 FLOW_FAILED` details computed `repairable` as the single
+expression `status === 'stranded'`. That word is stamped on exactly one exit —
+the run that consumed its OWN pause and then threw downstream. The subflow
+DELEGATION exit stamps nothing on purpose: a caller resumes the PARENT, the
+signal is forwarded down, the child strands, and the parent frame answers
+`{ success: false, error, durationMs }`, because nothing re-arms an ancestor by
+resuming it and stamping `'stranded'` there would send an operator to retry a
+recovery that cannot succeed.
+
+Since the nested-chain restore landed, that parent's consumed pause IS
+journalled and one `restoreConsumedSuspension(parentRunId)` re-arms the whole
+chain leaf-first. So the wire answered `repairable: false` about a run the
+operator verb WILL repair, and a client written exactly as the reference page
+instructs closed it as terminal. Measured through the HTTP route, before and
+after, on the same parked delegation:
+
+```json
+before  400 { "error": { "code": "FLOW_FAILED",
+               "details": { "runId": "run_…", "repairable": false } } }
+after   400 { "error": { "code": "FLOW_FAILED",
+               "details": { "runId": "run_…", "repairable": true } } }
+```
+
+…while at that same instant the engine answered
+`inspectConsumedSuspension(runId) → { repairable: true, witness: 'journal' }`
+and `restoreConsumedSuspension(runId) → { restored: true, chain: [child, parent] }`.
+
+**`@objectstack/spec` — additive, `minor`.** `IAutomationService` declares the
+optional read-only member `inspectConsumedSuspension(runId)`, which
+`AutomationEngine` already implements publicly: would the restore verb have a
+consumed suspension to put back for this run? It re-arms nothing and reads the
+same two witnesses that verb reads, so what it calls repairable IS what that
+verb restores. The declared result is deliberately narrower than the
+implementation's, the way `restoreConsumedSuspension`'s already is — `reason` is
+typed as the string the implementation answers, not as an enumeration this
+contract would have to keep in step, and the engine's wider type satisfies it
+under `implements`. `ResumeFailureDetailsSchema.repairable`'s `.describe()` is
+rewritten to the truth and the generated reference page regenerated with it. No
+key is added, renamed or retired on any wire schema.
+
+**`@objectstack/runtime` — the door.** On a `400 FLOW_FAILED` whose result
+carries a `status`, that stamp still decides, and the engine is not consulted at
+all. On a result that carries none, the door asks the declared member and relays
+its `repairable`. Both ways of not getting an answer are FAIL-CLOSED: a service
+that declares no inspection member answers `false` exactly as it did before, and
+an inspection that REJECTS (a store it could not read) answers `false` and says
+so once at `warn` — an unreadable store is UNKNOWN, not "nothing to restore",
+and it is never allowed to replace the `400` the caller asked for with a `500`.
+
+⛔ The fence is untouched: a cascade-failed ancestor is still never STAMPED
+`'stranded'`. Its repairability is carried by the journal and REPORTED by the
+inspection, which is exactly why the door asks instead of reading a word. ⛔ And
+no new `AutomationResult.status` member is minted for this exit — there is
+nothing new for a client to learn, and `details.repairable` is the member a
+client was already told to branch on.

content/docs/api/client-sdk.mdx

@@ -767,17 +767,22 @@ try {
   if (!isApiError(err) || err.code !== 'FLOW_FAILED') throw err;
   const verdict = ResumeFailureDetailsSchema.safeParse(err.details);
   if (verdict.success && verdict.data.repairable) {
-    // verdict.data.status === 'stranded' — verdict.data.runId names the run
-    // an operator can re-arm; offer that instead of closing as terminal.
+    // verdict.data.runId names the run an operator can re-arm; offer that
+    // instead of closing as terminal. Read `repairable` alone: `status` is
+    // absent on the subflow delegation failure, which is repairable without
+    // ever carrying the word `stranded`.
   }
 }
 ```
 
 `repairable` is always present on the resume door's `400` — `false` on a plain
 terminal failure, deliberately, so an absent member reads as an older server
-rather than as "not repairable". `status` is the engine's own stamp
-(`'stranded'` or `'failed'`), forwarded when the engine set one and never
-invented by the door. The code stays `FLOW_FAILED`: there is no `FLOW_STRANDED`
+rather than as "not repairable". It is not a reading of `status`: where the
+engine stamped one, that stamp decides; where it stamped none — the subflow
+delegation failure — the door asks the engine's read-only inspection, so
+`repairable: true` can arrive with no `status` beside it. `status` is the
+engine's own stamp (`'stranded'` or `'failed'`), forwarded when the engine set
+one and never invented by the door. The code stays `FLOW_FAILED`: there is no `FLOW_STRANDED`
 sibling, and the trigger door's `400` carries `errorMessage` / `summary` only —
 it never resumes, so "repairable" has no referent there.
 

content/docs/automation/flows.mdx

@@ -790,11 +790,18 @@ per-node `summary`:
                     "errorMessage": "…", "summary": { … } } } }
 ```
 
-Branch on `details.repairable`, never on the message: it is `true` exactly when
-`status` is `stranded`, and it is **always present** on this route's `400` —
-`false` on a plain terminal failure, so an absent member reads as an older
-server rather than as "not repairable". `status` is the engine's own stamp,
-forwarded when it set one and never invented by the route. The code does not
+Branch on `details.repairable`, never on the message. It is **always present**
+on this route's `400`, and it is the engine's own answer rather than a reading
+of the status word: where the engine stamped a `status`, that stamp decides
+(`stranded`); where it stamped none, the route asks the engine's read-only
+inspection and relays the verdict. The second case is a real exit, not a
+fallback — a subflow **delegation** failure carries no status, because nothing
+re-arms an ancestor by resuming it, and yet the ancestor's consumed pause is
+journalled and one operator restore re-arms that whole chain, so it answers
+`repairable: true` with no `status` beside it. `false` on a plain terminal
+failure, so an absent member reads as an older server rather than as "not
+repairable". `status` is the engine's own stamp, forwarded when it set one and
+never invented by the route. The code does not
 change (there is no `FLOW_STRANDED`), and the trigger route's `400` carries
 `errorMessage` / `summary` only — it never resumes, so the question has no
 referent there. The shape is `ResumeFailureDetailsSchema` in

content/docs/references/api/automation-api.mdx

@@ -583,7 +583,7 @@ const result = AutomationApiErrorCode.parse(data);
 | :--- | :--- | :--- | :--- |
 | **runId** | `string` | ✅ | The run the resume was addressed to - the run that failed, and on the `stranded` arm the run an operator verb can re-arm. Named so a caller acts on an identifier instead of parsing one out of the message |
 | **status** | `Enum<'failed' \| 'stranded'>` | optional | The engine's own lifecycle verdict for the run, forwarded verbatim when the producer stamped one and never synthesised by the door - absent when the engine reported no status (a subflow child that failed terminally, an engine that predates the discriminator). `stranded` is the terminally-failed-but-repairable run of `AutomationResult.status`; `failed` says the run ran and was rejected. These two terminal-failure members of that union are the only ones that can reach a 400 |
-| **repairable** | `boolean` | ✅ | Whether the engine says this run can still be re-armed by an operator verb - `true` exactly when `status` is `stranded`, derived from the engine's discriminator and never from the message text. Always present on this arm: `false` is the honest answer for every other exit, the ones that report no status included, because an absent member would be indistinguishable from a server that predates this field, and promising a repair verb that will refuse is worse than promising nothing |
+| **repairable** | `boolean` | ✅ | Whether the engine says this run can still be re-armed by an operator verb. Answered from the engine, two ways, and never from the message text: where the engine stamped a `status`, that word decides (`stranded` is the run whose OWN pause a resume consumed before a downstream node threw); where it stamped none, the door asks the engine's read-only inspection (`IAutomationService.inspectConsumedSuspension`) and relays its verdict. The second half is not a fallback but the honest answer for a real exit: a subflow DELEGATION failure carries no status, because nothing re-arms an ancestor by resuming it, and yet the ancestor's consumed pause is journalled and the restore verb re-arms that chain as one unit. Always present on this arm: an absent member would be indistinguishable from a server that predates this field. Every way of not getting an answer is fail-closed - a service that declares no inspection member, and a store the inspection could not read - because promising a repair verb that will refuse is worse than promising nothing |
 
 
 ---

packages/runtime/src/domains/automation-resume-delegation-repairable.test.ts

@@ -0,0 +1,194 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * #17541 — the resume door's `repairable` is answered by the ENGINE on the
+ * exits that stamp no `status`, instead of being read off a stamp that is not
+ * there (the maintainer's ruling on this card, 2026-09-17, letter 1).
+ *
+ * ## The defect, as a wire reading
+ *
+ * `repairable` used to be the single expression `status === 'stranded'`. That
+ * word is stamped on exactly one exit — the run that consumed its OWN pause
+ * and then threw downstream. The subflow **delegation** exit deliberately
+ * stamps nothing: a caller resumes the PARENT, `resumeInternal` forwards the
+ * signal down, the child strands, and the parent frame answers
+ * `{ success: false, error, durationMs }`, because nothing re-arms an ancestor
+ * by resuming it and stamping `'stranded'` there would send an operator to
+ * retry a recovery that cannot succeed. Since #15222 that parent's consumed
+ * pause IS journalled and `restoreConsumedSuspension(parentRunId)` re-arms the
+ * chain as one unit — so the wire answered `repairable: false` about a run the
+ * operator verb WILL repair, and a client written exactly as the docs instruct
+ * closed it as terminal.
+ *
+ * ## What this file pins, and what it deliberately does NOT
+ *
+ * This is the DOOR's shaping, with a fake service, so every arm is reachable
+ * by name — including the two that a real engine will not produce on demand (a
+ * host with no inspection member; a store the inspection cannot read). The
+ * end-to-end fact, driven through the real engine and the HTTP route, is
+ * `@objectstack/verify`'s `automation-resume-delegation-repairable.test.ts`;
+ * the engine's own halves are `service-automation`'s
+ * `nested-strand-chain-restore.test.ts` and
+ * `consumed-suspension-inspection.test.ts`.
+ *
+ * ⛔ The fence #15222 was dispatched with is untouched and is not re-pinned
+ * here: a cascade-failed ancestor is still never STAMPED `'stranded'`. Its
+ * repairability is carried by the journal and REPORTED by the inspection,
+ * which is precisely why the door has to ask instead of reading a word.
+ *
+ * ⛔ And the door asks a member the CONTRACT declares — `IAutomationService`
+ * gained the optional read-only `inspectConsumedSuspension` for this card.
+ * Reaching for an undeclared member would be the fail-open shape the route's
+ * own `501` arm exists to prevent, so every way of not getting an answer here
+ * is FAIL-CLOSED: no member, and a rejected read, both answer `false`.
+ *
+ * The sibling file `automation-resume-stranded-details.test.ts` is the CONTROL
+ * for this change and is deliberately left untouched: every non-delegation
+ * exit it pins — the stamped `'stranded'`, the stamped `'failed'`, the
+ * status-less arm on a service with no inspection member, the regex control,
+ * the trigger door's absent member — still answers exactly what it did.
+ */
+
+import { describe, it, expect, vi, afterEach } from 'vitest';
+
+import { ResumeFailureDetailsSchema } from '@objectstack/spec/api';
+import type { AutomationResult, IAutomationService } from '@objectstack/spec/contracts';
+
+import { HttpDispatcher } from '../http-dispatcher.js';
+
+const CTX = { request: {}, executionContext: { userId: 'user_1' } } as any;
+const RESUME = '/parent_flow/runs/run_parent/resume';
+
+/**
+ * The engine's answer on the delegation exit, measured through the real
+ * engine while this card was written: the parent frame carries no `status`,
+ * no `errorMessage` and no `summary` — only the subflow failure text.
+ */
+const DELEGATION_FRAME: AutomationResult = {
+    success: false,
+    error: "subflow run 'run_child' (child_flow) failed: update_record(crm_leave_request) failed: Record 9SEmlyRfw8D9-J7Z not found",
+    durationMs: 7,
+};
+
+type Inspect = NonNullable<IAutomationService['inspectConsumedSuspension']>;
+
+function makeDispatcher(resumeResult: AutomationResult, inspect?: Inspect) {
+    const spies = {
+        resume: vi.fn(async () => resumeResult),
+        ...(inspect ? { inspectConsumedSuspension: vi.fn(inspect) } : {}),
+    };
+    const services: Record<string, unknown> = { automation: spies };
+    const resolve = (name: string) => services[name];
+    const kernel: any = {
+        getService: resolve,
+        getServiceAsync: async (name: string) => resolve(name),
+        context: { getService: resolve },
+    };
+    return { dispatcher: new HttpDispatcher(kernel), spies };
+}
+
+afterEach(() => { vi.restoreAllMocks(); });
+
+describe('#17541 — the status-less resume exit asks the engine whether the run is still repairable', () => {
+    it('DELEGATION — a frame with no status answers repairable: true when the engine holds the consumed suspension', async () => {
+        const { dispatcher, spies } = makeDispatcher(
+            DELEGATION_FRAME,
+            async (runId: string) => ({ repairable: true, runId }),
+        );
+
+        const result = await dispatcher.handleAutomation(RESUME, 'POST', {}, CTX);
+
+        // The door asked about the run it was addressed to — the path's
+        // `:runId`, which on this exit IS the run whose pause was consumed.
+        expect(spies.inspectConsumedSuspension).toHaveBeenCalledWith('run_parent');
+        expect(result.response?.status).toBe(400);
+        const error = result.response?.body?.error;
+        expect(error?.code).toBe('FLOW_FAILED');
+        // ⛔ Still no `status`: the door relays the producer's stamp and the
+        // producer stamped none. `repairable` is the answer, not the word.
+        expect(error?.details).toEqual({ runId: 'run_parent', repairable: true });
+        expect(ResumeFailureDetailsSchema.safeParse(error?.details).success).toBe(true);
+        // Not prose: the message names the subflow failure, not the verdict.
+        expect(error?.message).not.toMatch(/repairable/i);
+    });
+
+    it('FIRING CONTROL — the same status-less arm answers false when the engine says the suspension is gone', async () => {
+        // Without this the pin above would pass over a door that simply
+        // flipped the status-less arm to `true` and never asked anything.
+        const { dispatcher, spies } = makeDispatcher(
+            DELEGATION_FRAME,
+            async (runId: string) => ({ repairable: false, runId, reason: 'NO_CONSUMED_SUSPENSION' }),
+        );
+
+        const result = await dispatcher.handleAutomation(RESUME, 'POST', {}, CTX);
+
+        expect(spies.inspectConsumedSuspension).toHaveBeenCalledWith('run_parent');
+        expect(result.response?.status).toBe(400);
+        expect(result.response?.body?.error?.details).toEqual({ runId: 'run_parent', repairable: false });
+    });
+
+    it('FAIL-CLOSED — a service that declares no inspection member answers false, exactly as it did before this card', async () => {
+        const { dispatcher } = makeDispatcher(DELEGATION_FRAME);
+
+        const result = await dispatcher.handleAutomation(RESUME, 'POST', {}, CTX);
+
+        expect(result.response?.status).toBe(400);
+        expect(result.response?.body?.error?.details).toEqual({ runId: 'run_parent', repairable: false });
+    });
+
+    it('FAIL-CLOSED — an inspection that REJECTS still answers the 400 it was asked for, with repairable false and one warning', async () => {
+        // An unreadable store is UNKNOWN, not "nothing to restore" — so it is
+        // said out loud once. ⛔ And it never replaces the answer: the caller
+        // asked about a run that failed, and turning a store outage into a
+        // 500 would withhold that in order to report a detail.
+        const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
+        const { dispatcher } = makeDispatcher(
+            DELEGATION_FRAME,
+            async () => { throw new Error('connection reset'); },
+        );
+
+        const result = await dispatcher.handleAutomation(RESUME, 'POST', {}, CTX);
+
+        expect(result.response?.status).toBe(400);
+        expect(result.response?.body?.error?.code).toBe('FLOW_FAILED');
+        expect(result.response?.body?.error?.details).toEqual({ runId: 'run_parent', repairable: false });
+        const said = warn.mock.calls.map(c => String(c[0])).filter(l => l.includes('inspectConsumedSuspension'));
+        expect(said).toHaveLength(1);
+        expect(said[0]).toContain('connection reset');
+        expect(said[0]).toContain('run_parent');
+    });
+
+    it('CONTROL — a stamped `stranded` is answered by the STAMP: repairable true, and the engine is never asked', async () => {
+        const { dispatcher, spies } = makeDispatcher(
+            { success: false, error: 'tail blew up', durationMs: 12, status: 'stranded' },
+            async (runId: string) => ({ repairable: false, runId, reason: 'NO_CONSUMED_SUSPENSION' }),
+        );
+
+        const result = await dispatcher.handleAutomation(RESUME, 'POST', {}, CTX);
+
+        // The inspection would have answered `false` here. It is not consulted,
+        // so the stamp is demonstrably what decided — and the pre-existing
+        // exits keep costing exactly one engine call.
+        expect(spies.inspectConsumedSuspension).not.toHaveBeenCalled();
+        expect(result.response?.body?.error?.details).toEqual({
+            runId: 'run_parent', status: 'stranded', repairable: true,
+        });
+    });
+
+    it('CONTROL — a stamped `failed` is answered by the STAMP: repairable false, and the engine is never asked', async () => {
+        const { dispatcher, spies } = makeDispatcher(
+            { success: false, error: 'rejected', durationMs: 3, status: 'failed' },
+            async (runId: string) => ({ repairable: true, runId }),
+        );
+
+        const result = await dispatcher.handleAutomation(RESUME, 'POST', {}, CTX);
+
+        // The inspection would have answered `true` here — a stamped `failed`
+        // is a run that ran and was rejected, and no reading of a leftover
+        // snapshot is allowed to overturn the producer's own verdict.
+        expect(spies.inspectConsumedSuspension).not.toHaveBeenCalled();
+        expect(result.response?.body?.error?.details).toEqual({
+            runId: 'run_parent', status: 'failed', repairable: false,
+        });
+    });
+});