Skip to content

Commit 8341ed2

Browse files
Trumpclaude
andauthored
feat(automation): once-per-tick-window dispatch claims for scheduled flows (#16585)
* feat(automation): once-per-window dispatch claims for scheduled flows A `schedule` (cron) flow now takes a persisted `(flow, tick-window)` claim in the same `sys_flow_dispatch` ledger a `time_relative` flow claims per `(flow, window, record)`, and settles that claim with the run's outcome. That closes the two live duplicate-delivery doors: a restart inside a tick window, and an operator `IJobService.replay()` of a window that was already delivered. `DbJobAdapter.replay` refuses a delivered window with the ADR-0112 `RESOURCE_CONFLICT` / 409 envelope the contract declares, and `{ force: true }` is the door past it. The error-isolation catch in the schedule trigger keeps protecting the ticker exactly as before; it stopped being silent, recording the throw on the claim as a failed outcome instead of leaving the run indistinguishable from a delivered one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37 * test(trigger-schedule): pin the ticker property on the trigger's own handler Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37 * chore: changeset for the scheduled-flow dispatch-claim ledger Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37 * fix(automation): canonical by-id update for the claim settle, and split the pins by subject The ledger's settle() wrote update(object, id, data, options); no engine dispatches on that four-argument form. It is now the by-id payload shape (update(object, { id, ...fields }, options)) the ObjectQL engine takes, and the test doubles route through assertEngineUpdateDispatch so a fake can never again be looser than the engine it stands in for. The DbJobAdapter.replay pins move to service-job's own suite: reaching them from trigger-schedule needed an entry in the shrink-only test-source-alias registry, and that package's rootDir forecloses the paths route. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37 * fix(trigger-schedule): bound the replay pass to one per flow A pass a job service asked for and then abandoned no longer outlives its window: at most one outstanding pass per bound flow, dropped with the binding. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37 * test(service-job): route the replay-guard fake's update through the engine predicate Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37 * fix(automation): succeeded is absorbing in the dispatch-claim ledger Patch round 1 on the contract review. settle() refuses succeeded -> failed in both stores and in the engine's in-process fallback: a forced replay that throws must not rewrite a delivered window, because that silently reopens the unforced re-delivery door this card exists to close. failed -> succeeded stays required. The two headers that recorded the immutability retirement stated a rule the code did not hold; both now state the rule as implemented. Barrel surface narrowed to what has consumers, and ReplayGuard is nameable from service-job. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37 * test(automation): pin the counting engine double to the dispatch predicate It restates update() rather than passing the base double's through, so it is a double in its own right and carries the predicate itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37 * docs(automation): correct four statements this PR's own prose got wrong Text only; no behaviour change and no new pins. - settle()'s absorbing rule is enforced read-then-write against the persisted store, so it is not atomic. The window is named rather than closed: the engine's by-id update cannot express a conditional write. - the replay-pass residue comment claimed a later fire clears an abandoned pass. It does not -- the handler deletes only on a key match -- so the pass outlives every fire in every other window, inert throughout. - the readDispatch degradation warning described the both-absent ledger while reading as if it covered a store with settle() but no read(), which is weaker still. - the once-schedule changeset line asserted an unmeasured restart behaviour instead of the measured consequence: a replay before the due instant claims the single window, so the real fire then no-ops. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37 --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent 55bbd92 commit 8341ed2

18 files changed

Lines changed: 1849 additions & 71 deletions
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
"@objectstack/trigger-schedule": minor
3+
"@objectstack/service-automation": minor
4+
"@objectstack/service-job": minor
5+
---
6+
7+
A scheduled (cron) flow is now delivered once per tick window, and replaying a window that was already delivered is refused instead of silently sent again.
8+
9+
A `time_relative` flow has taken a persisted dispatch claim per `(flow, window, record)` since #10220, so per-record once-only delivery is free for it. A `schedule` flow runs once per tick with no record and had no claim surface at all, so "this batch already went out" fell back to whatever each app remembered for itself. A scheduled digest that was replayed by an operator, or whose process restarted inside its window, delivered twice.
10+
11+
Scheduled flows now claim `(flow, tick-window)` in the same `sys_flow_dispatch` ledger, and settle that claim with what the run turned into:
12+
13+
- **A second fire inside one window does nothing.** The window key is a pure function of the schedule descriptor and the clock — the previous occurrence of the very same cron expression in the very same timezone, computed with the same library the job adapter schedules with — so a restart inside the window computes the same key and hits the same claim.
14+
- **`IJobService.replay()` refuses a delivered window**, with the ADR-0112 envelope its contract declares: `code: 'RESOURCE_CONFLICT'`, `status: 409`, and a message naming the window and the claim that refused it. The promise rejects — an operator who presses replay and sees nothing happen is exactly the outcome this replaces.
15+
- **`replay(name, data, { force: true })` sends anyway.** The duplicate is the operator's, taken knowingly.
16+
- **A window whose claim is absent, failed or unsettled re-runs** on a plain `replay()`, with no force needed. A job that takes no claim at all — every job that is not a scheduled flow — is the absent row and behaves exactly as before.
17+
- **`succeeded` is absorbing.** A replay that repairs a failed window records `succeeded`, so the next unforced replay is refused. A *forced* replay that throws leaves the window recorded delivered rather than rewriting it to `failed` — otherwise a failed re-send would silently reopen the unforced re-delivery door. An operator whose forced replay failed forces again.
18+
- **A `once` schedule now has a tick window too** — the single instant it is due, which is one window for the job's whole life. The visible consequence is on replay: an operator who replays a one-shot job *before* its due instant claims that single window, so the real fire then finds the claim and does nothing. Previously both ran.
19+
20+
The error-isolation `catch` that keeps a throwing flow from crashing the ticker is unchanged and still swallows. What it no longer does is leave the run indistinguishable from a delivered one: the throw settles the window's claim as `failed`, so a replay repairs it.
21+
22+
`sys_flow_dispatch` gains two optional columns, `outcome` and `settled_at`. Rows written before this release read as unsettled, which reads as not delivered — the safe direction, since a replay of one re-runs rather than being refused. Only `schedule:` claims are ever settled; a `time_relative` sweep's rows stay `null` by design.
23+
24+
⚠️ **If you manage this table's DDL out of band** — anything other than letting the platform sync `sys_flow_dispatch` from its object definition — add `outcome` (text) and `settled_at` (datetime) yourself before upgrading. Without them every `settle()` throws against the driver. Dispatch dedup still works and no flow fails (the settle is best-effort and logged), but no claim ever records an outcome, so the replay refusal never fires and this release's headline change is silently absent.
25+
26+
Interface changes for hosts that implement the ledger themselves:
27+
28+
- `FlowDispatchStore` gains **optional** `settle()` and `read()`. A store without them still deduplicates; it announces once that the refusal cannot fire.
29+
- `FlowDispatchStoreEngine` — the narrow ObjectQL slice the bundled store demands — now **requires** `update` alongside `find` and `insert`. A custom engine adapter typed against it must add the method.
30+
- New exported types: `FlowDispatchClaim` and `FlowDispatchOutcome` from `@objectstack/service-automation`; `ReplayGuard` and `ReplayGuardDecision` from `@objectstack/service-job` (the parameter type of `DbJobAdapter.setReplayGuard`, exported so it can be named); `ScheduleDispatchLedger`, `ScheduleDispatchClaim`, `ScheduleDispatchOutcome`, `ReplayGuard` and `ReplayGuardDecision` from `@objectstack/trigger-schedule`.
31+
- `croner` moves from a devDependency to a **dependency** of `@objectstack/trigger-schedule`, which now imports it at runtime to compute the cron tick window. It is already a runtime dependency of `@objectstack/service-job` at the same range, so the platform's dependency set does not grow.

‎content/docs/permissions/tenant-audit-census.mdx‎

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ are reported as `undecidable` rather than assumed either way.
9898

9999
The same holds twice over for the context. An options argument spelled as a
100100
literal can be read; one spelled `options`, `{ ...opts }`, or handed through a
101-
forwarding shim cannot, and **67 of the 222 sites are spelled that way**. A
101+
forwarding shim cannot, and **67 of the 223 sites are spelled that way**. A
102102
context resolved from an inline literal or a local `const` can be tested for
103103
`isSystem`; one arriving from a helper call cannot.
104104

@@ -147,10 +147,10 @@ reproduce them. Where it disagrees, it disagrees on the page:
147147

148148
| carried figure | where it survives | this census |
149149
| :--- | :--- | ---: |
150-
| 175 write call sites | quoted in the merged changeset | **222** |
150+
| 175 write call sites | quoted in the merged changeset | **223** |
151151
| 24 carrying no tenant context | quoted in the merged changeset | **9** provable and tenancy-enabled; **32** more whose options argument is unreadable |
152-
| 127 of 175 statically decidable, 48 runtime-parameter-name sites | restated on the `isSystem`-scoping card | **148 of 222** decidable, **74** undecidable |
153-
| 135 (77%) silenced by the `isSystem` guard before the posture gate | the lost issue body — **no surviving corroboration** | **not reproduced**: 104 decidably elevated, 0 decidably not, 101 undecidable |
152+
| 127 of 175 statically decidable, 48 runtime-parameter-name sites | restated on the `isSystem`-scoping card | **149 of 223** decidable, **74** undecidable |
153+
| 135 (77%) silenced by the `isSystem` guard before the posture gate | the lost issue body — **no surviving corroboration** | **not reproduced**: 105 decidably elevated, 0 decidably not, 101 undecidable |
154154
| 141 and 132, two independent re-derivations | the card that filed this work | — |
155155

156156
**The differences are not reconciled, and deliberately so.** The old census's
@@ -161,17 +161,17 @@ at any commit.
161161

162162
Two structural facts do plausibly widen this reading against any hand or regex
163163
one, and both are counted in the generated tables below: the 45 sites reached
164-
through an erased (`any`) receiver, and the 39 that name their object through a
164+
through an erased (`any`) receiver, and the 40 that name their object through a
165165
`const` rather than inline. An instrument that read either the way a person does
166166
would report a smaller number and would not say so.
167167

168168
The fourth row is the one worth flagging to anyone citing it. **The 135 / 77%
169169
figure has no surviving corroboration anywhere in the tree.** This census reads
170-
104 of 222 (47%) as decidably elevated, with 101 more whose elevation is a
170+
105 of 223 (47%) as decidably elevated, with 101 more whose elevation is a
171171
run-time fact — so the claim is neither confirmed nor refuted, and the honest
172172
answer is that a static reading cannot settle it.
173173

174-
⇒ **Cite `9 / 222`, and say what it is**: the sites whose options argument was
174+
⇒ **Cite `9 / 223`, and say what it is**: the sites whose options argument was
175175
READ and holds no tenant context, against a decidably tenancy-enabled object.
176176
That is the control's provable yield surface. ⛔ Do not cite it as "the sites
177177
without tenant context" — **32 further sites** have an options argument this
@@ -183,29 +183,29 @@ cannot read, and they are neither in nor out.
183183

184184
| what | count |
185185
| :--- | ---: |
186-
| write call sites on the application surface | **222** |
187-
| …whose object name is statically decidable | 148 |
186+
| write call sites on the application surface | **223** |
187+
| …whose object name is statically decidable | 149 |
188188
| …whose object name is chosen at run time | 74 |
189-
| …against an object with tenancy ENABLED | 148 |
189+
| …against an object with tenancy ENABLED | 149 |
190190
| …against an object that declares tenancy off | 0 |
191-
| threading a tenant context | 138 |
191+
| threading a tenant context | 139 |
192192
| PROVABLY carrying none (options read, no context key) | **17** |
193193
| …of those, against a decidably tenancy-enabled object | **9** |
194194
| options argument UNREADABLE — may or may not carry one | 67 |
195195
| …of those, against a decidably tenancy-enabled object | 32 |
196-
| threading a decidably ELEVATED (`isSystem`) context | 104 |
196+
| threading a decidably ELEVATED (`isSystem`) context | 105 |
197197
| threading a context that is decidably NOT elevated | 0 |
198198
| threading a context whose elevation is a run-time fact | 101 |
199199

200200
| how the instrument reached the site | count |
201201
| :--- | ---: |
202-
| receiver carried a readable engine type | 177 |
202+
| receiver carried a readable engine type | 178 |
203203
| receiver erased, placed by the object NAME | 19 |
204204
| receiver erased, placed by an `object: string` PARAMETER | 15 |
205205
| receiver erased, placed by an `UNTYPED_RECEIVERS` row | 11 |
206206

207207
| object name spelled inline | 109 |
208-
| object name spelled through a `const` | 39 |
208+
| object name spelled through a `const` | 40 |
209209
| object name is an `object: string` parameter | 19 |
210210
| object name is some other run-time expression | 55 |
211211

@@ -224,13 +224,13 @@ holds still. They are required to be HERE and to say WHEN they were true;
224224
their values are not compared. The reasoning, and the measurement behind it,
225225
are in `scripts/check-tenant-audit-census.mjs`.
226226

227-
Measured on 2026-09-05 at `63a1a410e`.
227+
Measured on 2026-09-07 at `9cefca9a3`.
228228

229229
| corpus scale (not enforced) | count |
230230
| :--- | ---: |
231-
| tracked non-test sources scanned | 548 |
232-
| engine-shaped types recognised | 58 |
231+
| tracked non-test sources scanned | 557 |
232+
| engine-shaped types recognised | 59 |
233233
| declared objects in the registry | 298 |
234-
| same-named calls subtracted as non-engine | 134 |
234+
| same-named calls subtracted as non-engine | 137 |
235235

236236
{/* END GENERATED: tenant-audit-census */}

‎docs/audits/2026-08-tenant-audit-write-call-sites.counts.md‎

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -29,17 +29,17 @@ silent, and `node scripts/tenant-audit-census.mjs --write` is the resolution.
2929

3030
| Measure | Value |
3131
|---|---:|
32-
| Write call sites | 222 |
33-
| Object name statically decidable | 148 |
32+
| Write call sites | 223 |
33+
| Object name statically decidable | 149 |
3434
| Object name chosen at run time | 74 |
35-
| Against a tenancy-enabled object | 148 |
35+
| Against a tenancy-enabled object | 149 |
3636
| Against an object declaring tenancy off | 0 |
37-
| Threading a tenant context | 138 |
37+
| Threading a tenant context | 139 |
3838
| Provably carrying none | 17 |
3939
| …and decidably tenancy-enabled | 9 |
4040
| Options argument unreadable | 67 |
4141
| …and decidably tenancy-enabled | 32 |
42-
| Threading a decidably elevated context | 104 |
42+
| Threading a decidably elevated context | 105 |
4343
| Threading a decidably non-elevated context | 0 |
4444
| Threading a context of undecidable elevation | 101 |
4545

@@ -52,14 +52,14 @@ holds still. They are required to be HERE and to say WHEN they were true;
5252
their values are not compared. The reasoning, and the measurement behind it,
5353
are in `scripts/check-tenant-audit-census.mjs`.
5454

55-
Measured on 2026-09-05 at `63a1a410e`.
55+
Measured on 2026-09-07 at `9cefca9a3`.
5656

5757
| corpus scale (not enforced) | count |
5858
| :--- | ---: |
59-
| tracked non-test sources scanned | 548 |
60-
| engine-shaped types recognised | 58 |
59+
| tracked non-test sources scanned | 557 |
60+
| engine-shaped types recognised | 59 |
6161
| declared objects in the registry | 298 |
62-
| same-named calls subtracted as non-engine | 134 |
62+
| same-named calls subtracted as non-engine | 137 |
6363

6464
## Every site
6565

@@ -160,6 +160,7 @@ Measured on 2026-09-05 at `63a1a410e`.
160160
| `packages/services/service-automation/src/builtin/crud-nodes.ts` | `insert` | `objectName` | undecidable | context, elevation undecidable | 1 |
161161
| `packages/services/service-automation/src/builtin/crud-nodes.ts` | `update` | `objectName` | undecidable | context, elevation undecidable | 1 |
162162
| `packages/services/service-automation/src/flow-dispatch-store.ts` | `insert` | `sys_flow_dispatch` | enabled | elevated | 1 |
163+
| `packages/services/service-automation/src/flow-dispatch-store.ts` | `update` | `sys_flow_dispatch` | enabled | elevated | 1 |
163164
| `packages/services/service-automation/src/suspended-run-store.ts` | `delete` | `sys_automation_run` | enabled | elevated | 3 |
164165
| `packages/services/service-automation/src/suspended-run-store.ts` | `insert` | `sys_automation_run` | enabled | elevated | 2 |
165166
| `packages/services/service-automation/src/suspended-run-store.ts` | `update` | `sys_automation_run` | enabled | elevated | 2 |

0 commit comments

Comments
 (0)