Repository navigation
Commit b280546
Fixes #20805
Clause-②: yes (narrowing)
A caller-supplied value for a `formula` field is now stripped by the
engine on every write path, in every context, and reported through
`droppedFields` / `onFieldsDropped` under a new `reason` arm,
`computed`. `ObjectQL.validate` now runs the write's own field doors, so
a dry run built on it predicts what the write does. The REST half (the
import runner's row report) stays on #20701; #20701 remains open.
## Reproduction first (base `5bed1f6caf`, real ObjectQL +
`ObjectStackProtocolImplementation` + `RestServer` routes, SQLite
`:memory:` and the in-memory driver)
A throwaway probe (never committed) wrote to an object with `n: number`,
`doubled: formula (record.n * 2)`, `ro: readonly text`, `task_count:
summary`, as a non-system caller unless noted. The raw row was read from
the driver, below the engine's formula hydration.
| door | SQLite, before | memory, before | both drivers, after
(`b21d13b05b`) |
|:--|:--|:--|:--|
| `protocol.createData` with `doubled: 5` | throws `SqliteError`
`SQLITE_ERROR`, no status, no field | ok, `droppedFields` null, raw row
stores `doubled: 5` | ok, `droppedFields: [{ fields: ['doubled'],
reason: 'computed' }]`, raw row has no `doubled` |
| `POST /api/v1/data/:object` with `doubled: 5` | `400 INVALID_FIELD`
"Unknown field 'doubled'" (REST's driver-string branch, for a declared
field) | `201`, raw row stores `doubled: 5` | `201`, the same
`droppedFields`, header `doubled;reason=computed` |
| `engine.update` by id with `doubled: 99` | throws `SqliteError` | ok,
no drop, raw row stores `doubled: 99` | ok, one `computed` event, raw
row has no `doubled`, read returns `doubled: 8` from `n: 4` |
| `PATCH /api/v1/data/:object/:id` with `doubled: 7` | `400
INVALID_FIELD` | `200`, raw row stores `doubled: 7` | `200`, the same
`droppedFields` |
| `engine.insert`, `isSystem`, `doubled: 5` | throws `SqliteError` | ok,
raw row stores `doubled: 5` | ok, one `computed` event, raw row has no
`doubled` |
| full read of a row, written back (`GET` then `PUT`) | throws
`SqliteError` | ok | ok, `computed` for `doubled` beside the existing
`readonly` drops |
| `engine.validate` with `doubled: 5` | `valid: true` | `valid: true` |
`valid: true`, and the listener receives the `computed` event |
| `engine.validate` with `nope: 5` | `valid: true` while `insert`
refused it | same | throws `INVALID_FIELD` / 400 / `field: 'nope'`, the
envelope `insert` throws |
| `engine.insert` / `update` with `task_count: 42` / `7` (summary, H4) |
stored 42, then 7, then recomputed to 1 by a child write, no drop | same
| unchanged: summary is persisted by the engine (see Acceptance notes) |
## What changed
- **`packages/objectql/src/engine.ts`**
- `stripComputedWriteFields` — a new module function beside
`undeclaredWriteFieldErrors`: takes every own key naming a declared
`formula` field out of the payload (copy, never mutating the caller's
object) and returns the union of what it took. Keyed on `type ===
'formula'` literally, the test `fieldHasColumn` and the read projection
use.
- `insert()` — runs it right after the declared-field door, over the
rows that door did not refuse, with no `isSystem` gate; `opCtx.data`
carries the stripped rows, so the snapshot, the defaults and the hooks
all see the payload that is stored. The report joins the existing site
(`insertDrops`: one `computed` event, then the existing `readonly`
event), so `strictReadonlyWrites` refuses it by the derived coverage its
contract states.
- `update()` — the same door after the pre-hook declared-field door
(before the read-only hide pass and the hook recording); reported with
`reportDroppedFields(..., 'computed')` at the confluence after the
post-hook door, beside the other strips' reports. `ctx.submitted` still
carries the submission as sent.
- `validate()` — runs, by the same functions and in the write's order:
the declared-field door (throws, as `insert` does), the computed strip,
and the caller-write strips under the write's `isSystem` gate (insert
mode: `stripRuntimeOwnedFields`, then the static-`readonly` strip;
update mode: `stripReadonlyFields`, with a supplied `id` read as the
address). Drops go to a new optional `onFieldsDropped` option. The
docblock's "second named limit" (a readonly reference field) is closed
and rewritten.
- `staticReadonlyCreateStrip` — the create path's static-`readonly`
strip with its re-default, lifted out of `insert()`'s loop unchanged in
behaviour so `validate` calls the same door rather than a copy (one
function per door).
- **`packages/spec/src/data/data-engine.zod.ts`** — the `computed` arm
on `DroppedFieldsEventSchema.reason`, its docblock bullet and describe
string. **`packages/spec/src/contracts/data-engine.ts`** — the
`WriteObservabilityOptions` docblock named "all three arms"; it now
names four (a comment-only edit, outside the claim's one-arm surface,
because the sentence would otherwise be false).
- **Consumers of `reason` (H3)** — the two exhaustive maps tsc
re-checks: `REASON_PHRASE` in
`packages/objectql/src/readonly-strict-errors.ts` (with a remedy clause
spoken only when a `computed` drop is present, so every existing message
is byte-identical; `computed` is deliberately not in
`READONLY_CLASS_REASONS`) and `DROPPED_REASON_LABEL` in
`packages/services/service-automation/src/builtin/crud-nodes.ts`.
Reason-agnostic pass-throughs need no change: REST's
`X-ObjectStack-Dropped-Fields` echo, metadata-protocol's
`mergeDroppedFieldEvents`, the client SDK types. objectui is NOT
MEASURED (not checked out in this container).
- **Docs** — `content/docs/references/**` regenerated by
`check:generated --fix` (the enum and describe string only); the two
hand-written pages that enumerate the arms
(`kernel/contracts/data-engine.mdx`, `protocol/objectql/security.mdx`);
`permissions/system-context.mdx` gains row 21b for the preview's
`isSystem` gate and amends row 22 (strict still refuses a formula value
under `isSystem`), with the census counts regenerated by
`gen:system-context-census`.
- **Changeset** `.changeset/20805-formula-write-strip.md`:
`@objectstack/spec` minor, `@objectstack/objectql` minor with the
BREAKING banner, `@objectstack/service-automation` patch, one ADR-0087
disposition (`not-required (no-migration-prescription)`).
## Zone 2 hypotheses, as measured
- **H1 — falsified in its "one place" half.** The existing strips live
in several places: `insert()` has two strip passes
(`stripRuntimeOwnedFields`, and the static strip over
`staticReadonlyInsertSubject`) behind one `isSystem` gate, reporting at
one site (`insertDropped`, reason `readonly`); `update()` has per-branch
passes (primary key, `readonlyWhen`, static) on by-id and multi,
reporting through the `reportDroppedFields` closure. So there are two
report sites. The new strip's REPORT joins both; its STRIP sits at the
declared-field door instead, because those passes are post-hook and
`isSystem`-gated, while this one must run for system writers, must not
hand hooks a value that will not be stored, and must be callable by
`validate`, which runs no hooks. The function is
`stripComputedWriteFields`.
- **H2 — held**: `undeclaredWriteFieldErrors` judges undeclared keys
only and `validate` called no door. Served as one function per door
(`undeclaredWriteFieldErrors`, `stripComputedWriteFields`,
`stripRuntimeOwnedFields`, `staticReadonlyCreateStrip`,
`stripReadonlyFields`), each called by the write and by `validate`, not
one combined judge.
- **H3** — listed above; both exhaustive maps updated, and tsc reddens
when one is not (reverse check below).
- **H4 — summary is persisted by the engine**, measured on both drivers
(table above), so it takes no arm and no pin here.
## Tests
- New `packages/objectql/src/engine-formula-write-strip.test.ts` (19
cases, recording driver): insert single / batch / `insertMany` (a
refused row counts toward nothing) / `isSystem`; update by id / multi /
`isSystem`; a formula also declared `readonly` reported once as
`computed`; controls (static `readonly` still `readonly`, writable
fields land); the hooks' view and `ctx.submitted`; strict refusal on
both verbs; `validate` in both modes, under `isSystem`, the unknown-key
envelope equal on `validate` and `insert`, the caller-write strips'
report, and the update-mode `id` address rule on an object whose `id` is
declared readonly.
-
`packages/runtime/src/sandbox/undeclared-field-write-driver-split.integration.test.ts`
gains a `#20805` block, one assertion run on both families (SQLite on a
real table, in-memory): insert and update succeed, report `computed`,
store no such key, `isSystem` included; `validate` reports the same and
refuses an unknown key as `insert` does; controls. The memory arm lives
in this file because it is a ruled `driver-memory` consumer, and the
census counts declarations, so no ledger row moves. PostgreSQL is not a
cell here: the one live-PostgreSQL job for `@objectstack/runtime`
(Temporal Conformance, step "Run the runtime cascade-delete matrix
against live PostgreSQL") runs only
`cascade-delete-multivalue-lookup-real-driver`, and this file is not in
it (sentence corrected by the seat after review 5909189785).
- Runs (all `VERDICT command-exit 0` under the shared lock): objectql
`local` 344 files / 6769 tests at `55c81da438` (objectql unchanged to
`b21d13b05b`); spec `local` 578 files / 17065 tests and
service-automation 157 files / 1974 tests at `06e16e23f0` (both
unchanged to `b21d13b05b`); consumer suites at `c8b823cf16` —
metadata-protocol `validate-data` + dropped-fields 4 files / 33, REST
every `import-*` file + `rest-dropped-fields` + `rest-batch-endpoint` 22
files / 529, verify `handle.test.ts` 24; the runtime file 28 / 28 at
`b21d13b05b`; typecheck (tsc + test layer) green for spec, objectql,
service-automation and runtime.
- **Reverse verification** (fix committed first;
`scripts/ablation-replace.mjs` in wrap mode, restore trap armed):
`stripComputedWriteFields` made to return before stripping (anchor 1 to
0, blob `2719ac6bc184` to `15ace97894c2`), objectql rebuilt,
`ablation-dist-preflight` found the marker in 4 built files. Result:
objectql pins 17 of 19 red (the two strip-independent cases — unknown
key, nothing to strip — stayed green), runtime 4 of 28 red (both
families' write and `validate` cells; the controls and every
pre-existing case green). Restore: blob equal to HEAD, `git diff HEAD`
empty, rebuilt, marker absent from all 14 built files, whole tree clean,
pins 19 / 19 and 28 / 28.
- **Type reverse check**: deleting `computed` from
`DROPPED_REASON_LABEL` made `service-automation` tsc exit 2 with
`TS2741: Property 'computed' is missing` — it reads the rebuilt spec
declarations; restored byte-identical.
- **Gates** at `b21d13b05b`: `node scripts/pm/dispatch-gates.mjs
--commands` derived 116; all 116 run, each exit captured before any
pipe, all exit 0; `--ran` reconciliation: 116 derived, 116 run, 0
NOT-MEASURED, 0 UNRUN. Three that first answered `PREREQUISITE NOT MET`
(`check:skill-examples`, `check:dual-build-cjs-loads`,
`check:type-check-debt`) were re-run green after `turbo run build
--filter='./packages/*' --filter='./packages/*/*'`;
`check:system-context-census` first reddened on the new `isSystem` read
and is green with row 21b.
- **Lint, narrowed**: `eslint --no-inline-config --format json` over the
7 TS files this diff touches reports 7 files, 0 errors, 0 warnings;
`--print-config` shows no `parserOptions.project` / `projectService` on
any of them (the config enables no type-aware linting), so the diff
cannot move a verdict on an untouched file. The repo-wide `pnpm lint` is
CI's.
## Acceptance notes
- **`summary` (H4)**: a caller-supplied roll-up value is stored as sent
and overwritten by the next child write, on both drivers, with no drop.
That is the contract `RUNTIME_OWNED_FIELD_TYPES` and its docblock in
`rule-validator.ts` record on purpose; not changed here.
- **FLS runs before the strip**: plugin-security's field-write gate
judges `opCtx.data` before the engine body, so a permission set that
marks a `formula` field not editable would 403 a round trip before the
strip runs — the same order a `readonly` field already has. Read, not
measured.
- **A hook-written formula key**: the door strips the caller's value
before the hooks; a `before*` hook that writes a formula key itself
passes the post-hook declared-field door (the field is declared) and
still reaches the driver. Read, not measured; no producer found.
- **Neighbouring describe strings**: the `droppedFields` describe
strings in `api/batch.zod.ts` / `api/protocol.zod.ts` name only the
read-only strips (already incomplete since `primary_key`); left as they
are, outside this surface.
- REST's driver-string branch still labels a missing column "Unknown
field" for schema drift; the import runner's use of the new listener and
its row report are #20701's.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent fe463b4 commit b280546
14 files changed
Lines changed: 983 additions & 109 deletions
File tree
- .changeset
- content/docs
- kernel/contracts
- permissions
- protocol/objectql
- references
- api
- data
- packages
- objectql/src
- runtime/src/sandbox
- services/service-automation/src/builtin
- spec/src
- contracts
- data
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
315 | 315 | | |
316 | 316 | | |
317 | 317 | | |
| 318 | + | |
318 | 319 | | |
319 | 320 | | |
320 | 321 | | |
| |||
332 | 333 | | |
333 | 334 | | |
334 | 335 | | |
335 | | - | |
| 336 | + | |
336 | 337 | | |
337 | 338 | | |
338 | 339 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
123 | 123 | | |
124 | 124 | | |
125 | 125 | | |
126 | | - | |
| 126 | + | |
| 127 | + | |
127 | 128 | | |
128 | 129 | | |
129 | 130 | | |
| |||
137 | 138 | | |
138 | 139 | | |
139 | 140 | | |
140 | | - | |
| 141 | + | |
141 | 142 | | |
142 | 143 | | |
143 | 144 | | |
| |||
280 | 281 | | |
281 | 282 | | |
282 | 283 | | |
283 | | - | |
| 284 | + | |
284 | 285 | | |
285 | 286 | | |
286 | 287 | | |
| |||
354 | 355 | | |
355 | 356 | | |
356 | 357 | | |
357 | | - | |
| 358 | + | |
358 | 359 | | |
359 | 360 | | |
360 | 361 | | |
361 | | - | |
362 | | - | |
| 362 | + | |
| 363 | + | |
363 | 364 | | |
364 | 365 | | |
365 | 366 | | |
366 | | - | |
| 367 | + | |
367 | 368 | | |
368 | 369 | | |
369 | 370 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
366 | 366 | | |
367 | 367 | | |
368 | 368 | | |
369 | | - | |
370 | | - | |
371 | | - | |
372 | | - | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
373 | 375 | | |
374 | 376 | | |
375 | 377 | | |
| |||
0 commit comments