Skip to content

Commit 96b0e31

Browse files
fix(service-automation): the flow get_record node refuses a filter that evaluates the stored-metadata family, with the data door's refusal (#21623) (#21641)
Fixes #21623 Clause-②: no ## What this changes A flow's `get_record` node no longer evaluates the stored-metadata family's body or content hash. PR #21621 (#21519) closed the node's serve and copy exits: a family read is served with the body projected and the hash keyed. It did not close the evaluate exit. The node still ran its `filter` against the stored values as written, so whether a row came back answered a predicate over the body column or a content-hash column. That is the predicate-oracle shape the family's refusals name. The fix is one bounded call at the node, built only from the data door's own functions (triage ruling `5972899653`): - `crud-nodes.ts` gains `storedMetadataFilterRefusal(nodeType, objectName, query)`. For a family object (`isStoredMetadataBodyObject`), it collects the columns the filter reads with the family's one collector, `collectStoredMetadataFilterFields`. It then asks the door's two evaluate refusals in the door's order: `storedMetadataBodyPredicateRefusal` first, then `storedMetadataHashEvaluateRefusal`. A refusal comes back as a guard refusal (`refuseNode`, the node's existing channel) that carries the door's own error code, read off the door's refusal rather than spelled again. The code is `INVALID_FIELD`, and no code is minted. - The `get_record` executor calls it after the filter is interpolated and the erased-condition guard has run. It runs before the data engine is looked up, so it runs before either engine read (`findOne`, and `find` when `limit` is above 1). - **Query shape.** The collector is handed `{ where: filter }`, which is the interpolated filter in the slot both engine reads pass it in. The collector reads `where` and the `filter` alias the engine accepts, so the node's key is covered under either spelling. - **Option shape.** The door passes `{ filterFields, sortFields }` to the body refusal and `{ groupBy, filterFields, sortFields }` to the hash refusal. The node's config declares no sort and no grouping (strict `GetRecordConfigSchema`: `objectName`, `filter`, `fields`, `limit`, `outputVariable`), so both are fed `{ filterFields }` only. - Nothing is copied: there is no `metadata-protocol` edit, no `packages/spec` edit and no write-node edit. There is no new dependency edge either: the three functions come from the `@objectstack/metadata-protocol` dependency that PR #21621 added. ## What a refused `get_record` does to the run (measured) - The node fails as a guard (`errorClass: 'guard'`), so a `fault` edge does not route it. - The run answers `success: false`, `status: 'failed'`, with no declared output. No node downstream of the refused node runs, and the family engine read never runs. - The run result itself carries no `code`, since `AutomationResult.code` belongs to the trigger and resume refusals. The step log's failure code is the engine's `NODE_FAILURE`, as for every failed node. - The door's code reaches the flow on `{$error.code}`, which reads `INVALID_FIELD`. A `try_catch` catch region reads it there and on its own error variable. ## Reproduction on `main` before the fix (by class) The composition is a kernel with `ObjectQLPlugin`, the real `AutomationServicePlugin` and `driver-sql` on better-sqlite3 `:memory:`. One family row was written with a synthetic credential in a withheld slot and its canonical content hash. The matrix covered both run identities, both node branches and both family tables. In each cell, a filter over the body column, or over the hash column, was run once with a value that matches the stored row and once with one that does not. 16 of 16 cells answered the matching filter with the row and the non-matching one with none. A body condition that a flow variable supplies (`{ $and: '{record.conds}' }`) gave the same reading under both identities (2 of 2). The data door refused both filter shapes with `INVALID_FIELD` / 400. After the fix, every cell is refused and the family engine read count is 0. No stored value is recorded here. ## Pins `get-record-stored-metadata-filter-refusal.integration.test.ts` has 16 cases on the same composition: - **Control:** the data door refuses each filter shape on both family tables with `INVALID_FIELD` / 400. - **The matrix (8 cases):** `runAs: 'system'` and `runAs: 'user'`, crossed with the `findOne` and `find` branches, crossed with a body-column and a hash-column filter. Each case runs a matching filter and a non-matching one. Both must give a failed run, no output, no family engine read and no downstream write. The code a flow reads on `{$error.code}` (and on the `try_catch` error variable) must equal the door's code for the same filter. - **The history table:** its parent-hash column, its hash column, its change-note column (which can quote a hash), and a cross-field `{ $field }` comparand that reads the body. Each is refused with the door's code. - **Guard routing:** a refused read with a `fault` edge fails the run, and the handler never runs. - **Interpolation (both identities):** a filter whose body condition list arrives through a flow variable is refused, judged after interpolation. So is a body condition whose value is a flow variable. - **A scalar-column filter on a family read (both identities):** it is served projected, with the door's keyed hash, as PR #21621 serves it. - **A non-family read is unchanged:** on an ordinary object whose columns share the family's column names, the same filter shapes run and serve the row as stored. ## Ablation The fix was committed first. The ablation was run at `b92c808613` and again at `d06a84e67a`, with identical readings. The direction was predicted before the run: with the refusal's effect removed, the 12 refused-shape cases go red, because the engine read runs and answers, and the 4 controls stay green (door control, two scalar-filter cases, non-family). - **Mutation:** `scripts/ablation-replace.mjs` replaced the executor's `if (familyRefusal) return familyRefusal;` with a no-op carrying a marker. The anchor count went from 1 to 0, the marker count from 0 to 1, and the file's blob changed. The subject is reached through a relative `src` import, so no `dist` rebuild or preflight applies. - **Result:** Tests 12 failed and 4 passed (16), matching the prediction. On every red case, the first assertion to fail was "the run must fail". - **Restore:** the tool reported the blob after restore equal to the HEAD blob, and `git diff HEAD` empty. A bash trap (`git checkout HEAD --` on the absolute path) re-confirmed the HEAD blob, with 0 diff lines. `git status --porcelain` showed 0 lines, the marker grep 0 and the anchor grep 1. ## Verification The final readings are at HEAD `d06a84e67a`, which merged `origin/main` at `5b5e83f446` and then rebuilt the tree (`turbo run build`, 72 of 72 tasks). Every heavy run went through `scripts/pm/os-verify-lock.sh`. - `pnpm --filter @objectstack/service-automation exec vitest run --maxWorkers=2` (the whole suite, the new pins included): Test Files 168 passed (168), Tests 2078 passed (2078), VERDICT command-exit 0. - `pnpm --filter @objectstack/service-automation typecheck`: VERDICT command-exit 0, and `check:test-typecheck` is OK. `tsc --listFiles` shows the new pin file in both `tsconfig.json` and `tsconfig.test.json`. - The `metadata-protocol` family door tests, run as read-only controls (`protocol.data-door-stored-content-hash`, `protocol.data-door-stored-metadata-filter-reads`, `protocol.data-door-stored-metadata-redaction`, `protocol.served-content-hash`, `stored-metadata-body-family.pin`): Test Files 5 passed (5), Tests 122 passed (122). - Gates: running `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands` (no paths) at `d06a84e67a` derives 64 commands (30 pnpm, 34 direct node). All 64 were run there, and each exited 0. `--ran` reconciles them as 64 derived, 64 run, 0 NOT-MEASURED and 0 UNRUN (exit 0). An earlier pass at `b92c808613`, before the merge, derived the same 64. There, `check:dual-build-cjs-loads` exited 3 (PREREQUISITE NOT MET, an unbuilt tree) and exited 0 once the tree was built. Six workflow-valued families print as NOT MEASURED in the derivation, so they belong to CI: the three shard attestations, the issue-citation census and the two test-completeness checks. The same holds for the CI-shell jobs this path set schedules: Test Core, Temporal Conformance, the Dogfood Regression Gate, Dogfood Verify CLI and Build Core. `check:nul-bytes` exited 0, and a control-byte scan of the 3 changed files found none. No turbo-driven gate left an `AGENTS.md` block, and the tree was clean after each pass. - Lint, as a proven narrowing (`pnpm lint` itself is CI's). The population, read from eslint's own config, is 2 of 3 changed paths: `crud-nodes.ts` and the pin file. For the changeset, eslint answers "no matching configuration". The `--format json` count is 0 errors and 0 warnings on those 2. Invariance: `eslint.config.mjs` enables no type-aware linting (no `parserOptions.project`, no typed rules), so this diff cannot move the verdict on an untouched file. ## Acceptance notes - **Not wired into the write nodes.** `storedMetadataFilterRefusal` takes the node type, so it can serve the write nodes' filters too. That would matter if #21624's ruling were overturned: #21624 refuses family targets on `create_record` / `update_record` / `delete_record` outright, which also closes their filter exit. It is not wired there, and #21624 remains open. - **A `try_catch` region still catches this refusal.** A `try_catch` region catches a failing region whatever its class, which is pre-existing behaviour for every guard refusal. The read never ran, so the answer is the same refusal whatever the stored value is, and the oracle stays closed. - **A code comment this makes false (not edited).** The `NodeExecutionResult.code` doc comment in `packages/services/service-automation/src/engine.ts` says `create_record` is the only executor that sets `code` today. `get_record` now sets it for this refusal. The file is outside this claim's surface, so the comment is not edited here. Carrier: none. - **An unreleased changeset sentence this makes false (not edited).** `.changeset/21519-flow-read-node-family-serve.md` is unreleased and says the node's `filter` behaves as before. For a family read, it no longer does. This PR does not edit a changeset it did not add. Both changesets compile into the same release, and this PR's own changeset states the change. Carrier: none. - **A docs list that is now incomplete (not edited).** In `content/docs/automation/flows.mdx`, the guard-refusal callout's "In practice that is …" list does not name this refusal. That leaves the list incomplete, not false, so it is not edited. - **Docs grep.** Nothing in `content/docs/**` (outside `releases/`) or `skills/**` states how the `get_record` filter treats stored metadata, so no sentence there is made false. --- _Generated by [Claude Code](https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 1e4ae08 commit 96b0e31

3 files changed

Lines changed: 453 additions & 0 deletions

File tree

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
'@objectstack/service-automation': patch
3+
---
4+
5+
fix(service-automation): a flow's `get_record` node refuses a filter that evaluates the stored-metadata tables' body or content hash, as the generic data door does (#21623)
6+
7+
Clause-②: no
8+
9+
The two stored-metadata tables (the current metadata bodies and their version history) hold each body as stored, credential material included, and content-hash columns computed over it. A flow's `get_record` node now serves those rows projected and keyed, but it still ran its `filter` against the stored values as written, under either run identity (`runAs: 'system'` and `runAs: 'user'`). A filter over the body column or a content-hash column was evaluated row by row, so whether a row came back answered the filter: a predicate over the withheld values. The generic data door refuses those filters before its query runs.
10+
11+
**What changes.** When the node reads either table, it judges its filter the way the data door judges the same filter, before the data engine is asked, on both branches (one row, and a row list when `limit` is above 1). The columns the filter reads are collected after interpolation, so a condition that a `{token}` supplies is judged too. A filter that reads the body column, or a content-hash column (the history table's parent hash and change note included), refuses the node with the data door's own message and error code, `INVALID_FIELD`. The refusal is a guard failure: the run fails, nothing downstream of the node runs, and a `fault` edge does not route it. A `try_catch` catch region reads the code on `{$error.code}`. To read a stored-metadata row from a flow, filter by `name`, `type`, `state` or another scalar column.
12+
13+
**What does not change.** A filter over scalar columns is served as before: the body projected and the hash keyed. Every other object is filtered and read exactly as before, including columns that share these names. The write nodes are unchanged. The node consumes the data door's own functions from `@objectstack/metadata-protocol` (`collectStoredMetadataFilterFields`, `storedMetadataBodyPredicateRefusal`, `storedMetadataHashEvaluateRefusal`) and keeps no copy of them.

‎packages/services/service-automation/src/builtin/crud-nodes.ts‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,13 @@ import type { DroppedFieldsEvent } from '@objectstack/spec/data';
2020
import { StandardErrorCode } from '@objectstack/spec/api';
2121
import { isStoredMetadataBodyObject } from '@objectstack/spec/kernel';
2222
import {
23+
collectStoredMetadataFilterFields,
2324
ephemeralStoredHashDigest,
2425
redactStoredMetadataRows,
2526
serveStoredMetadataHashColumnRows,
27+
storedMetadataBodyPredicateRefusal,
2628
storedMetadataBodyProjection,
29+
storedMetadataHashEvaluateRefusal,
2730
type StoredHashDigest,
2831
} from '@objectstack/metadata-protocol';
2932
import type { AutomationEngine } from '../engine.js';
@@ -270,6 +273,56 @@ async function serveFamilyRead<A>(
270273
return answer;
271274
}
272275

276+
/**
277+
* [#21623] Refuse a node whose filter EVALUATES the stored-metadata family's
278+
* body or content hash, the way the generic data door refuses the same filter.
279+
*
280+
* {@link serveFamilyRead} closes the node's serve and copy exits; this closes
281+
* its evaluate exit. A filter over the stored body column, or over a stored
282+
* content-hash column, is evaluated against the stored values row by row, so
283+
* whether a row comes back answers the predicate even though the row itself
284+
* is served projected: a guessed prefix of withheld credential material, or a
285+
* guessed hash, returns the row exactly when it is right (the predicate oracle
286+
* the family's refusals name). Under `runAs: 'system'` the engine reads
287+
* elevated and cannot tell this read from the platform's own internal
288+
* readers, so the rule is applied here, before the engine is asked.
289+
*
290+
* Built only from the door's own functions (`@objectstack/metadata-protocol`),
291+
* in the door's own order, never a copy:
292+
* - the columns the filter reads come from the family's ONE filter-field
293+
* collector (`collectStoredMetadataFilterFields`): every key's head and
294+
* every cross-field `{ $field }` comparand, at any depth;
295+
* - the body refusal (`storedMetadataBodyPredicateRefusal`) is asked first,
296+
* then the content-hash refusal (`storedMetadataHashEvaluateRefusal`).
297+
*
298+
* `query` is the option bag the node hands the engine, so the collector reads
299+
* exactly the filter the engine would run: the INTERPOLATED one, since a
300+
* `{token}` can resolve to a whole condition (a `$and` list, a comparand) whose
301+
* columns the authored template does not show. The node configs declare no
302+
* sort and no grouping, so the refusals are fed filter fields only.
303+
*
304+
* The answer is a guard refusal ({@link refuseNode}: the metadata is wrong,
305+
* and re-running it unchanged never succeeds) carrying the door's own error
306+
* code, read off the door's refusal rather than spelled again. `undefined`
307+
* outside the family ({@link isStoredMetadataBodyObject}) and for a filter that
308+
* reads neither column.
309+
*/
310+
function storedMetadataFilterRefusal(
311+
nodeType: string,
312+
objectName: string,
313+
query: { where: Record<string, unknown> },
314+
): (ReturnType<typeof refuseNode> & { code: string }) | undefined {
315+
if (!isStoredMetadataBodyObject(objectName)) return undefined;
316+
const filterFields = collectStoredMetadataFilterFields(objectName, query);
317+
const refuse = (refusal: Error) =>
318+
({ ...refuseNode(`${nodeType}: ${refusal.message}`), code: (refusal as Error & { code: string }).code });
319+
const bodyPredicateRefusal = storedMetadataBodyPredicateRefusal(objectName, { filterFields });
320+
if (bodyPredicateRefusal) return refuse(bodyPredicateRefusal);
321+
const hashEvaluateRefusal = storedMetadataHashEvaluateRefusal(objectName, { filterFields });
322+
if (hashEvaluateRefusal) return refuse(hashEvaluateRefusal);
323+
return undefined;
324+
}
325+
273326
/**
274327
* CRUD built-in nodes — `get_record` / `create_record` / `update_record` /
275328
* `delete_record`, wired to the runtime data layer (ObjectQL / IDataEngine).
@@ -353,6 +406,14 @@ export function registerCrudNodes(engine: AutomationEngine, ctx: PluginContext):
353406
const limit = cfg.limit;
354407
const outputVariable = cfg.outputVariable;
355408

409+
// [#21623] A filter that evaluates the stored-metadata family's
410+
// body or content hash is refused before the engine is asked,
411+
// with the data door's own code, under either run identity. It
412+
// reads the interpolated filter, in the `where` slot both
413+
// engine reads below hand it in.
414+
const familyRefusal = storedMetadataFilterRefusal('get_record', objectName, { where: filter });
415+
if (familyRefusal) return familyRefusal;
416+
356417
const data = getData();
357418
if (!data) {
358419
ctx.logger.warn(`[get_record] no data engine; skipping ${objectName}`);

0 commit comments

Comments
 (0)