Repository navigation
fix(metadata-protocol)!: one stored-metadata filter collector and search narrowing, owned by the door and called by the reader seam; cross-field and deep family reads refused - #21619
Conversation
…narrowing and filter-field collector; the reader seam calls both (WIP) Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…ng at the door, and door/seam parity (WIP) Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…l, patch runtime) Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…filter Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…or-narrowing-export Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…or-narrowing-export Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…indData never reads one) Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…aches Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 2 package(s): 22 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: ⛔ 9 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails. What this run could not see
Coarse fallback — 31 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin de2ffce13c017f6359e65c7c149f8860f2de0b4e && git checkout de2ffce13c017f6359e65c7c149f8860f2de0b4e
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 5c9138b4b672ade8cbc99d3e48e0479a9ebabf74 0ac5749662d7dddffd9dc011e0e71375d8d79451 && git checkout -B drift-repro 5c9138b4b672ade8cbc99d3e48e0479a9ebabf74 && git merge --no-ff 0ac5749662d7dddffd9dc011e0e71375d8d79451
node scripts/docs-audit/affected-docs.mjs --json 5c9138b4b672ade8cbc99d3e48e0479a9ebabf74
|
Contract reviewServed-tier: Inputs read: card #21544 (its body and all 5 comments), PR #21619 (body, file list, net diff against ① Derived judgmentsPublic surface,
Door accept set,
Non-family objects — NO ANSWER MOVED, RIGHT.
Runtime reader seam (
Seam
Pins the ruling owes — PRESENT.
Check-runs on the head — not mine to wait on; the landing waits for them.
② Semver level
③ Boundary flags
Implemented-by: VERDICT: PASS |
Fixes #21544
Clause-②: yes (narrowing)
The generic data door now owns the stored-metadata family's one filter-field collector and its one default-search narrowing. Both are exported module functions, and the door and the in-process reader-context seam (
@objectstack/runtime) call the same two. The card's measure-first step found a door gap: two filter shapes read a family column at the generic data door without the family's refusal ever seeing them. Per the ruling, the door therefore adopts the stricter collector in this landing, and under the card's raise rule the card is p1. Measurement below.The door-gap measurement (measured before any collector was chosen)
Measured at
b610eabf72with a composed kernel (ObjectQL, a default datasource, HTTP server, platform objects, auth, security, sharing, REST and the dispatcher) and the administrator signed in. The family credential was stored by the production writer (PUT /meta/datasource/NAME). Every request went through three doors:POST /api/v1/data/OBJECT/query(HTTP), the in-process door (protocol.findData) and the seam (serveStoredMetadataReadsThroughover the engine). Both family tables were covered, ondriver-sqlite-wasmand ondriver-memory.POST /api/v1/data/sys_metadata/query){"where":{"metadata":{"$ne":"zzz"}}}metadataINVALID_FIELD(family refusal){"where":{"type":"datasource","name":{"$ne":{"$field":"metadata"}}}}metadata$eqtwin answers 0 rows;type$lt$field metadataanswers every row and$gtanswers none (SQL). The family column is evaluated.checksum(andprevious_checksum/change_noteonsys_metadata_history){"groupBy":["type"],"aggregations":[{"function":"count","alias":"n","filter":{"name":{"$ne":{"$field":"metadata"}}}}]}metadatan= 1 (the$eqtwin givesn= 0){"metadata":{"$contains":"STORED_CREDENTIAL"}}wrapped in 33 nested one-armed$andlevelsmetadatawhereand in an aggregation filter. At 32 levels the same filter is refused (control).{"where":{"metadata.x":"zzz"}}INVALID_FIELDfrom the door's dotted-path verdict (param: where). Moot: never evaluated.havinghavingnamingmetadata/checksum, or{ "$field": … }to oneINVALID_FILTER: the engine judges everyhavingname against the aggregated row's columns. Grouping by a family column is refused, andmin/maxof thetextarea/textfamily columns is refused by the engine's aggregate-field-type door. Moot.The two bold shapes are the door gap. The cross-field one is a SQL-driver reach. On
driver-memorythe reference is not resolved at all (see Acceptance notes). The depth one is a reach on both drivers.What this changes
Public surface:
@objectstack/metadata-protocol(additive,minor)collectStoredMetadataFilterFields(object, query): string[]is the family's one filter-field collector. It returns every column a read query's filters read, acrosswhere, the engine'sfilteralias and eachaggregations[i].filter. That means each key's head plus each cross-field{ $field }comparand's head, at any depth: the walk is iterative and cycle-safe, with no depth backstop. Anything beneath an unrecognised$key is read as a condition, and aFilterArrayis lowered first. It is[]outside the family. It does not readhaving, whose names are the aggregated row's.narrowStoredMetadataSearch(object, query, schema, wireSpelling?): string[] | undefinedis the family's one default-search narrowing. It moved, unchanged in its answers, from the door's private method of the same name:searchFields;undefinedmeans "run as is".type StoredMetadataSearchSchemais the definition slice the narrowing reads.Accept-set changes: the generic data door,
sys_metadata/sys_metadata_historyonlyThis applies to
GET /api/v1/data/:object,POST /api/v1/data/:object/queryand in-processfindData.checksum,previous_checksumorchange_notechanges from 200 to 400INVALID_FIELD, withparam: filterandfieldset to the column, before the engine is asked. This coverswhere,$notand an aggregation filter.INVALID_FIELD, in the same envelope.$key wrapping a family column, an array-form aggregation filter naming one, and a malformed$fieldreference to one were the engine'sINVALID_FILTER. They now get the family'sINVALID_FIELD. Over HTTP, the array-form aggregation filter is still refused earlier by REST validation.Unchanged:
[]outside the family, so the door collects nothing there.@objectstack/runtime(patch)The seam (
stored-metadata-reader-seam.ts) changes as follows:narrowFamilySearchis deleted, along with thefilterHeadFieldswrapper and the@objectstack/plugin-securitycollectConditionFieldsimport. The seam now calls the door's two exports.countruns the query the guard returns (H2).The seam's accept set is unchanged: everything it refused before it still refuses. Two seam refusals change code, from the engine's
INVALID_FILTERto the family'sINVALID_FIELD: an unrecognised$key wrapping a family column, and a malformed$fieldreference to one.Readings on the dispatch's hypotheses
countdiscards the guard's return), confirmed and corrected.countnow consumes the guard's return. The pin: a default search throughcountarrives narrowed.having), moot. See the table. The collector deliberately does not readhaving: an aggregation alias spelled like a family column is a legitimate count, and a parity control pins it as run.#21207search, hash and note,#21120body.collectConditionFieldsreaders). After this PR its one reader is@objectstack/plugin-security's own FLS predicate guard (collectQueryFields/assertReadableQueryFields). That guard does not judge the family.@objectstack/runtimestill depends on@objectstack/plugin-securitythroughsecurity/resolve-execution-context.ts. Noplugin-securityedit.Tests
packages/metadata-protocol/src/protocol.data-door-stored-metadata-filter-reads.test.ts:code/status/param/field/objectand the engine never asked;packages/runtime/src/stored-metadata-reader-seam.test.ts: the door / seam parity table. It is one table of 24 cases: 7 search cases (explicit list ×4, default ×2, emptied); 14 collector cases (direct, dotted key ×2, cross-field comparand ×3, dotted reference, list reference,$not, unknown$key, depth 33 ×2, aggregation filter ×2); and 3 controls. Each case runs throughfindDataand through the seam, and the two outcomes must be equal. There is also a narrowed-default pin and thecountpin.pnpm --filter @objectstack/metadata-protocol exec vitest run: 208 files passed, 3 skipped; 3266 tests passed, 19 skipped (at5513190ca5, after mergingmain).pnpm --filter @objectstack/runtime exec vitest run --project local: 318 files passed; 4514 passed, 19 skipped (at5513190ca5).--project repo: 3 files, 751 passed (at9be38c5987).0ac5749662:typecheck: both packages green (at5513190ca5).--listFilesconfirms both new tests are inside each package's tsc program.Reverse verification (both at
be99ceee6a, throughscripts/ablation-replace.mjs, mutation and restore proven on disk)27efc3412999; on disk,narrowFamilySearch= 1,collectConditionFields= 3 and the new collector = 0. Result: 2 red (the parity case for an unrecognised$key wrapping a body filter, and thecountpin) and 41 green. Restored withgit checkout HEAD -- PATH: blob == HEAD51fe56bb5e73,git diff HEADempty.#21207/#21120door suites). Restored: blob == HEADfa64d2b26cd9,git diff HEADempty.Both are src-resolved: each subject is imported by relative path, so no
dist/leg was needed.Gates (at
0ac5749662)node scripts/pm/dispatch-gates.mjs --commandsderived 64 families; all 64 were run and exited 0. Reconciled with--ran: 64 run, 0 NOT-MEASURED, 0 UNRUN.check:dual-build-cjs-loadsprintedPREREQUISITE NOT METbefore a fullturbo build, then exited 0.check:engine-double-contractflagged the new door doubles'findOne.findDatanever readsfindOne, so the doubles carry none, and the pinned ledger is untouched.--no-inline-config --format jsonon the 6 touched TS files at0ac5749662: 6 files, 0 errors, 0 warnings. The checked population is read fromeslint.config.mjs(--print-configper file). Type-aware linting is not enabled anywhere: noparserOptions.projectorprojectService,project=nullper file. So this diff cannot move any untouched file's verdict.File surface
packages/metadata-protocol/src/protocol.ts. The hydration region is untouched; PR fix(metadata-protocol): a hydrated view expansion carries its container's tenant marker, so an unscoped kernel answers an expanded view as env_local does (#21511) #21603 was merged in frommain.packages/metadata-protocol/src/index.tspackages/runtime/src/stored-metadata-reader-seam.ts.changeset/21544-door-narrowing-export.mdpackages/metadata-protocol/src/metadata-redaction.ts. It is thestoredMetadataBodyPredicateRefusaldocblock's parenthetical, which named the old collector as the source offilterFieldsand would have been false after this change. No code.Clause-②: yes (narrowing)and an ADR-0087not-required (no-migration-prescription)disposition marker.check:adr-0087-registrationreads it.Acceptance notes (observations, not filed)
driver-memorydoes not resolve a cross-field reference inwhere. It compares against the literal reference object, so on a non-family object{ "name": { "$eq": { "$field": "name" } } }answered 0 rows (SQL: every row). It was measured atb610eabf72through the generic data door on a memory-backed composition. Public reach is not measured after9a4182a752(the in-memory engine is no longer a boot store), so it is not filed. Carrier: none.count/count_distinctover a family column is still served, at the door and the seam alike. It discloses equality only, which the keyed content hash already serves per row.collectFilterFieldKeyskeeps its 32-level backstop for the existence question. That question is not the family's. The backstop's reach on an unknown field nested below it is an unmeasured inference.Generated by Claude Code