Repository navigation
fix(formula, driver-mongodb): refuse == / != against a list literal at the CEL lowering and $ne arrays at the mongodb face - #19947
Conversation
…he CEL lowering and $ne array at the mongodb face compileCelToFilter lowered `record.f != [...]`, `record.f == [...]` and the same comparisons against a current_user membership array to `$ne: [...]` / a bare-array field spec. The RLS `using` clause is composed after the engine's comparand-shape seam, so on driver-mongodb nothing refused it and the read widened to every scalar row. The lowering now refuses both operators with an array comparand (`unsupported`), so the RLS compiler drops the policy and the sharing seeder skips the rule. driver-mongodb's translateFilter refuses `$ne` with an array on its shape walk with INVALID_FILTER / 400, the envelope driver-sql and driver-memory give. The equality position is left to the shared face, whose ruling pins this translator passing it through. Claude-Session: https://claude.ai/code/session_01Sfe5YjBLwB9J3y8fvm2xq1 Co-authored-by: Claude <noreply@anthropic.com>
Refusing a current_user variable that resolves to an array silenced the authoring lint's reference pass: it compiles every predicate with each kernel-resolved key, `id` included, bound to an array probe, and two of its pins went red. The lowering now refuses a list LITERAL only; the resolved-array half waits on a lint change and is named in the docblock, with where its lowered shapes are refused today. Claude-Session: https://claude.ai/code/session_01Sfe5YjBLwB9J3y8fvm2xq1 Co-authored-by: Claude <noreply@anthropic.com>
…d refusals and their fail-closed consequence Formula pins the lowering refusal for `==` / `!=` against a list literal in every position, at request time and in the authoring shape check, with the neighbours unchanged. driver-mongodb pins `$ne` with an array refused at every depth and at the driver door before the server is asked. plugin-security pins the consequence through the real plugin: a `using` read under such a policy returns zero rows and a `check` write is refused 403 with nothing stored. Plus the changeset. Claude-Session: https://claude.ai/code/session_01Sfe5YjBLwB9J3y8fvm2xq1 Co-authored-by: Claude <noreply@anthropic.com>
…l-mongodb-refusal
📓 Docs Drift CheckThis PR changes 4 package(s): 27 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: ⛔ 5 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails. What this run could not see
Coarse fallback — 137 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 7bd0233762621074e33e3b873feaf9c40b2cc5d3 && git checkout 7bd0233762621074e33e3b873feaf9c40b2cc5d3
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin f5a7250b7a52fa93bce8b31b60abbcda4304b762 f49e075189b193c3ac16d69e7eaaae9afafe8bd6 && git checkout -B drift-repro f5a7250b7a52fa93bce8b31b60abbcda4304b762 && git merge --no-ff f49e075189b193c3ac16d69e7eaaae9afafe8bd6
node scripts/docs-audit/affected-docs.mjs --json f5a7250b7a52fa93bce8b31b60abbcda4304b762
|
…robe kernel keys by runtime type The CEL lowering now refuses `==` / `!=` whose comparand is a list in both forms: a list literal, as before, and a `current_user` variable that resolves to an array (a membership set). The reason is the same `unsupported`; the shape check still sees literals only, because a variable's value exists only per request. The authoring lint's reference pass bound every kernel-resolved key to an array probe, which relied on `==` accepting an array. It now binds each key to a probe of the type its ExecutionContext field declares: a scalar for id / email / organization_id, an array for the membership sets. Claude-Session: https://claude.ai/code/session_01Sfe5YjBLwB9J3y8fvm2xq1 Co-authored-by: Claude <noreply@anthropic.com>
…nd the runtime-typed kernel probes formula: == / != against a current_user variable that resolves to an array is refused (unsupported), naming the variable and withholding its members, while the shape check still passes the source; in / not in against the same set, and scalar keys, lower as before. lint: each kernel-resolved key is probed with its runtime type; a scalar key still reaches the field check under ==, a membership key under in. plugin-security: a using read under != / negated == against a resolved membership set reads zero rows, and a check insert is refused 403 with nothing stored; not in against the same set keeps its meaning. Claude-Session: https://claude.ai/code/session_01Sfe5YjBLwB9J3y8fvm2xq1 Co-authored-by: Claude <noreply@anthropic.com>
…g and register its prescription Clause-② no (narrowing) with the BREAKING banner; formula, driver-mongodb, plugin-security, plugin-sharing and lint graded minor under the launch-window convention for accept-set narrowings, spec patch for the ledger entry. The hand-migration prescription is registered under protocol major 18 as cel-predicate-list-comparand-refused (registry.ts regenerated by gen:migration-registry). Claude-Session: https://claude.ai/code/session_01Sfe5YjBLwB9J3y8fvm2xq1 Co-authored-by: Claude <noreply@anthropic.com>
…l-mongodb-refusal
… lowering refuses first With the lowering refusal, a CEL-authored check carrying a list comparand is dropped at compile (RLS_DENY_FILTER, 403) and never reaches the evaluator's INVALID_FILTER / 400. The rls-predicate-array-comparand-refused entry now claims no envelope for the authored predicate: the explain parenthetical and the write-envelope sentence are cut, and the lowered shapes are past tense. registry.ts regenerated by gen:migration-registry. This PR's changeset says what a CEL-authored check gets relative to the 400 that matchesFilterCondition keeps for a filter passed to it directly. Claude-Session: https://claude.ai/code/session_01Sfe5YjBLwB9J3y8fvm2xq1 Co-authored-by: Claude <noreply@anthropic.com>
…ck pin for the lowering refusal A CEL check comparing against a list is now dropped at compile, so the forbidden insert is refused with the row-level CHECK envelope (PERMISSION_DENIED / 403) before the evaluator's INVALID_FILTER / 400 is reached. The pin keeps its property (the forbidden insert is refused and nothing is stored) and asserts the envelope it gets at head. Docblock spans false at head are cut, in the pin and in matches-filter.ts: the claims that these CEL spellings lower to the refused shapes, that the refusal propagates out of the check seam, and that the empty-field constraint is the evaluator's one throw. No code or error-message change in formula. Claude-Session: https://claude.ai/code/session_01Sfe5YjBLwB9J3y8fvm2xq1 Co-authored-by: Claude <noreply@anthropic.com>
Contract reviewServed-tier: Reviewed and posted 2026-09-24T06:07Z by the at-tier review subagent the ① Derived judgmentsClosure, re-derived. Regression. Shipped carriers: 100 files declare The 2a alignment. The edited 2a entry: every remaining sentence holds at head (nit: its Pins and ablation (re-measured). Lowering refusal removed ( Every shipping sentence. Refusal messages: constant identity + remedy, no field, value or tracker number; ② Semver levelConsistent. ③ Boundary flagsBlocking: the 2a changeset is false at this head, and this diff is what falsifies it — three spans in Non-blocking:
CI at this head: 46 check runs, 35 names after de-duplication on the latest Implemented-by: VERDICT: FAIL |
…l-mongodb-refusal
…es false In the pending 2a changeset, three spans are false once a CEL check with a list comparand is dropped at compile: the write envelope (INVALID_FILTER / 400; the check now gets 403), the explain sentence (explain reports a denial, it does not refuse), and the positive form's move to 400 (it is still 403). Each is cut; no replacement prose. A deliberate correction of an unreleased changeset under the #17712 gate. In cel-predicate-list-comparand-refused, the acceptanceCriteria universal (a policy carrying the shape reads no rows and refuses every write, so one read finds it) is false beside an OR-sibling policy and for a bulk update; it is cut. registry.ts regenerated by gen:migration-registry. Claude-Session: https://claude.ai/code/session_01Sfe5YjBLwB9J3y8fvm2xq1 Co-authored-by: Claude <noreply@anthropic.com>
…alsifies
Measured at the merged head on driver-sql: beside a sibling policy whose
check admits, a policy carrying a list comparand is dropped and the sibling
alone decides, so the forbidden insert is admitted. Two sentences claimed
otherwise and are cut, with no replacement prose: the rest of the 2a
changeset's span-1 sentence ("Both shapes now fail the write"), and the
2a entry's acceptanceCriteria universal, the same sentence class this PR's
own entry lost in the previous commit. registry.ts regenerated.
Claude-Session: https://claude.ai/code/session_01Sfe5YjBLwB9J3y8fvm2xq1
Co-authored-by: Claude <noreply@anthropic.com>
…cy case The RLS compiler drops a policy carrying a list comparand, and the clause fails closed (RLS_DENY_FILTER) only when no other policy applies: beside an OR-sibling, the sibling decides. The qualifier "when no other policy applies" is added after "fails closed" in this PR's changeset and in the cel-predicate-list-comparand-refused entry's reason, as ruled. registry.ts regenerated by gen:migration-registry. Claude-Session: https://claude.ai/code/session_01Sfe5YjBLwB9J3y8fvm2xq1 Co-authored-by: Claude <noreply@anthropic.com>
|
Landing —
|
…$eq instead of binding it (objectstack-ai#19994) Fixes objectstack-ai#19975 Clause-②: no (narrowing) ## What this changes `compileScopedFilterToSql` (`packages/services/service-analytics/src/read-scope-sql.ts`) lowers a row-level read scope into the SQL that the analytics NativeSQL path executes and the `/analytics/sql` echo prints. It already refused a list in the implicit equality slot (`{ f: [...] }`) with `READ_SCOPE_COMPILE_FAILED` / 500. The explicit spelling, `{ f: { $eq: [...] } }`, compiled to an equality with the whole list bound as one parameter, which left the meaning of the predicate to the executing database. A new gate, `assertNoListInEqualitySlot`, refuses that spelling in `compileField`, at any depth under `$and` / `$or` / `$not`, in this module's own envelope. It runs before the member gates, so a list is reported as a list and not by one of its members. This applies ruling 乙 (objectstack-ai#19757, record `5793368540`: 「an array in the implicit-equality slot is refused at the shared face, for every driver at once」) to a compiler that never reaches the shared face. ## Declaration **BREAKING**: this narrows what `compileScopedFilterToSql`, exported from `@objectstack/service-analytics`, accepts. A read scope carrying `{ f: { $eq: [...] } }` compiled before this change and is refused after it. The remedy is `{ f: { $in: [...] } }`. The changeset ships the narrowing as `minor` under the launch-window convention for accept-set narrowings, with a `!` on its headline, the `Clause-②: no (narrowing)` line and an ADR-0087 `not-required (no-migration-prescription)` disposition: no authorable key, spelling or stored shape moves, and an authored policy never emits this spelling. ## Measured first The measurement was recorded on this branch as `c647adb6cc`, before any source change. This is an abstract summary; the tests are the pins. - **Authoring door.** The published RLS policy schema and the RLS authoring lint's decision procedure both admit an equality predicate whose comparand is a list, whether a list literal or a membership variable. - **Lowering.** That predicate lowers to the implicit spelling. The CEL lowering emits `$eq` only around a `{ $field }` reference, so no authored policy produces a list under `$eq`. The tenant layer, the sharing read filter and the controlled-by-parent filter do not emit `$eq` at all. - **This compiler.** The implicit spelling reaches it through the security service's read filter and is refused (500). A list under `$eq` reaches it only from a host-supplied `getReadScope` or from a direct caller of the export, and it compiled. - **Engines.** On the NativeSQL execute path the bound list got four different answers depending on the engine: a driver error, zero rows, rows the scope never named, and every row when negated. Measured on better-sqlite3 and sql.js through the drivers, and on a local PostgreSQL 16 through `driver-sql` and through a plain `pg` pool. MySQL is NOT MEASURED: there is no server in the container. ## Deliberate choices - **500, not the shared face's 400.** The objectstack-ai#5367 ruling, re-affirmed as objectstack-ai#7598 Q2 = A and recorded in this module's header, keeps every refusal of this compiler at `READ_SCOPE_COMPILE_FAILED` / 500 with the message withheld. The scope is a policy the caller cannot author, and a 4xx would echo it back to them. The card's 400 belongs at the policy's authoring door, which is not this file. - **The module's own wording, not a call into the shared face.** `assertListComparandShapes` throws `INVALID_FILTER` / 400. It also judges more than the equality slot: list-operator shapes, null members, null ordering comparands and `$between` bounds. Calling it here would change other refusals of this compiler, and each of those has its own ruling on this door. The new sentence follows this module's bare-array refusal, so both spellings of the one condition read the same way in the operator's log. - **`$ne` with a list is not judged.** Ruling 乙 names equality only. `$ne` falls under ruling A of objectstack-ai#19886 and is handled on that card. ## Compile surfaces (a list in the equality slot) | surface | verdict | |:--|:--| | `compileScopedFilterToSql` (service-analytics read scope) | **changed.** A list under `$eq` is refused. The implicit list was already refused and is now pinned at every depth. | | `assertListComparandShapes` (spec shared face) | **already compliant.** This is ruling 乙's own face (objectstack-ai#19882, landed). Measured: `INVALID_FILTER` / 400 for the implicit list, for `$eq`, and under `$not`. | | `matchesFilterCondition` (formula) | **already compliant.** Measured: `INVALID_FILTER` / 400 for the same three shapes (objectstack-ai#19886 stage 2a). | | `applyFilterCondition` (driver-sql) | **already compliant.** It refuses with 400 at the driver and behind the engine's shared-face seam (table in the objectstack-ai#19882 changeset; not re-measured here). | | `buildWhereSQL` (driver-turso RemoteTransport) | **already compliant.** 400 according to the compile-face table in the objectstack-ai#19886 stage-2a report; not re-measured here. | | `checkCondition` (driver-memory) | **already compliant.** 400 at every depth (table in the objectstack-ai#19882 changeset). | | `translateFieldOperators` (driver-mongodb) | **out of scope.** The driver answers with MongoDB array equality. Platform doors reach it only through the shared face, which refuses (the declared scope of objectstack-ai#19882). | | `lowerAnalyticsWhere` (analytics caller `where`) | **out of scope.** This is the caller-authored filter door (the `INVALID_FILTER` / 400 family), not a read scope. Its object-form `$eq` list cell still reads `accept` in the frozen comparand matrix. The claim records objectstack-ai#19888 against this file. | | `applyHaving` / `matchesHaving` (objectql HAVING) | **out of scope.** A caller-authored filter applied after aggregation, not a read scope. Its answers are recorded in the objectstack-ai#19886 stage-2a report. | The analytics ObjectQL execute route never calls this compiler. It hands the scope to `engine.aggregate`, and the engine's shared-face seam refuses the list with `INVALID_FILTER` / 400 (measured). See the acceptance notes. ## Tests and evidence (head `feb810c3d4`, after merging `origin/main` at `276d96dd23`) - New `src/__tests__/read-scope-eq-array-refusal.test.ts`, 29 tests: - `$eq` lists at every depth, including negation, and beside another operator in either key order; - list-before-member precedence (`[undefined]`, `[{ $field }]`); - the implicit list at every depth; - seven neighbouring shapes that must compile unchanged; - the NativeSQL execute face and the echo face over a real sql.js engine. Both refuse, and no statement reaches the engine. The prescribed `$in` serves exactly the rows it names. - `read-scope-refusal-envelope.test.ts`: inventory row ⑯ added, and the ratchet moves to 16 rows over 14 sites. - `comparand-door-single-source.test.ts`: the frozen matrix's read-scope `$eq` array cell changes from `accept` to the refusal, with a note. It pinned exactly the bind this PR removes. - `pnpm --filter @objectstack/service-analytics test`: 116 files and 2484 tests passed. `typecheck` exited 0, and `tsc --listFiles` includes all three touched test files. - Ablations. Each was run from the committed fix. The mutation went through `scripts/ablation-replace.mjs`, and each restore was proven by the blob hash matching HEAD. - **A:** removing the gate call turned 18 tests red. These include every `$eq` pin, both real-engine faces (zero rows served, and every row served under the negation), and inventory ⑯. - **B:** making the bare-array arm bind turned 11 tests red. - Gates. `dispatch-gates` derives the same 61 at `feb810c3d4` as at `11c11c7dc3`, and all 61 were re-run on `feb810c3d4`: 59 exited 0. Two are NOT MEASURED because their prerequisite was not met (`check:dual-build-cjs-loads` and `check:type-check-debt` need the whole workspace built, and CI builds it). Among the 59: `check-adr-0087-registration --base origin/main` (1 declared-breaking changeset, carrying its disposition), `check-changeset-no-major --base origin/main`, `check:changeset-gate-self-tests` and `check-issue-citations` in its board-probing mode, all exit 0. - Lint, narrowed to the change. `eslint --no-inline-config --format json` over the four touched TypeScript files: 4 files, 0 errors, 0 warnings. `eslint --print-config` resolves a config for each of them. `eslint.config.mjs` never enables type-aware linting (its own note, near line 326), so this diff cannot change the verdict on any untouched file. ## Acceptance notes - **Authoring door, implicit spelling.** The authoring-door half of the card for the implicit spelling is work in the objectstack-ai#19886 lane: draft PR objectstack-ai#19947 refuses `==` against a list at the CEL lowering and in the lint. objectstack-ai#19975 needs nothing more from it. - **Adjacent finding, filed by the seat, not addressed here.** The analytics ObjectQL execute route answers a read-scope list with the engine's `INVALID_FILTER` / 400, not this compiler's 500. - **Premise correction.** PR objectstack-ai#19882, ruling 乙's shared face, merged at 2026-09-24T14:44Z, before this branch was cut from `ae7a35a63b`. The dispatch described it as in flight; it was not. --------- Co-authored-by: Claude <noreply@anthropic.com>
…efused by every driver that answers `$like`, instead of being cut at the NUL on SQLite (objectstack-ai#20041) (objectstack-ai#20124) Fixes objectstack-ai#20041 Clause-②: yes (narrowing) ## What changed On the SQLite faces `$like` / `$ilike` compile to `GLOB`, and SQLite reads a pattern only up to its first U+0000. A pattern holding U+0000 was cut there and answered a different question, with nothing raised. There is no NUL-safe SQLite pattern primitive to compile to instead (`LIKE` cuts the same way, `replace()` cannot target U+0000, `instr()` has no wildcards; measured on objectstack-ai#20024). So the accept set becomes one contract: such a pattern is refused, the way a pattern ending in a lone unpaired backslash already is. - **`packages/spec`:** one exported predicate, `hasNulInLikePattern(pattern)`, beside `hasDanglingLikeEscape` in `src/data/filter.zod.ts`, with its own docblock. `hasDanglingLikeEscape` is byte-identical. The converters (`likePatternToRegexSource`, `likePatternToGlobPattern`, `matchesLikePattern`) are unchanged (see H3). - **Every door that refused a dangling escape now asks the new predicate right after it**, in that door's existing envelope, `INVALID_FILTER` / 400: | door | file | envelope | |:--|:--|:--| | `SqlDriver`'s filter walk (`classifyFilterKey`), before a dialect is chosen | `driver-sql/src/sql-driver.ts` | new builder `nulLikePatternError`, born in the withheld seam (`withheldFilterError`, the operator map as the node, as for the dangling escape) | | `RemoteTransport.buildWhereSQL`'s `$like` / `$ilike` arm, before anything is sent | `driver-turso/src/remote-transport.ts` | new method `nulLikePattern`, through `withheldRefusal`; its withheld sentence is `driver-sql`'s behind the `[RemoteTransport]` prefix | | `driver-memory`'s shape gate (`assertFieldConstraintShape`: the query path and the reference matcher `match()`) | `driver-memory/src/filter-refusal.ts` | new exported `nulLikePatternError`, `driver-sql`'s author text word for word | | `driver-memory`'s QueryAST `comparison` `like` / `ilike` arm, and the `$like` translator floor | `driver-memory/src/memory-driver.ts` | the same builder | - The new check runs AFTER the dangling-escape check at every door, so a pattern with both keeps the refusal it already had. - **Messages.** The withheld class statement names neither the operator variant, the field, the path nor the pattern. The author text (and the server log) reads, for example: `Operator "$like" on field "v" at filter.v.$like has a pattern holding the NUL character U+0000 ("%\u0000"). ...` — the pattern goes through `JSON.stringify`, so U+0000 reaches no message as a raw byte. - **Bounded in-place fix, declared:** the `$like` operator's `.describe()` (`LIKE_DESCRIPTION`, same file) said "A pattern ending in a lone unpaired backslash is refused (INVALID_FILTER)". It now reads "A pattern ending in a lone unpaired backslash, or holding the NUL character U+0000, is refused (INVALID_FILTER)", so the declared contract names the refusal this PR enforces. `content/docs/references/data/filter.mdx` is its regeneration (`gen:schema && gen:docs`): 5 rows, that phrase only. - **A pending release note corrected, which needs your confirmation:** the last bullet of `.changeset/20024-sqlite-glob-stored-nul.md` (not yet released) said "a pattern holding U+0000 is still cut there". At this head that sentence is false, so it now says the pattern is refused by every driver that answers `$like`. `node scripts/check-empty-changeset.mjs` is RED on this by design: its DELIBERATE CORRECTION class says "do NOT restore it -- say so on the PR and get it confirmed". This is that statement. The alternative is to drop the edit and leave that false sentence in the release's changelog beside this PR's own entry. ## H1: every face, base vs head (measured, the PM's hypothesis holds on the SQLite faces) A probe (scratch, not committed) drove the public `find` of each face over one row set: 13 stored values, 12 non-NULL, U+0000 at the start, middle and end, alone, and none (`'a'` + U+0000 + `'b'`, `'ab'` + U+0000, U+0000 + `'z'`, U+0000 alone, `'A'` + U+0000 + `'B'`, `plain`, `''`, `ab`, `a`, `b`, `axb`, `AB`, NULL). Cases: 10 patterns holding U+0000 (`'%'`+NUL, NUL+`'%'`, `'%'`+NUL+`'%'`, `'a'`+NUL+`'b'`, `'a'`+NUL+`'%'`, `'_'`+NUL+`'_'`, backslash+NUL, NUL alone, `$ilike` `'%'`+NUL+`'B'` and `'AB'`+NUL) and 9 NUL-free controls, each bare and under `$not`. Base is `8d76c2d38c`, head is this branch (dists rebuilt at each). | face | U+0000 pattern, base | U+0000 pattern, head | NUL-free control, base = head | |:--|:--|:--|:--| | `SqlDriver` on better-sqlite3 | 20 of 20 differ from `formula`, 0 refused | 20 of 20 refused, `INVALID_FILTER` / 400 | 0 of 18 differ on the NUL-free rows; 12 of 18 differ over rows holding a stored U+0000 (objectstack-ai#20024 item 2 (ii), not this card) | | `SqliteWasmDriver` | 20 of 20 differ, 0 refused | 20 of 20 refused | same as above | | `TursoDriver` local | 20 of 20 differ, 0 refused | 20 of 20 refused | same as above | | `TursoDriver` remote, `makeLibsqlSqliteStub` | 20 of 20 differ, 0 refused | 20 of 20 refused | same as above | | `TursoDriver` remote, real `@libsql/client` `file::memory:` | 20 of 20 differ, 0 refused | 20 of 20 refused | same as above | | `InMemoryDriver.find` (`$`-spelling) | 0 of 20 differ (answers correctly), 0 refused | 20 of 20 refused | 0 of 18 differ | | `InMemoryDriver.find`, QueryAST `comparison` `like` / `ilike` | answers (`like '%'`+NUL gives the one value ending in U+0000) | refused | `like 'ab%'` answers the same rows | | `driver-memory` `match()` | answers (`true` for `'ab'`+NUL) | refused | pinned equal to the query path | | `@objectstack/formula` | the oracle | unchanged (see H2) | the oracle | | `driver-mongodb` `translateFilter` | 38 of 38 cases refused: `$like` is not translated at all | unchanged, not touched (held by draft PR objectstack-ai#19947) | refused | - The five SQLite faces gave byte-identical answer lists on every case, at base and at head. The control answers are byte-identical base vs head on all six driver faces. - Examples at base, SQLite faces: `$like: '%'` + U+0000 returned all 12 non-NULL rows, where `formula` returns the two ending in U+0000; `$like: 'a'` + U+0000 + `'b'` also returned `'a'`; `$like: '_'` + U+0000 + `'_'` also returned `'a'` and `'b'`; `$ilike: 'AB'` + U+0000 also returned `'AB'` and `'ab'`; `$not $like '%'` + U+0000 returned only the NULL row. - Also measured at base, not touched: objectql `applyHaving` refuses every `$like` (`INVALID_FILTER` / 400, "Unsupported operator '$like' in `having`"); service-analytics `compileScopedFilterToSql` refuses it (`READ_SCOPE_COMPILE_FAILED` / 500, fail-closed) and `normalizeAnalyticsFilterTree` refuses it (`INVALID_FILTER` / 400). `lowerAnalyticsWhere` alone passes the node through; the tree build after it refuses. - The Postgres and MySQL arms of `driver-sql` were not measured live (no server here). At head the refusal fires on the walk before the dialect is chosen, pinned by compiling with the `pg` and `mysql2` clients and no server. ## H2: the doors (census) `git grep -n -E '\bNAME\(' HEAD -- 'packages/**/*.ts' ':!**/*.test.ts'`, comments and the definition excluded: - `hasDanglingLikeEscape`: 7 call sites at base and at head. Five are face doors: `filter-refusal.ts` (1), `memory-driver.ts` (2), `sql-driver.ts` (1), `remote-transport.ts` (1). Two are the converters' own backstops in `filter.zod.ts`. Positive control: the PM's list (`sql-driver.ts`, `remote-transport.ts`, `memory-driver.ts` x2, `filter-refusal.ts`) is exactly the five doors. - `hasNulInLikePattern`: 0 at base, 5 at head, one beside each of the five doors. - Every one of the five doors refused a dangling escape at base, and every one must refuse U+0000. `formula` does NOT refuse a dangling escape: `matchesLikePattern` throws, and the arm answers `false` (measured: `matchesFilterCondition({ v: 'abc\\' }, { v: { $like: 'abc\\' } })` is `false`). So by the claim's condition its file is not touched, and it still evaluates a U+0000 pattern. ## H3: where the predicate is called, option (a) Converter consumer census, same grep over non-test sources: - `likePatternToGlobPattern`: 3 calls: `sql-driver.ts` (`likePatternPredicate`), `remote-transport.ts` (`pushLikePattern`), and `driver-memory/src/memory-analytics.ts` (`globSubstringPattern`, the analytics echo of a `$contains` comparand). - `likePatternToRegexSource`: 3 calls: `memory-driver.ts` (2) and the spec's own `matchesLikePattern`. - `matchesLikePattern`: 2 calls: `driver-memory/src/memory-matcher.ts` and `formula/src/matches-filter.ts`. - 8 calls in all: `packages/drivers/**` 6, `packages/formula` 1, `packages/spec` 1. `packages/services/**`, `packages/objectql` and `packages/plugins/**`: 0 (the same grep, whose drivers count is the positive control). Option (b), a throw inside the converters, would have changed three consumers beyond this surface: `memory-analytics`' `$contains` echo would throw a plain `Error` on a comparand holding U+0000; `formula`'s `$like` would answer `false` (its caught throw) instead of the right rows; the reference matcher the same (though its shape gate runs first). So the predicate is called at each door (a), and `filter-like-nul-pattern.test.ts` pins that the JS translation keeps its meaning. ## H4: the withheld seam - `nulLikePatternError` has a row in `sql-driver-compile-refusal-seam.test.ts` and `nulLikePattern` in `remote-transport-compile-refusal-seam.test.ts`; without the row, "every builder that goes through the seam is driven by a row below" is red. The remote pin's local-vs-remote table gained the class too (the withheld sentence is one sentence on both compilers). - policy-marked: `INVALID_FILTER` / 400, the error's own keys exactly `code,status`, no field or pattern on the wire, both in the log / sink. `'author'`: the full text. Unmarked: byte-identical to policy. Merged `$and`: the refusing arm's mark decides, in both arm orders. - Through `TursoDriver` in remote mode no mark survives `toRemoteFilter`, so an author gets the class statement there (the objectstack-ai#20093 fail-closed cost, pinned in `turso-20041-like-nul-pattern.test.ts`). ## H5: declaration - `Clause-②: yes (narrowing)`: the PR adds one public export, `hasNulInLikePattern` on `@objectstack/spec/data` (`api-surface/data.json` +1), so the value is `yes` (AGENTS.md's Clause-② rule, the PR objectstack-ai#20104 precedent), and the arm is `(narrowing)` for the refused patterns. The claim carried `no (narrowing)`; corrected in the patch round after contract review 5828387721. The changeset grades `@objectstack/spec`, `driver-sql`, `driver-sqlite-wasm`, `driver-turso` and `driver-memory` `minor`, with **BREAKING** and a `!` title, following PR objectstack-ai#19971's changeset. - `node scripts/check-changeset-no-major.mjs --base 8d76c2d`: "✓ This diff introduces no `major` bump." (exit 0) - `node scripts/check-adr-0087-registration.mjs --base 8d76c2d`: "✓ check-adr-0087-registration: 1 declared-breaking changeset(s), each carrying an ADR-0087 disposition." — `[BREAKING+bang+clause-②-narrowing] not-required (no-migration-prescription)` (exit 0). The gate chose `not-required`: no key, schema, object definition or stored representation moves, and no rewrite of a stored pattern keeps its meaning. ## Compile surfaces (`references/compile-surfaces.md`, re-verified with its grep: 92 hits in non-test sources) | # | face | this PR | |:--|:--|:--| | 1 | `driver-sql` `applyFilterCondition` (`sql-driver.ts:15953`); `driver-sqlite-wasm` and Turso local by inheritance | CHANGED: the walk refuses a U+0000 pattern on every dialect | | 2 | Turso `RemoteTransport.buildWhereSQL` (`remote-transport.ts:2668`) | CHANGED: the `$like` / `$ilike` arm refuses it | | 3 | service-analytics `compileScopedFilterToSql` (`read-scope-sql.ts:602`) | not touched (claim excludes it): refuses every `$like` already, measured | | 4 | service-analytics `lowerAnalyticsWhere` (`filter-normalizer.ts:2015`) | not touched: passes the node through; `normalizeAnalyticsFilterTree` refuses every `$like`, measured | | 5 | `formula` `matchesFilterCondition` (`matches-filter.ts:212`) | not touched: it refuses no dangling escape, so it is not one of the doors; it evaluates a U+0000 pattern correctly (the oracle above) | | half | objectql `applyHaving` / `matchesHaving` (`having-filter.ts:279` / `:292`) | not touched: refuses every `$like`, measured | | thawed | `driver-memory` `checkCondition` (`memory-matcher.ts:361`) and the query path | CHANGED: the shape gate both run first refuses it; the QueryAST arm and the translator floor too | | thawed | `driver-mongodb` `translateFieldOperators` (`mongodb-filter.ts:832`) | not touched (draft PR objectstack-ai#19947 holds it): refuses every `$like` through its `default:` arm, measured through `translateFilter` | ## Tests New pins, each asserting `code`, `status` and the path (never a bare `toThrow()`), plus NUL-free controls and the dangling escape beside U+0000: - `packages/spec/src/data/filter-like-nul-pattern.test.ts` (5): the predicate, the escaped U+0000, its independence from the dangling escape, and the converters unchanged. - `driver-sql/src/sql-driver-20041-like-nul-pattern.test.ts` (84): 9 patterns x bare / `$not` / `$or` / `$and`, unmarked (class only, path in the log, through `find` and `count`) and author-marked (operator, field, path); the dangling escape keeps its refusal also beside U+0000; 8 controls equal `formula`; `sqlite`, `pg` and `mysql2` compiles refuse on the walk. - `driver-sqlite-wasm/src/sqlite-wasm-20041-like-nul-pattern.test.ts` (16), the same through sql.js, controls checked against `formula`. - `driver-turso/src/turso-20041-like-nul-pattern.test.ts` (17): local, remote over the stub and remote over a real libSQL engine; no `FROM` statement reaches the engine for a refused filter; 8 controls, equal on all three transports. - `driver-memory/src/memory-20041-like-nul-pattern.test.ts` (21): the query path, the QueryAST spelling and `match()`, with controls. - Consumer pins changed: the two seam enumerations above (one row each; the remote one also one local-vs-remote row). No other consumer pin reads a U+0000 `$like`: `git grep` over every `*.test.ts` that names `$like` / `$ilike` and U+0000 finds none outside this PR. Suites (`vitest run --maxWorkers=2`), at `5f2e6f246d` (code equal to this head but for a comment in `memory-driver.ts`), dists rebuilt: - `driver-sql`: 186 files passed, 11 skipped; 3024 tests passed, 170 skipped. - `driver-turso`: 69 files, 1629 tests passed. - `driver-sqlite-wasm`: 33 files, 622 tests passed. - `driver-memory`: 53 files, 1269 tests passed (re-run at `049b3a6c95`, same). - `formula`: 35 files, 978 tests passed. - `spec`: 570 files, 16369 tests passed, 2 todo. - `plugin-auth` (consumer): 114 files, 2440 tests passed. - `typecheck` exits 0 in `spec`, `driver-sql`, `driver-sqlite-wasm`, `driver-turso`, `driver-memory`. `tsc --listFiles` counts each new driver test once in its package program; the spec test once in `tsconfig.test.json`, which `check:test-typecheck` compiles. ## Before-red and ablations (one-off, no file left behind) All from committed state, through `scripts/ablation-replace.mjs` (anchor 1 -> 0 on disk, blob changed), restored with the blob equal to `HEAD`, `git diff HEAD` empty and `git status --porcelain` empty. 1. **Every driver's predicate replaced by a never-true local** (the four source files at once, which is base behaviour at every door). `driver-sql` rebuilt and `ablation-dist-preflight` found the marker in `dist/` before the run. Predicted and observed exactly: - `sql-driver-20041`: 75 failed, 9 passed (the controls, the dangling escape, and nothing else green); - `sql-driver-compile-refusal-seam`: 4 failed, 91 passed (the new row); - `sqlite-wasm-20041`: 10 failed, 6 passed; `turso-20041`: 8 failed, 9 passed; `remote-transport-compile-refusal-seam`: 6 failed, 96 passed; `memory-20041`: 15 failed, 6 passed; - the dangling-escape suites `sql-driver-like-pattern` (22) and `memory-like-pattern` (15) stayed green. Restored, rebuilt, and `ablation-dist-preflight --absent` passed with the tree clean. 2. **The new builder bypassing the seam** (`nulLikePatternError` calling `unsupportedFilterError`, `nulLikePattern` calling `invalidFilterError`). Predicted and observed: `driver-sql` 78 failed of 179 (both enumeration assertions, the row's 4, and the 72 path / log cases of the new suite); remote seam pin 6 failed of 102. ## Gates (at `049b3a6c95`) - `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` from the real diff (17 paths, merge base `8d76c2d38c`): 110 commands, the PM's 79 plus 31 the docs, `driver-memory` and changeset paths add. All 110 run after a full `turbo run build --filter='./packages/*' --filter='./packages/*/*'` (72 tasks); `--ran`: "110 derived, 110 run, 0 NOT-MEASURED, 0 UNRUN". - 109 exit 0. **One exits 1, on purpose:** `node scripts/check-empty-changeset.mjs --base origin/main`, the DELIBERATE CORRECTION of `.changeset/20024-sqlite-glob-stored-nul.md` above. - Roster families beside these paths, run by hand: `check-changeset-fixed`, `check:meta-url-spelling`, `check:spec-changes`, `check:authz-resolver`, `check:error-code-casing`, `check:filter-alias-parity`, `check:engine-double-contract`, `check:error-status-conformance`: all exit 0. - `pnpm --filter @objectstack/spec check:generated`: all 15 artifacts up to date after `gen:api-surface`, `gen:export-origins`, `gen:schema` and `gen:docs` (`api-surface/data.json` and `export-origins/data.json` gain `hasNulInLikePattern`; `authorable-surface.base.json` untouched). - `pnpm check:driver-conformance`: "50 covered cell(s), 0 in the DEBT ledger, 0 exempt" at base and at head. - `node scripts/check-issue-citations.mjs --base 8d76c2d`: 17 citations across 5 files, all resolve. - `pnpm check:nul-bytes` passes, and a control-byte scan of the changed files finds none. U+0000 is spelled `String.fromCharCode(0x00)` in every file. - **Re-run at the merged head `b88ca46d90`** (merge base `6780e34af5`, 17 paths, after a full `turbo run build`): the derivation gives the same 110 commands; 109 exit 0 and `check-empty-changeset` exits 1 on the objectstack-ai#20024 correction; `--ran`: "110 derived, 110 run, 0 NOT-MEASURED, 0 UNRUN". `check-changeset-no-major` and `check-adr-0087-registration` exit 0; `check:generated` all 15 up to date. - ESLint, narrowed: `--no-inline-config --format json` over the 12 changed `.ts` files reports 12 files, 0 errors, 0 warnings. `eslint.config.mjs` enables no type-aware linting (the printed `parserOptions` for `sql-driver.ts` are `{"ecmaVersion":"latest","sourceType":"module"}`), so this diff cannot move the verdict of a file it does not touch. The full `pnpm lint` is CI's. ## Acceptance notes - **Not this card:** a NUL-free pattern matched against a STORED value that holds U+0000 still differs on the SQLite faces (12 of 18 control cases in the probe, unchanged). That is objectstack-ai#20024 item 2 (ii), and objectstack-ai#20024 remains open for it. - `content/docs/protocol/objectql/query-syntax.mdx` still names only the dangling escape among refused patterns. It is not false, only incomplete, and it is outside this claim's file surface, so it is left for the next PR that touches the page. - `formula` keeps evaluating a U+0000 pattern (correctly). A write-side `check` with such a pattern therefore answers where the read side refuses; the read side refusing loudly means the two can no longer silently disagree. - `origin/main` at `6780e34af5` is merged into this branch (merge commit `2b8dd90200`, no conflicts): the 10 commits past the old merge base `8d76c2d38c` are `aa04ea2964`, `fa00ebf447`, `7b27bd00c7`, `7a13e0562a`, `7c1039b388`, `55daf89d74`, `226e00c038`, `7b068877ce`, `0d73ff6245`, `6780e34af5`. One of them touches this PR's paths: `55daf89d74` (PR objectstack-ai#20114) changes `packages/drivers/driver-turso/src/remote-transport.ts` and `remote-transport-compile-refusal-seam.test.ts`; both auto-merged, and the branch's delta against the new merge base is still exactly its 17 files, +1017/−16. After the merge: `turso-20041-like-nul-pattern` 17/17 and `remote-transport-compile-refusal-seam` 136/136 (102 plus objectstack-ai#20114's rows), `driver-turso` 69 files / 1663 tests; `sql-driver-compile-refusal-seam` and `sql-driver-20041-like-nul-pattern` 179/179; `sqlite-wasm-20041` 16/16, memory 36/36, spec 31/31. --------- Co-authored-by: Claude <noreply@anthropic.com>
…ad of sending it to MongoDB as a literal (objectstack-ai#19949) (objectstack-ai#20182) Fixes objectstack-ai#19949 Clause-②: no (narrowing) ## What changed `translateFilter` in `@objectstack/driver-mongodb` now refuses a `{ $field }` cross-field reference in any comparand position. The refusal is `INVALID_FILTER` / 400, the envelope every other filter refusal on this driver uses. Before, the driver sent the reference to MongoDB as a literal sub-document. This is the maintainer's ruling B on the card (triage comment 5807932277), quoted verbatim: 「维护者答:「19949 B」。」 The driver refuses the reference and does not implement it: there is no `$expr` column-to-column lowering. The driver-mongodb investment freeze (objectstack-ai#5499) stays in force for everything else. - `packages/drivers/driver-mongodb/src/mongodb-filter.ts`: one new gate in `classifyFilterKey`, on the shape walk that PR objectstack-ai#19947 extended. It runs before the other comparand gates, so every reference gets the same refusal whichever operator carries it. `carriesFieldReference` checks three positions: the whole constraint (the bare form, or a list holding a reference), each `$`-prefixed operator's comparand, and each member of a list comparand. The reference test matches the spec schema door's `isFieldReferenceShape` (a non-array object with a `$field` key), so a malformed `{ $field: 42 }` is refused too. - The message names the unsupported feature ("The MongoDB driver does not support field-to-field comparison") and leaves out the fields, the operator and the position. This follows the `$ne`-array refusal from PR objectstack-ai#19947, because the filter may be an RLS policy the caller did not write. - `packages/drivers/driver-mongodb/src/mongodb-field-reference-refusal.test.ts`: a new test file for this card, separate from PR objectstack-ai#19947's files. - `.changeset/19949-mongodb-field-reference-refused.md`: a `minor` bump for `@objectstack/driver-mongodb`, marked BREAKING (accept-set narrowing). The ADR-0087 disposition is `not-required (no-migration-prescription)`. No stored policy can be converted to keep its meaning, because this driver has nothing to convert it to. ## Rows: base `3bd28e2b2e` vs head `00612182e9` MongoDB selection was read through **mingo 7.2.4 as the proxy** (the query-semantics library `driver-memory` uses), over the card's rows `r1 {s:'a', t:'a'}` and `r2 {s:'a', t:'b'}`, where `ref` is `{ $field: 't' }`. A live `mongod` is NOT MEASURED: this container cannot fetch the binary, and the package's `mongodb-memory-server` suites are opt-in and were skipped. The readings were taken with a scratch script outside the suite, because mingo is not a dependency of this package. | shape | base | head | |:--|:--|:--| | `s $ne ref` (what `record.s != record.t` lowers to) | translated as a literal: selects **r1, r2** | 400 | | `s $eq ref` (what `record.s == record.t` lowers to) | translated as a literal: selects none | 400 | | `$gt` / `$gte` / `$lt` / `$lte` ref | literal: none | 400 | | `$in: [ref]` | literal: none | 400 | | `$nin: [ref]` | literal: selects **r1, r2** | 400 | | `$between: [ref, 'z']` / `['a', ref]` | literal bound: none | 400 | | `$and: [s $ne ref]` / `$or: [s $ne ref, s='zz']` | selects **r1, r2** | 400 | | `$or: [{}, s $ne ref]` | TRUE identity, `{}`: r1, r2 | 400 (the gate is on the walk) | | `$not: {s $eq ref}` (lowered to `$nor`) | selects **r1, r2** | 400 | | `$not: {s $ne ref}` | none | 400 | | `s $ne {$field:'t', addDays:1}` | selects **r1, r2** | 400 | | `s $eq {$field:'t', addDays:{$field:'n'}}` | none | 400 | | `s $ne {$field:42}` (malformed) | selects **r1, r2** | 400 | | `$notContains: ref` | regex on the text of a plain object: selects **r1, r2** | 400 | | `$contains` / `$startsWith` / `$endsWith` ref | the same regex: none | 400 | | `$exists: ref` | lowered to `$eq: null`: none | 400 | | bare `{ s: ref }` | already 400 (unknown operator `$field`, message names the field) | 400, the cross-field message | | `$icontains: ref` / `$null: ref` / `$ne: [ref]` | already 400 (their own comparand gates) | 400, the cross-field message | | `$nor` at node level | already 400 (undeclared combinator) | unchanged | | **CONTROL** `s $ne 'a'` / `s $eq 'a'` | none / r1, r2 | identical document, identical selection | | **CONTROL** `t $ne 'b'` / `t $eq 'b'` | r1 / r2 | identical | | **CONTROL** `t $in ['b']` / `t $nin ['b']` / `t $between ['a','a']` / `$not {t $eq 'b'}` | r2 / r1 / r1 / r1 | identical | ## The RLS read path fails closed Measured end to end with a scratch script. The path was `compileCelToFilter`, then the real `RLSCompiler`, the real `SecurityPlugin` with a `rowLevelSecurity` policy (`operation: 'all'`), `ObjectQL`, and the real `MongoDBDriver` over a stub `Db` whose collection selects with mingo. The caller is a MEMBER holding the permission set. - `compileCelToFilter('record.s != record.t')` and `'s != t'` both return `{ s: { $ne: { $field: 't' } } }`. `RLSCompiler.compileFilter` keeps it, because `s` and `t` are declared. - Base: under `using: 's != t'`, `find` returned **r1, r2**, `count` returned **2**, and `findOne({ id: 'r1' })` returned **r1**, the row the policy excludes. The server received `{"s":{"$ne":{"$field":"t"}}}`. - Head: `find`, `findOne` and `count` each throw `INVALID_FILTER` / 400, and the collection is asked **0** times. The refusal is not swallowed and the read never falls back to the unfiltered set. `using: 's == t'` (base: 0 rows, the wrong answer in the fail-closed direction) is also refused. - Controls, identical at base and head: `using: 't == "a"` gives r1 / count 1, `using: 't != "a"` gives r2, and `using: 's == "a"` gives r1, r2. Every driver door reads `where` through `translateFilter` (`find`, `findOne`, `count`, `updateMany`, `deleteMany`, `aggregate` via `buildAggregationPipeline`, `explain`), so the one gate covers them all. The suite pins each door and the engine rethrow. It uses an ObjectQL middleware that composes the policy the way the security middleware does, because `@objectstack/plugin-security` is not a dependency of this package. ## Collateral - Every literal comparand translates to the same document, in the controls above and in the new suite. - The `$ne`-array refusal from PR objectstack-ai#19947 is unchanged for literal arrays: its own file and a control in the new file both stay green. A `$ne` array that holds a reference now gets the cross-field message instead. Same envelope. - The equality-slot array pin from objectstack-ai#19757 still holds: `translateFilter({ tags: ['a'] })` and `$eq: ['a']` pass through unchanged. The gate only fires when a list member is a reference. - Three shapes were already refused and now get the cross-field message instead of their old one: the bare `{ s: ref }` (old message named the field and path), `$icontains: ref`, and `$null: ref`. Same `INVALID_FILTER` / 400 envelope. No test pinned their old wording for a reference: `$field` had 0 hits in this package's tests at base. - No workspace consumer's tests use `$field` with this driver. That was checked with a grep in `runtime`, `service-datasource` and `cli`. Exports and types are unchanged. ## Tests at `00612182e9` - `pnpm --filter @objectstack/driver-mongodb test`: **28 files passed, 5 skipped (the opt-in live-`mongod` suites); 630 tests passed, 147 skipped**. - `pnpm --filter @objectstack/driver-mongodb exec vitest run --maxWorkers=2 src/mongodb-field-reference-refusal.test.ts`: 39 passed. - `pnpm --filter @objectstack/driver-mongodb typecheck`: `tsc --noEmit`, then `check:test-typecheck` with 0 errors. `tsc -p tsconfig.test.json --listFiles` includes the new test file among 33 test files. - **Reverse verification (ablation)**, run after the fix was committed at `a53249d5e5`. `node scripts/ablation-replace.mjs` deleted the gate line (anchor 1 to 0, blob `7b33578be43b` to `343b8c5aa323`). With the gate gone, the new file went **33 failed / 6 passed**: every refusal red, all six controls green. The tool then restored the file (blob back to `7b33578be43b`, the HEAD blob, and `git diff HEAD` empty). No `dist/` step was needed, because the suite imports the translator by relative path from `src`. - The CJS entry `dist/index.js` loads and refuses, built at `00612182e9`. ## Gates - `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 59 commands from this diff at `00612182e9`. All 59 were run, and the `--ran` reconciliation reports 57 run and 2 NOT MEASURED. - 57 exited 0. - `pnpm check:dual-build-cjs-loads` exited 3: PREREQUISITE NOT MET, because it needs the whole workspace built. NOT MEASURED. Narrowed instead: this package's CJS entry loads, as above. - `pnpm check:type-check-debt` exited 3: PREREQUISITE NOT MET, because the ledgered packages' closure is not built. NOT MEASURED. This package carries no DEBT entry, and its own test-layer typecheck is green. - Also run: `node scripts/check-issue-citations.mjs --base 3bd28e2` (exit 0, 5 citations resolve), plus the five artifact-roster gates whose rosters sit under this diff's directories: `check:authz-resolver`, `check:error-code-casing`, `check:filter-alias-parity`, `check:object-def-param-keys` and `check:tenant-chokepoint`, all exit 0. - `check-changeset-no-major` and `check-adr-0087-registration` both pass on the changeset. The first accepts the `minor` level, and the second reads the `not-required (no-migration-prescription)` disposition. - Lint was narrowed to the two changed TypeScript files: `eslint --no-inline-config --format json` reports 2 files, 0 errors, 0 warnings. Both files are in the config's population, because `--print-config` gives each one a rule set. The narrowing cannot hide anything: the config enables no type-aware linting (no `parserOptions.project` or `projectService`), so this diff cannot change the verdict on any other file. The full `pnpm lint` is left to CI. ## Acceptance notes - Not in scope, per the ruling: implementing field-to-field comparison on MongoDB (a `$expr` lowering). A policy that needs it cannot be enforced on this driver. It is now refused instead of read without the restriction. - An observation, not a card. `$exists` with a non-boolean comparand still translates to `{ $eq: null }` ("has no value") on this driver, because the arm tests `value === true`. Measured on `translateFilter` at head: `'yes'`, `1`, `null` and `'false'` all give `$eq: null`. It has no comparand gate like `$null`'s, and the engine's comparand-type door lists `$exists` as a scalar operator. Only a reference is refused here. Whether a public door delivers this shape, and what the other drivers answer, was not measured. It is left alone under the driver-mongodb freeze. Carrier: none. --- _Generated by [Claude Code](https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…ace and at FieldOperatorsSchema.$ne (objectstack-ai#20204) Part of objectstack-ai#19886 Clause-②: no Stage **2b** of objectstack-ai#19886, and only 2b: ruling A item 1 (record `5805254639`), at its two named positions. The shared comparand-shape face refuses an array under `$ne` for every driver, and `FieldOperatorsSchema.$ne` refuses it at parse. Both print one remedy text, which names `$nin`, the list-negation operator `FieldOperatorsSchema` declares. objectstack-ai#19886 stays open for stage 2d (the three same-class leaks recorded in `5806608550`), which this PR does not touch. `Clause-②: no` above is the claim's line (`5853529590`), copied as it stands. The changeset carries `Clause-②: no (narrowing)`, because AGENTS.md makes a narrowing BREAKING and `check:adr-0087-registration` reads the arm there. Both give the value `no`. Dispatched by the `domain:spec` seat 1 PM loop, session `session_01Rjy9MeetSfq34PKn81CRiN`, branch `claude/issue-19886-ne-array-shared-face-and-schema-door`, base `9e7824a4`, then `origin/main` `560b724c` merged in (`90b9d496`). Every reading below names the tree it was taken on. ## 1. Measured first (before the change, `origin/main` `9e7824a4`) The ruling quoted, verbatim: 「The shared comparand-shape face refuses an array under `$ne` for every driver, and `FieldOperatorsSchema.$ne` (`filter.zod.ts:269`) refuses it at parse — one remedy text, naming the declared list-negation operator by its spec spelling (the dev reads it off `FieldOperatorsSchema`; ⛔ not invented here). ⛔ No alias, ⛔ no window.」 The line number was `:269` at ruling time. On `9e7824a4` the documentation copy `EqualityOperatorSchema.$ne` sits at `:300` and the enforced `FieldOperatorsSchema.$ne` at `:1483`. Both were `z.any()`. Stages 2a (PR objectstack-ai#19946) and 2c (PR objectstack-ai#19947) had already closed the two faces that ANSWERED the shape: the formula evaluator and `driver-mongodb`'s walk. What still accepted an array under `$ne`, read on `9e7824a4` with a `tsx` probe against `src/`: | door | `{ tags: { $ne: ['a'] } }` (and `[]`, and under `$or` / `$not`) | |:--|:--| | `assertListComparandShapes` (the shared face) | **PASS**, at every depth | | `parseFilterAST([['tags', 'ne', ['a']]])` | **lowered** to `{"tags":{"$ne":["a"]}}` and passed | | `FieldOperatorsSchema` / `EqualityOperatorSchema`, `{ $ne: ['a'] }` and `{ $ne: [] }` | **`success: true`** | | `NormalizedFilterSchema`, `{ $and: [{ s: { $ne: ['a'] } }] }` | **`success: true`** | | `FilterConditionSchema` / `DatasetSchema` `filter: { stage: { $ne: [...] } }` | `success: true` (not a named position; see section 7) | | `ViewFilterRuleSchema`, `not_equals` with `['a']` | already refused at authoring | | control: the face on `$eq: ['a']` (the landed equality arm) | refused, `INVALID_FILTER` / 400 | | controls: `$ne: 'a'`, `$ne: null`, `$ne: { $field: 'budget' }` | pass at every door | `objectql`'s engine binds the face through its delegating wrapper (`packages/objectql/src/filter-comparand-shape.ts`), so it passed too. The analytics `where` door runs the face, so on `main` it **compiled** the shape instead of refusing it. `normalizeAnalyticsFilterTree({ where: { stage: { $ne: ['won', 'lost'] } } })` returned `stage` not-set OR `stage` not-equals `["won","lost"]`. Both analytics strategies render a not-equals member from its first value (`values[0]` in the native SQL strategy, `v0` in the ObjectQL strategy; read at source, not executed). So `'lost'` was dropped in silence, and a chart counted rows its filter named. This tree was measured with the face's `$ne` arm removed (the ablation in section 4), which is `main`'s face. The analytics door's own code is unchanged by this PR. ## 2. What changed Three source files in `packages/spec/src/data/`. Nothing in any driver, in formula, or in objectql. - **`filter-comparand-refusal-text.ts`**: the module both doors import, which already carried the equality-slot sentence. It gains `NIN_OPERATOR_SPELLINGS`, the `$ne` remedy `ARRAY_INEQUALITY_COMPARAND_REMEDY`, and `arrayInequalityComparandMessage`. The `$eq` and `$ne` sentences now share one private template, and the equality sentence is byte-identical, as its existing pins prove. - **`filter-comparand-shape.ts`**: a `$ne` arm in the existing walk (⛔ no second walk), beside the `$eq` arm, with its own error builder. The `$nin` row of `LIST_COMPARAND_OPERATORS` now reads its spellings from the shared module, as the `$in` row already did. The module docblock gains the ruling's section, and the equality section's bullet that said `$ne` was not judged is corrected. - **`filter.zod.ts`**: `inequalityComparandSchema()`, one factory shared by `FieldOperatorsSchema.$ne` and its documentation copy `EqualityOperatorSchema.$ne`. It mirrors `equalityComparandSchema()` exactly: `z.any()` except an array, and the describe `NE_DESCRIPTION` is unchanged. The face's refusal, verbatim: ```text Operator "$ne" on field "tags" requires a single comparable value, but received an array (["a"]) at where.tags.$ne. For "none of these values" use {"$nin": […]} (authoring: nin, not_in, notin). The filter was NOT applied, and an unapplied filter would have returned the UNFILTERED result set. ``` The operator slot's issue (code `custom`, path `$ne`), verbatim: ```text Operator "$ne" requires a single comparable value, but received an array (["won","lost"]). For "none of these values" use {"$nin": […]} (authoring: nin, not_in, notin). The filter was NOT applied, and an unapplied filter would have returned the UNFILTERED result set. ``` The first sentence is `driver-memory`'s `arrayComparandError` for `$ne`, word for word. The remedy is ONE operator, as the ruling says: `$nin`, read off `FieldOperatorsSchema` ("Not in list"), with the authoring spellings that lower to it. `$notContains` is not offered, because it is declared on a STRING comparand and is not a list operator. The equality slot's `$in` / `$contains` are not offered either, because they answer a different question. `$nin` is also the remedy the formula and `driver-mongodb` faces already name for this shape. Also in the diff: one ADR-0087 semantic entry (`18.filter-ne-array-comparand-refused.ts`) and the regenerated `registry.ts`. The `dropped-refinements.baseline.json` ledger gains the three `$ne` sites the build named (`data/EqualityOperator` `$ne`, `data/FieldOperators` `$ne`, `data/NormalizedFilter` `lazy.$not.options[0].valueType.$ne`; sites 574 to 577). The changeset is `@objectstack/spec` minor. It carries the BREAKING banner, `Clause-②: no (narrowing)`, and the ADR-0087 `registered` marker for `filter-ne-array-comparand-refused`. `check:generated` reads all 15 artifacts current on `90b9d496`. `spec-changes.json` and the upgrade guide do not list major-18 entries (the sibling equality entries are absent too), so they did not move. ## 3. Pins Every accept/refuse change is pinned, and every refusal case asserts its envelope. - `packages/spec/src/data/filter-comparand-shape.test.ts` (the face). The `it.todo` that stood for this arm is replaced by real pins: - 9 refusal rows: the lowered `ne`, `not_equals` and `!=`; the object passthrough; `[]`; nested under `$and` (lowered), `$or` and `$not`; and beside a legal `$eq`. Each asserts `code` `INVALID_FILTER`, `status` 400, the path, and the `$ne` leading sentence. - The full sentence and the single `$nin` remedy, including that `$in`, `$contains` and `$notContains` are absent. The context prefix is kept. - Every AST spelling that lowers to `$ne` refuses an array. The spellings are derived with a scalar probe, and a guard pins exactly six: `!=`, the angle-bracket pair, `ne`, `neq`, `not_equals`, `notequals`. - `$nin` is a key of `FieldOperatorsSchema`, and the spellings the message lists are exactly those that lower to it. - Controls: `$ne: null` (both spellings), scalars, `0`, `false`, `''`, a `Date`, and a `{ $field }` reference. - The no-`$`-key boundary. - Three rows in the 500-char client-bound test. - The array-valued `LIT CONTROL` set is now exactly `$between`, `$in`, `$nin`. - `packages/spec/src/data/filter-ne-array-schema-door.test.ts` (new; the operator slot and the one-text rule): - §1: both copies refuse `$ne: [...]` and `$ne: []`, asserting the issue code `custom`, the path `$ne` and the full prescription. A `$ne` array beside a legal `$eq` raises one issue, at `$ne`. `NormalizedFilterSchema` refuses at `$and.0.stage.$ne`, with a scalar control. - §2: the face's envelope at four positions. - §3: the slot's message equals the face's, character for character, after removing only ` on field "stage"` and ` at where.stage.$ne` (both proved present first). This is checked over four lists. Every FilterArray spelling gives the face's sentence, and the remedy is declared and single. - §4: controls at BOTH doors, accepted and KEPT. - §5: one `it.todo` for the stored-carrier walk (section 7), ⛔ deliberately not pinned green. ## 4. Proof the pins can fail (ablation, two legs, each position alone) Both legs ran from committed state (`3757d64a`), with `scripts/ablation-replace.mjs` (anchor must hit; blob hash asserted), a `trap` restore on `EXIT INT TERM` against an absolute path, and restore proven by `git diff HEAD` = 0 bytes. - **M1, the face arm cut** (`throw` guarded by an impossible field name): on disk marker 1 and anchor 0. Spec rebuilt, and `ablation-dist-preflight` found the marker in 6 dist files. Results: - Spec: **24 failed** / 143 passed. Every face `$ne` pin, the §2 / §3 two-door parity, and the equality door's re-judged `$ne` test went red. The slot's §1 pins stayed **green**, which is correct, because only the face was cut. - `service-analytics`: **1 failed** (the flipped pin), and its parity file stayed green by design. - The probe printed the pre-fix analytics tree quoted in section 1. - Restore: 0 diff bytes. Rebuilt, preflight `--absent` read the marker absent from all 222 dist files, the tree was clean, and both suites were green again (167 passed / 2 todo; 160 passed). - **M2, the operator slot cut** (the `addIssue` guarded by an impossible length; spec tests read `src/`): on disk marker 1 and anchor 0. Spec: **12 failed** / 155 passed. Every slot pin in §1 and §3 went red, plus the `LIT CONTROL` set, which read `$ne` as array-valued again. The face pins stayed **green**. Restore: 0 diff bytes, and the tree was clean. The two red sets are disjoint where they should be, so each pin is tied to the position it names. ## 5. Pin sweep and consumer suites Pin sweep. Error code and message were grepped repo-wide: `$ne` followed by an array literal, the FilterArray `ne`-family triples carrying an array, and `not_equals` view rules. Pins the new refusal made FALSE, flipped in one round, each to assert the new substance: - `filter-comparand-shape.test.ts`: the `it.todo` and the `LIT CONTROL` set. - `filter-equality-array-schema-door.test.ts` §4, the row `$ne carrying an array — not this ruling`. It asserted that the face PASSES the shape, and that half is false now. It is re-judged into its own test, which asserts the face's `$ne` refusal (envelope, the `$nin` remedy, not the `$in` one). It keeps the row's real guard: the carrier walk's EQUALITY arm raises nothing for `$ne`. That is asserted on the equality sentences, not on `success`, so no ruling-less acceptance is pinned green. - `service-analytics` `where-equality-slot-list-refusal.test.ts`: `.not.toThrow(/requires a single comparable value/)` was false. It now asserts the envelope, byte equality with the face, the `$ne` sentence and the `$nin` remedy, and that the words are not this door's equality-slot sentences. Pins that move by construction and were read, not edited: - `objectql` `engine-aggregate-having-comparand-shape.test.ts` (the `$ne: [500]` row, through the engine's wrapper) is written as parity with the face. It now takes its refusal branch, asserting the envelope and the face's message on both doors. - `service-analytics` `where-face-arms-refusal.test.ts` (the parity block) now compares two refusals. - `driver-memory`'s AST-vocabulary probe helper now reads `undefined` for the `ne` spellings and hands them the same scalar it always did. Consumer suites. Every suite below ran through `scripts/pm/os-verify-lock.sh`, with its exit code captured before any pipe. Two heads: | suite | `3757d64a` (before the merge of `main`) | `90b9d496` (after it) | |:--|:--|:--| | `@objectstack/spec`, the whole `local` project (3 shards) | 541 files passed, 0 failed (15874 tests, 2 todo) | 542 files passed, 0 failed (15935 tests, 2 todo) | | `@objectstack/service-analytics` | full: 128 files, 3017 tests passed | the 2 pin files: 160 passed | | `@objectstack/objectql`, the 29 files touching the face, `parseFilterAST`, comparands or `$ne` | 1053 passed | not re-run (untouched by the merge) | | `@objectstack/formula` | full: 36 files, 1002 passed | full: 37 files, 1027 passed | | `@objectstack/lint` | full: 108 files, 4156 passed | not re-run | | `@objectstack/driver-memory` | full: 53 files, 1269 passed | not re-run | | `@objectstack/driver-mongodb` | full: 27 passed, 5 skipped (live `mongod`) | full: 28 passed, 5 skipped | | `@objectstack/driver-sql`, 44 filter / comparand files | 42 passed, 2 skipped (live PG / MySQL) | not re-run | | `@objectstack/plugin-security`, the RLS / write-check files | 33 files, 1123 passed | 34 files, 1138 passed | The post-merge re-run is a declared narrowing. It covers the packages the merged commits touched (`spec`, `formula`, `driver-mongodb`, `plugin-security`) and my two analytics pin files. The rest were green on `3757d64a`, and the merge changed neither them nor this diff. Typecheck on `3757d64a`: `pnpm --filter @objectstack/spec run typecheck` exited 0; its test layer held `test-typecheck-debt.json` unchanged. `pnpm --filter @objectstack/service-analytics run typecheck` exited 0, and `tsc --listFiles` confirms the edited test file is in that program. On the merged head `90b9d496` the spec typecheck is NOT MEASURED at the time this PR opened. Two lock acquisitions returned 99 across about 20 minutes, behind a single holder of more than 17 minutes. The merged-in commits touch no file this diff imports, and CI's required `TypeScript Type Check` lane measures this head. The report comment on objectstack-ai#19886 carries the reading if it lands before the report. eslint (`--no-inline-config --format json`) on the 9 touched `.ts` files at `90b9d496`: 9 files linted, 0 errors, 0 warnings. The population is read from `eslint.config.mjs`: every file sits in its `**/*.{ts,…}` and `packages/**/*.{ts,…}` objects. The config never enables type-aware linting (no `parserOptions.project`, no typed rules), so this diff cannot move the verdict on any untouched file. ## 6. Census (Zone 2 item 5): no producer - This repository, `9e7824a4`. `$ne` followed by an array literal in `packages/**`, `examples/**`, `apps/**`, `scripts/**`, `content/**` and `skills/**`: 20 hits, all tests, refusal code, comments or migration prose. Control: `$in` followed by an array in `packages/**` and `examples/**` has 901 hits. The FilterArray `ne`-family with an array has 0 hits. A CEL `!=` against a list literal in platform objects, examples and `packages/qa/**` (non-test) has 0 hits. `$ne` fed by a variable in non-test source has 11 sites, and none builds a list. - objectui at the `.objectui-sha` pin `f8a9d0fb05`: 2 hits, both its own refusal test. Its dataset builder's `notEquals` is scalar-arity (the list-arity set is `in`, `not_in`), and the summary editor lists only `in` / `notIn`. The control has 46 hits. - cloud `main` `48d7066`: 0 hits. The control has 33 hits. A real producer would have changed what ADR-0087 owes. None exists, so the entry carries no D2 conversion. ## 7. What this does NOT do (acceptance notes) - **The stored-filter carrier walk.** `FilterConditionSchema`, which every stored carrier (dataset, widget, report, rollup, and the rest) parses through, does not route a field's operator map through `FieldOperatorsSchema`. Its walk judges `$eq` and not `$ne`. So `DatasetSchema` with `filter: { stage: { $ne: ['won', 'lost'] } }` still parses green on this head, and every query that uses it is refused at the face. Ruling A names the face and the operator slot, not that walk, and objectstack-ai#19889's ruling had to name `FilterConditionSchema` explicitly for the equality slot. Extending the walk narrows every carrier's accept set, so it is ⛔ not taken here. It is raised in the report for the seat, and pinned only as an `it.todo`. This is not a new split: on `main` every backend already refused the shape at query time. - Stage 2d (the ordering operators with an array, a nested array inside `$in`, a `{ $field }` referent to a multi-valued field) is untouched, and so are all driver and formula sources. - Sentences elsewhere that this change makes stale, noted and ⛔ not edited here because they are outside this claim's file surface: - `driver-memory`'s `arrayComparandError` text says the spec's comparand door "leaves this position to the driver". That has been untrue for the equality slot since objectstack-ai#19757, and is now untrue for `$ne`. It is reachable only by a caller that hands a raw filter to the driver. - The unreleased entry `filter-equality-array-comparand-refused-at-save` describes `$ne` as a position "which no ruling has decided". - The unreleased changeset `19889-filter-schema-door-array-equality.md` lists "`$ne` carrying an array is not judged" among what that change did not do. - The published JSON Schema still reads `{}` at `$ne`, which is declared in the dropped-refinements ledger. ## 8. Gates Derived on the actual change set (`node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack`, merge base `560b724c`, head `90b9d496`): 89 commands. That is the dispatch lead's 77 plus 12 this diff adds (the changeset families, `check:where-matcher`, `check:engine-double-contract`, `check:objectql-double-limit`, `check:type-check-coverage`, `check:type-check-debt`, and others). None of the lead's commands dropped out. Every line was run with its exit code written to disk, then reconciled with `--ran` using `command :: exit N` records: **89 derived, 87 run with exit 0, 2 NOT MEASURED, 0 unrun.** - ⊘ NOT MEASURED: `pnpm check:dual-build-cjs-loads` and `pnpm check:type-check-debt`. Both exited 3, `PREREQUISITE NOT MET`: each reads every workspace package's built `dist/`, and this worktree built only the closures of the suites above. They are whole-tree families that CI runs after its full build. This diff changes no package's exports or build shape. - Among the 87: `check:adr-0087-registration --base origin/main` (it reads the changeset's `(narrowing)` arm and the `registered` marker), `check:changeset-no-major`, the `check:changeset-gate-self-tests`, `check:nul-bytes`, `check:doc-authoring`, `check:where-matcher`, and the spec families `check:api-surface`, `check:authorable-surface`, `check:docs`, `check:spec-changes`, `check:migration-registry`, `check:upgrade-guide`, `check:liveness` and `check:exported-any`. Local runs are not a complete account of CI: the artifact-roster families, the wide-population families, the path-scheduled jobs and the type-check lanes are CI's. --- _Generated by [Claude Code](https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
Refs #19886
Clause-②: no (narrowing)
Rewritten short by the
domain:spec#5seat (2026-09-24T07:02Z; round 4 after record5808690296). Stage 2c of #19886. Seat rulings:5806391955,5806943154,5807743099,5808108434. Dev reports on the card:5806886759,5807587023,5808082032,5808419047,5809358163.Some row-level or sharing CEL predicates no longer lower: those comparing a field with
==/!=against a list, whether a list literal or acurrent_usermembership array. Every consumer fails closed:driver-mongodb's
translateFilterrefuses$newith an array comparand (INVALID_FILTER/ 400).What changed
packages/formula/src/cel-to-filter.tsadds the refusal, covering list literals and resolved arrays, both orientations, and forms under!.packages/drivers/driver-mongodb/src/mongodb-filter.tsadds the$nearray refusal at any depth.packages/lint/src/validate-rls-predicate-enforceability.ts: the reference pass probes each kernelcurrent_userkey with its runtime type (scalar keys as scalars, membership keys as arrays).checkcarrying the shape is now dropped at compile (403 when no other policy applies), somatchesFilterCondition's 400 remains for a filter passed to it directly. Its ADR-0087 entry, its plugin-security pin and the docblock spans made false by this are corrected by cuts. So are three spans of its PENDING changeset: a finding: random changeset filenames collide silently across parallel agents — a round overwrote a sibling PR'sminorchangeset and every gate stayed green #17712 DELIBERATE CORRECTION, soCheck Changesetis red by design and landing waits on the maintainer's confirmation (see the gate comment).minorfor formula, driver-mongodb, plugin-security, plugin-sharing and lint, and atpatchfor spec, which carries the ADR-0087 entrycel-predicate-list-comparand-refused.Compile faces
This PR changes one compile face, driver-mongodb
translateFilter($newith an array → 400). The equality-slot array is left to the shared face (PR #19882). For the other faces, see PR #19946's table.Verification (the dev's, at
3bf405b501; round 4 re-measured at the merged head)mongodNOT MEASURED) and driver-sql, every probe fails closed: reads return no rows, and acheckgets 403 with nothing stored. That includes!(record.x == current_user.org_user_ids), which read every row on driver-mongodb before.in,not in, scalar==/!=,nulland{ $field }.Not in this PR
validate-rls-predicate-enforceabilityis silent on==/!=against a kernel membership key (current_user.org_user_ids…), a policy the runtime drops on every request once PR #19947 lands #19951: the lint is silent on==/!=against a membership key.translateFilterpasses a{ $field }cross-field reference through as a literal document, sorecord.s != record.tmatches every row (an RLSusingread widens) #19949: driver-mongodb{ $field }.🤖 Generated with Claude Code
https://claude.ai/code/session_01Sfe5YjBLwB9J3y8fvm2xq1