Repository navigation
fix(service-analytics)!: the analytics where door runs every arm of the shared comparand-shape face on the object spelling (#20010) - #20032
Conversation
…before the fix Measurement only; no source change. Taken on this branch at its base (origin/main 4463966) before any fix, with a throwaway harness that is not committed: a real sql.js engine (driver-sqlite-wasm) over rows d1 amt 1 'won', d2 amt 5 'lost', d3 amt 10 'open', d4 NULL NULL, d5 amt 3 ''. Faces: normalizeAnalyticsFilterTree; NativeSQLStrategy.execute (the statement that runs); ObjectQLStrategy.generateSql (the /analytics/sql echo); ObjectQLStrategy.execute (the ObjectQL engine path: a probe engine.aggregate records the condition it receives, and the engine seam's two spec faces are then run on it as the engine would); and evaluateAnalyticsQueryOverRows (the draft preview). Shared comparand-SHAPE face (assertListComparandShapes) ------------------------------------------------------ arm | object spelling | FilterArray spelling | ruling 1. $in null member {stage:{$in:['won',null]}} | tree `in ['won',null]`; native `stage IN (?, ?)` -> d1; echo the same; engine path: the engine receives it and its own seam refuses (aggregate('deal') prefix); preview d1,d4 | INVALID_FILTER/400 on every face (null list member) | 2026-08-31, #13357 record 5472662504 2. $nin null member {stage:{$nin:['won',null]}} | tree `notSet OR notIn ['won',null]`; native -> d4 only; engine seam refuses; preview d2,d3,d5 | 400 | 2026-08-31 3. null ordering comparand {amt:{$gt|$gte|$lt|$lte: null}} | tree `gt [null]`; native `amt > NULL` -> no row; engine seam refuses; preview: $gt/$gte no row, $lt/$lte d1,d2,d3,d5 | 400 (null ordering comparand) | 2026-09-01, #14080 record 5494521768 4. $between null bound {amt:{$between:[null,5]}} | lowered to `gte [null] AND lte [5]`; native no row; the engine receives {amt:{$gte:null,$lte:5}} and refuses it as a $gte null ordering comparand, naming an operator the author never wrote; preview no row | 400 (null range bound) | 2026-08-31 (#13495's shape) 5. $between blank bound {amt:{$between:['',5]}} | `gte [''] AND lte [5]`; native no row; the engine receives {amt:{$gte:'',$lte:5}} and ACCEPTS it; preview d1,d2,d5 | 400 (blank bound, MIN named) | 2026-09-20, #19071 record 5748839561 6. $between undefined bound | refused, #6386 wording `[analytics] comparand at "amt".$between[0] is undefined` | refused, the face's blank-bound wording | 2026-09-20 names undefined; verdict already a refusal, the wording differs 7. $between {$field} bound | refused, analytics wording | refused, the face's wording | 2026-08-11 (#7596), on the face since #19377; wording only 8. $between arity [1] / scalar 5 | refused, analytics wording `needs a two-element [min, max] array` | refused, the face's wording | #5869's rule, on the face since #9228; wording only 9. $in scalar {stage:{$in:'won'}} | laundered to `in ['won']`; native d1; the engine receives {$in:['won']} and accepts | 400 (requires an ARRAY) | #5869 / #9228 10. $nin scalar | `notSet OR notIn ['won']`; native d2,d3,d4,d5; engine accepts | 400 | #5869 / #9228 11. $in: null | `in [null]`; native no row; the engine refuses it as a null MEMBER (the laundered list), not as a non-list | 400 (requires an ARRAY) | #5869 / #9228 12. depth: {$or:[{id:'d3'},{stage:{$in:['won',null]}}]} native d1,d3; {$not:{amt:{$gt:null}}} native d4 | refused on the FilterArray spelling 13. nested relation {acct:{amt:{$gt:null}}} and dotted {'acct.amt':...} compile `acct.amt gt [null]` | ['acct.amt','>',null] refused 14. $ne with a list {stage:{$ne:['won','lost']}} | BOTH spellings compile `stage IS NULL OR stage != 'won'` ('lost' dropped): the face does not judge $ne today | waits on #19886 stage 2 (ruling A, 5805254639) Shared comparand-TYPE face (normalizeFilterComparandTypes) ---------------------------------------------------------- 15. undefined ($gt, $in member, implicit) | refused 400, #6386 wording | refused 400, the type face's wording | verdict already a refusal 16. plain object under $gt / $ne / $eq / a $between member | ACCEPTED: native binds the JSON text '{"a":1}' ($ne served every row); the engine seam refuses | 400 (the type face) | 2026-08-12, #7872 record 5265944890 ($eq has no FilterArray spelling: '=' lowers to the implicit slot, i.e. a nested relation) 17. plain object as a $in member or $contains | refused by both, the wording differs 18. binary (Uint8Array) under $eq or as a $in member, a Map under $eq | ACCEPTED (bound as JSON text); the engine seam refuses | 400 | #7872; binary is also this package's recorded predicate-level admission (#5234 / #8186) 19. bigint within 2^53 | kept as a bigint and bound; same rows | narrowed to its number; same rows 20. bigint beyond 2^53 | ACCEPTED | 400 | #7872 21. {$null: undefined} / {$null: {a:1}} | lowered to `set` (IS NOT NULL) | no FilterArray spelling (is_null ignores its value) Claude-Session: https://claude.ai/code/session_01Evb5jFDZGKQE9KG4jbMfMF Co-authored-by: Claude <noreply@anthropic.com>
…e to the shared comparand-shape face (#20010) Claude-Session: https://claude.ai/code/session_01Evb5jFDZGKQE9KG4jbMfMF Co-authored-by: Claude <noreply@anthropic.com>
… where, re-judge the pins it moves (#20010) Claude-Session: https://claude.ai/code/session_01Evb5jFDZGKQE9KG4jbMfMF Co-authored-by: Claude <noreply@anthropic.com>
…e per-arm remedy (#20010) Claude-Session: https://claude.ai/code/session_01Evb5jFDZGKQE9KG4jbMfMF Co-authored-by: Claude <noreply@anthropic.com>
…alytics-where-face-arms
…resolves (#20010) Claude-Session: https://claude.ai/code/session_01Evb5jFDZGKQE9KG4jbMfMF Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift Check6 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run. What this run could not see
Coarse fallback — 9 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 5219153ad0cffd3dd89a3ceccaf660e4c294ed43 && git checkout 5219153ad0cffd3dd89a3ceccaf660e4c294ed43
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 7766b62282f456ef04fd37304658b7ecd5a0fff9 33c73fdff48f034ec00cff0808c61839d8518717 && git checkout -B drift-repro 7766b62282f456ef04fd37304658b7ecd5a0fff9 && git merge --no-ff 33c73fdff48f034ec00cff0808c61839d8518717
node scripts/docs-audit/affected-docs.mjs --json 7766b62282f456ef04fd37304658b7ecd5a0fff9 |
…shared face's words (#20010) Claude-Session: https://claude.ai/code/session_01Evb5jFDZGKQE9KG4jbMfMF Co-authored-by: Claude <noreply@anthropic.com>
Resolves the #8186 comparand matrix against #20032 (#20010): the null row carries both re-judged cells, whereIn INVALID_FILTER/400 and scopeIn READ_SCOPE_COMPILE_FAILED/500, and the six-accepted-types check names each cell with the change that moved it. Claude-Session: https://claude.ai/code/session_01Evb5jFDZGKQE9KG4jbMfMF Co-authored-by: Claude <noreply@anthropic.com>
…s/sql echo refuse a read scope the shared comparand faces refuse (objectstack-ai#20046) Fixes objectstack-ai#20018 Clause-②: no (narrowing) ## What changed `compileScopedFilterToSql` (`packages/services/service-analytics/src/read-scope-sql.ts`) is the read-scope lowering behind two analytics faces: - the NativeSQL execute face, `NativeSQLStrategy.applyReadScope`, for the base table and every joined hop; - the `/analytics/sql` echo, `ObjectQLStrategy.generateSql`. It is also a public export of the package. Once its own lowering returns, it now calls `assertReadScopeComparandsRunnable` on the scope. That is the helper PR objectstack-ai#20017 added in the same module, and it runs `@objectstack/spec/data`'s `assertListComparandShapes` and `normalizeFilterComparandTypes` on the scope alone. A scope those faces refuse now gets `READ_SCOPE_COMPILE_FAILED` / 500 with the message withheld (the objectstack-ai#5367 ruling, re-affirmed as objectstack-ai#7598 Q2 = A) on the native face and the echo. The refusal comes before any statement is built or executed. That is the answer the ObjectQL execute face has given since PR objectstack-ai#20017, so one read scope gets one verdict on all three analytics faces. - **Files:** the call itself is one line. The rest of the diff is: - the module-header section (objectstack-ai#20018); - a note on the helper's docblock; - a one-paragraph note on the binary extra in `comparand-shape.ts`, whose "accepted in every bind position" is no longer true of the read-scope door; - tests and the changeset. - **Not touched:** `objectql-strategy.ts` and `native-sql-strategy.ts`. Both faces reach the guard through the compiler, so neither needs a call site of its own. - **Also not touched:** `filter-normalizer.ts`, `preview-evaluator.ts` and `packages/spec`. ## Measurement, recorded before the fix Everything in this section was measured on `9d81af714f` (`origin/main`, pre-fix). The rows are **executed**, not read from the compiled string. - **Harness:** a scratch probe that is not committed, plus measurement commit `8c80d9c3d2` (the new test file alone). - **Database:** one real `SqliteWasmDriver` with four fixture rows. `region` is NULL on d3, and `owner` and `amount` are NULL on d4. - **ObjectQL face:** a real `ObjectQL` engine. - **Echo:** its SQL was run on the same database. - **Native:** `NativeSQLStrategy.execute` through `executeRawSql` on the same database. It was not stubbed. - **Scopes:** the `getReadScope` contract, filled by hand. | read-scope shape class | ObjectQL execute | echo (SQL executed) | native execute | |:--|:--|:--|:--| | plain-object comparand under `$eq` *(the card's first shape)* | `READ_SCOPE_COMPILE_FAILED` / 500 | compiles; the database refuses the bind | `DATABASE_ERROR` / 500 | | null member in `$in`, `['emea', null]` *(the card's second shape)* | 500 | d1 | d1: the NULL member matches nothing | | null-only `$in` | 500 | no rows | no rows | | null member in `$in` under `$not` | 500 | d3 | d3: **only** the NULL row, which the scope names as excluded | | null member in `$nin` | 500 | d3 | d3: **only** the NULL row, which the scope names as excluded | | null comparand under `$gt` / `$lte` | 500 | no rows | no rows | | null `$between` bound | 500 | no rows | no rows | | blank `$between` bound | 500 | d1 d2 d4 | d1 d2 d4 | | plain-object comparand under `$ne` / `$gt`; empty object under `$eq` | 500 | compiles; DB refuses | `DATABASE_ERROR` / 500 | | bigint beyond 2^53 (implicit, `$in`) | 500 | no rows | no rows | | binary comparand, binary `$in` member | 500 | no rows | no rows | | `Map` or function comparand | 500 | compiles; DB refuses | `DATABASE_ERROR` / 500 | | controls (9 well-formed scopes, including the null predicates and the live RLS composite) | the same rows on all three faces | | | The card named two shapes. The measurement found the gap is every shape the two shared faces refuse and this compiler's own gates did not: null list members and bounds, null ordering comparands, blank bounds, oversized bigints, binary, and non-scalar objects in a scalar position. That is the set that moves. ### The mechanism hypotheses - **Hypothesis 1 is confirmed, and the set is wider than the card's two shapes.** These are the compile arms that accepted them: - ``case '$eq': return val === null ? … : `${col} = ${bind(params, val)}`;``. A plain object is bound, and no gate judges a `$eq` comparand's type. `assertNoFieldReferenceComparand` steps past an object that is not `{ $field: string }`. - `case '$in'` → `assertCompilableMembers(op, field, val)` → `isBindableComparand(member)`, which admits `null` (an accepted comparand type) and binary (a package-local extra), then `IN (…)` binds each member. - The ordering arms and `$between` bind whatever passes those gates. The implicit-equality arm binds any non-object. - **Hypothesis 2: measured.** See the table above, and the producers section below. - **Hypothesis 3 is confirmed in verdict, but the placement is better than "at the entry".** The two walks are pure functions of the scope, so the placement cannot change which scopes are refused, only which log sentence a doubly-refused scope carries. - Ablation A2 below put the call at the entry and turned 37 tests red in 7 files: the new ordering pin, plus 36 existing log-sentence pins, for example `read-scope-eq-array-refusal` (15) and `read-scope-undefined-comparand` (8). - After the lowering, every shape this compiler already refused keeps its own sentence (the objectstack-ai#13926 ordering), and the faces add only what would otherwise have been lowered. ### Producers: who can emit these shapes today All readings below are on `9d81af714f`. | producer | emits a refused shape? | evidence | |:--|:--|:--| | RLS `using` predicates, through `compileCelToFilter` → `RLSCompiler.compileFilter` | **yes, from an authored predicate.** `f in ['a', null]`, `f in [null]`, `!(f in ['a', null])`, `f > null`, `f <= null` and `f != current_user` each lower verbatim into a refused shape. | Scratch probe. All six pass `isSupportedRlsExpression`, and `validateRlsPredicateEnforceability` returns 0 findings for each. | | authored policies in this repository | **none** | `git grep` of `using` / `check` predicates for a null list member or a null ordering comparand, over `examples/**` and `packages/**/*.ts` (tests excluded): 0 matches, exit 1. The control on the same tree and file set, predicates naming `current_user`, matched 77 lines in 7 files, exit 0. | | a resolved membership variable with a null member | no | The objectstack-ai#13496 guard. Measured: `f in current_user.teams` with a null member lowers to the deny sentinel. | | `plugin-sharing` `buildReadFilter` | no | Owner ids are `String(userId)` or a resolver's `string[]`. `grantedRecordIds` filters out `null` and `''`. | | a host `getReadScope` option, or a direct caller of the export | anything | The door the card names. | Verdict: **no producer both legitimately authors one of these shapes and relies on the native answer.** Three facts support that: - The in-repo producer emits these shapes only from predicates that the standing rulings, carried by the shared faces, refuse. No policy in this repository is one of them. - The ObjectQL analytics face already refused every such scope. - The native answer was not the predicate's meaning. It dropped the null member, admitted only the NULL rows the scope excludes, returned zero rows, or hit a database error. So this is execution under the rulings, not a `needs_decision`. The authoring half is reported separately as a finding; see the Acceptance notes. ## Tests The new file is `packages/services/service-analytics/src/__tests__/read-scope-comparand-three-faces.test.ts`, with 32 cases. It uses one `SqliteWasmDriver`, a real `ObjectQL` behind the ObjectQL face, and the echo's SQL executed. - **13 refusal classes.** Each asserts on all three faces: `code` `READ_SCOPE_COMPILE_FAILED`, `status` 500, and the prose withheld. "Withheld" means `serverFaultProvenance(resolveThrownHttpError(err, 500))` is `'declared'` and `declaredRefusalMessage(err)` is undefined. - **The native face refuses before any statement reaches the database.** Zero `executeRawSql` calls across all 13 classes. - **The joined hop.** `applyReadScope`'s per-hop lowering refuses a joined object's scope, named for that hop. - **The public export** refuses every class, and a well-formed scope compiles to the same bound predicate as before. - **Log sentences.** A shape the compiler already refused, a list under `$eq`, keeps its own sentence. A shape only the faces refuse carries their sentence, the same on all three faces. - **Controls:** - with no scope, every face serves all four rows; - 9 well-formed scopes admit the same rows on every face. They include `{ region: null }`, `$ne: null`, a non-empty `$nin`, the live RLS composite with an emptied `$in` beside an own-rows grant, and the spelling the null-member ruling prescribes (`$or` of `$in` and `$null: true`); - a well-formed caller `where` composes with a well-formed scope; - a caller `where` in a refused scope shape answers exactly as it does with no scope, and is never attributed to the read scope. **Existing pins that asserted the lowering binds a refused shape.** Each is re-judged with a `[objectstack-ai#20018]` note, and each keeps what it was there to pin: - `comparand-door-single-source.test.ts`. Three matrix cells now read `READ_SCOPE_COMPILE_FAILED/500`: `null` under `scopeIn`, `binary` under `scopeIn` and `scopeEq`, and `plain object` under `scopeEq`. This PR moves no `where`-door cell. - After merging objectstack-ai#20032 (objectstack-ai#20010), the `null` row carries both re-judged cells: `whereIn` is `INVALID_FILTER/400` (objectstack-ai#20010) and `scopeIn` is `READ_SCOPE_COMPILE_FAILED/500` (this PR). - The "six accepted types, every position" check names each moved cell with the change that moved it, and holds every other cell at `accept`. - `comparand-shape-refusal.test.ts`. "Keeps binding every legitimate `$in` member" drops `null`, and a new case pins `null` refused with the face's sentence. - `cross-field-reference-refusal.test.ts`. `{ $gt: { $field: 5 } }` is refused as a plain object, with the type face's sentence and not the field-reference gate's. - `read-scope-boolean-flag-comparand.test.ts`. An inherited `$null` still cannot trip the flag gate, because the refusal carries no flag sentence. The object is refused by the type face as a non-plain value. - `read-scope-undefined-comparand.test.ts`. `$in: [null]`, `$nin: [null]` and `$between: [null, 5]` leave the null control group for their own "refused by the shared face" block. Every null predicate stays in the group, unmoved. **Results:** | run | tree | result | |:--|:--|:--| | new file, measurement commit (test only) | `8c80d9c3d2` | `Tests 17 failed \| 15 passed (32)`. Every refusal class and the three pins built on them are red; every control is green. The first face to fail in each row is the echo; the ObjectQL face passed first. | | whole package | `0fbed27877` (final; `origin/main` `adbbc5d01e` merged) | `Test Files 120 passed (120)`, `Tests 2689 passed (2689)` | | `pnpm --filter @objectstack/service-analytics typecheck` | `0fbed27877` | exit 0; `tsc --noEmit --listFiles` includes the new test file | **Ablations.** Both ran through `node scripts/ablation-replace.mjs` in WRAP mode (the anchor must hit exactly once, the blob must change, and the tool's own trap restores). The subject resolves to `src/` through relative imports, so no `dist/` leg applies. Each restore was proven: blob `34705e267dba` equals `HEAD`, and `git diff HEAD` is empty. A1 was re-run on the final head `0fbed27877`. A2 ran on `f293340e94`. `read-scope-sql.ts` is the same blob, `34705e267dba`, on both trees (neither merge touched it), so A2's placement result stands. | leg | mutation | result (whole package) | |:--|:--|:--| | A1 (at `0fbed27877`) | delete the new `assertReadScopeComparandsRunnable(filter, alias);` call | `26 failed \| 2663 passed (2689)`: all 17 negative pins in the new file, plus the 9 re-judged pins (3, 1, 1, 1 and 3 across the five files). Every control stayed green. The same 17 + 9 went red at `f293340e94` (`26 failed \| 2567 passed (2593)`). | | A2 (at `f293340e94`) | call it BEFORE `compileNode` instead of after | `37 failed \| 2556 passed (2593)`: the new ordering pin, plus 36 existing log-sentence and precedence pins in 6 files | The first attempt at A2 was a void run. Its replacement text contained its anchor, so the tool refused with "anchor count moved 1 -> 1" and restored. Nothing was measured. It was re-run with a three-line anchor, and that is the result above. ## Gates **Derived gates.** Derived at `0fbed27877` with `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands`: 61 families, not stale. They are a superset of the 48-line dispatch-time list. - **59 exited 0.** - **`check:dual-build-cjs-loads` and `check:type-check-debt`** first answered `PREREQUISITE NOT MET` (exit 3), because the workspace had no `dist/`. After building every `./packages/**` workspace package (`VERDICT command-exit 0`), both exited 0: `check:dual-build-cjs-loads` passes its floors, and `check:type-check-debt` re-measured 4 ledger entries (53 raw tsc errors) with none above its recorded number. - **The `--ran` reconciliation:** 61 derived, 61 run, 0 NOT-MEASURED, 0 UNRUN. That zero is derived, not claimed: all 61 records carry an exit code and none is 3. **Other checks:** - `GITHUB_TOKEN=… node scripts/check-issue-citations.mjs`: exit 0; 8 citations judged, 8 resolve. - `node scripts/check-adr-0087-registration.mjs --base origin/main` (one of the derived families, at `0fbed27877`): exit 0. The changeset is `BREAKING+bang+clause-②-narrowing`, disposition `not-required (no-migration-prescription)`. - **Lint, narrowed to the 8 touched TypeScript files** with `eslint --no-inline-config --format json` at `0fbed27877`: 8 files read, 0 errors, 0 warnings, none ignored. - The changeset is outside eslint's population ("no matching configuration"). - `eslint.config.mjs` enables no type-aware linting: its `parserOptions` carry only `ecmaVersion` and `sourceType`, with no `project` and no `projectService`. So this diff cannot move a verdict on an untouched file. - `pnpm lint` itself is CI's. ## Acceptance notes - **BREAKING, `minor` with `!`.** Read scopes the native face and the echo used to serve are refused now; the table above lists what they served. The changeset carries the banner and the fix spellings: the null predicate beside a membership, `$gte` / `$lte` for a one-sided range, and a scalar comparand. - **The refusal set is the shared faces' set, not only the card's two shapes.** Judging the scope with the same function the ObjectQL face uses is what makes the verdicts equal. A narrower hand-picked subset would have left the other rows of the table as "one scope, two answers". - **Binary.** The package-local binary bindable (`isBindableComparand`) no longer reaches the read-scope door. The shared type face refuses binary, and the ObjectQL face already did. The `where` door and the predicate itself are unchanged. - **The joined-hop log sentence names the alias.** `compileScopedFilterToSql` knows only the alias, so the operator's log reads `read scope for "ALIAS"`: the object name on the base table, the join alias on a hop. The response withholds it either way. - **Precedence on the native face.** When a scope carries both a shape this compiler refuses and one only the faces refuse, the compiler's sentence answers. The verdict is the same either way. - **The CRUD path is not an analytics face and was not measured here.** The security middleware composes the same RLS filter into the engine's `where` after the engine's shared-face seam has run. What `driver-sql` answers for these shapes there is outside this card. - **Finding, class (c), not fixed here, for the seat to file.** The RLS authoring surface admits predicates that lower into scope shapes the shared faces refuse: a literal `null` in an `in` list, an ordering comparison against `null`, or a comparison against the whole `current_user` object. All three pass `isSupportedRlsExpression`, and `validateRlsPredicateEnforceability` returns no finding for them. They are refused (500) on every analytics face after this PR. The evidence is in the producers table. - **`origin/main` was merged in twice.** - At `b3735968ba`: three commits in `packages/objectql` and `packages/plugins/plugin-security`, sharing no path with this diff. - At `adbbc5d01e`: objectstack-ai#20032 (objectstack-ai#20010) and a driver-sql / driver-turso fix. That merge had one content conflict, in the comparand matrix, resolved as above. - `comparand-shape-refusal.test.ts` and `cross-field-reference-refusal.test.ts` auto-merged and were re-read on the merged tree. objectstack-ai#20032's edits there are to the `where`-door pins and this PR's are to the read-scope pins, and their notes agree: the null member is refused by the same ruling at each door, each in its own envelope. One sentence in the non-string `$field` case ("it binds as JSON") is now scoped to the `where` door, where it still holds. - The dependency closure was rebuilt before the final runs. - **Files not touched:** `filter-normalizer.ts`, `preview-evaluator.ts`, `objectql-strategy.ts`, `native-sql-strategy.ts`, `packages/spec`. - **objectstack-ai#19995** stays open behind objectstack-ai#20020, for its engine- and driver-door residue. This PR does not address it. --- _Generated by [Claude Code](https://claude.ai/code/session_01Evb5jFDZGKQE9KG4jbMfMF)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
Part of #20010: this PR carries the shared comparand-SHAPE face's arms. #20010 remains open for two arms:
$newith a list, which waits on #19886 stage 2, and the comparand-TYPE face, which the card has answered and #20035 carries.Clause-②: no (narrowing)
What this changes
The analytics
wheredoor (lowerAnalyticsWhereinpackages/services/service-analytics/src/strategies/filter-normalizer.ts) now hands every field entry of an object-form condition to the shared comparand-shape face,assertListComparandShapes(@objectstack/spec/data). Before, it ran only the equality arm (PR #20008). Each shape below is now refused withINVALID_FILTER/ 400, carrying the face's own message, path and prescription. TheFilterArrayspelling of the same condition gets the same bytes, because it met the face insideparseFilterASTall along.The face's docblock records the rulings that put these positions on it:
$in:[null]/$nin:[null]disagree while$null/$ne:nullagree #13357): "Anullmember of$in/$nin, and anull$betweenendpoint (driver-memory's reference matcher answers{$between: [null, null]}with EVERY valued row — the range arm's two comparisons are both false against a null bound, so a bounded range stops bounding #13495's shape), are refused at this door".$gt/$gte/$lt/$ltewith anullCOMPARAND is the one null-comparand position the filter contract still accepts — and the two driver-memory faces answer it differently #14080): "Anullcomparand of$gt/$gte/$lt/$lte… refused at this door, same envelope, so the divergent cells are constructively unreachable".$betweenendpoint that the published endpoint contract now refuses —parseFilterASTdisagrees withRANGE_ENDPOINT_DESCRIPTION#19071): "''andundefinedare refused, naming the blank side (MIN / MAX and the index)".not_in/in的比较值是标量(非数组)时答 500 DATABASE_ERROR,而不是带信封的 400 —— 而这个形状是 spec 合法的 ViewFilterRule #5869, moved to the face by The #5869 list-comparand rule is enforced only at the engine seam: a scalarin/nincomparand still reaches a driver, and on mingo >= 7.2.3 it escapes as a rawTypeError#9228):$in/$nin/$betweenreceive a list at all.How:
assertWhereComparandShapesmakes two passes over one shared traversal,forEachWhereFieldEntry. The traversal walks$and/$or/$notand field entries, the way the face walks them, and also descends a nested relation, because this compiler flattens one to its dotted member.IN, while ruling 乙 refuses the same shape at the shared face "for every driver at once" #19888 equality pass, unchanged, so a list in the equality slot is still diagnosed first.where. The face therefore reports exactly the path and field it reports on the whole condition.preview-evaluator.ts) calls the same exported gate, so a drafted chart refuses what the published chart refuses.Measured first (recorded on this branch as
a508dddc2c, before any source change)Base
origin/main44639665ee. The harness was a real sql.js engine (driver-sqlite-wasm) over rows d1 (amt 1, 'won'), d2 (5, 'lost'), d3 (10, 'open'), d4 (NULL, NULL) and d5 (3, ''). It ran five faces:normalizeAnalyticsFilterTree;/analytics/sqlecho;engine.aggregaterecords the condition it receives and the engine seam's two spec faces are then run on it;FilterArrayspelling$inmember{stage:{$in:['won',null]}}in ['won',null]; nativestage IN (?, ?): d1; the engine seam refused it; the preview served d1, d4$ninmembernotSet OR notIn ['won',null]; native: d4 only; the preview served d2, d3, d5$gt/$gte/$lt/$lteamt > NULL: no row; the preview served d1, d2, d3, d5 for$lt/$lte$betweenendpoint[null, 5]amt >= NULL AND amt <= 5: no row; the engine received{amt:{$gte:null,$lte:5}}and refused it as a$gte, an operator the author never wrote$betweenendpoint['', 5]amt >= '' AND amt <= 5; the engine path ACCEPTED{$gte:''}; the preview served d1, d2, d5$in'won'in ['won']: native d1; the engine accepted it as a list$nin'won'$in: nullin [null]; the engine refused it as a null MEMBER (after the laundering)$betweenendpointundefined; wording only{ $field }$betweenendpoint$between{acct:{amt:{$gt:null}}}, dotted{'acct.amt':…}acct.amt gt [null]$newith a list{stage:{$ne:['won','lost']}}stage IS NULL OR stage != 'won';'lost'is dropped$neyetAfter the change, re-measured at
142b554f0e, every row above except$neis refused on all five faces. The object message equals theFilterArraymessage byte for byte, for every arm that has aFilterArrayspelling. The one exception is the nested-relation spelling: the face names the leaf key ("amt"atwhere.acct.amt.$gt), which is the convention #19888 set, while the dotted spelling names"acct.amt". The$nerow is unchanged on both spellings.Stored filters in this repository. A text scan covered 4013 non-test tracked files under
examples/,packages/,content/,skills/,apps/anddocs/. Its detectors were: anullmember of a$in/$ninlist; anullordering comparand; anull/ blank /undefined$betweenendpoint; and a scalar$in/$nin. Positive controls: a synthetic fixture hit on all four detectors; C1 = 451$in/$ninlists, C2 = 495 ordering operators and C3 = 61$betweenlists were counted. Every hit is a false positive: a comment, an operator-name map, a doc line or a CHANGELOG line. No authored filter carries a moved shape. Limits: this is a text scan, not an AST scan, so a comparand held in a variable is invisible to it. Deployedsys_metadatarows: NOT MEASURED.Compile surfaces (the face's non-equality arms)
lowerAnalyticsWhere/normalizeAnalyticsFilterTree: callerwhere, dataset scopefilter, measurefilter; native execute,/analytics/sqlecho, ObjectQL engine pathengine.aggregatecall. Measured over sql.js before and after; pinned per face.evaluateAnalyticsQueryOverRows/matchesWhere(draft preview)assertListComparandShapes(the spec's shared face)parseFilterAST(this door'sFilterArrayspelling)FilterArrayspelling was refused.lowerWhereFilterArray,engine.ts)$in: ['won'],$gte: ''). That is no longer reachable from this door.compileScopedFilterToSql(service-analytics read scope)read-scope-sql.tswas held by PR #20017 during this dispatch. Not re-measured here.applyFilterCondition(driver-sql),checkCondition(driver-memory),buildWhereSQL(driver-turso),matchesFilterCondition(formula),translateFieldOperators(driver-mongodb)parseFilterASTor the engine seam, which run the face (the rulings' own scope). Not re-measured here.applyHaving/matchesHaving(objectql HAVING)Tests and evidence (head
33c73fdff4,origin/main26550c6603merged in)Round 2 (
33c73fdff4): a consumer pin inpackages/rest. CI was red ona1bdbf1d59(Test Core 3/6):src/analytics-filter-refusal-envelope.test.tsasserted the old analytics sentence for a one-bound$between(/needs a two-element \[min, max\] array/). The verdict was already right; only the message had moved. That one case is re-judged, test-only. Its verdict assertions stay (400INVALID_FILTERthrough the real dataset route, which is analytics 的 filter 拒收到不了调用方:service 侧多数拒收没有 ADR-0112 信封,REST 面又用 message 正则嗅探,一律答 500 #5352's point), and the message assertion now matches the face's sentence, with a note citing the face's rule (数据 API:集合算子not_in/in的比较值是标量(非数组)时答 500 DATABASE_ERROR,而不是带信封的 400 —— 而这个形状是 spec 合法的 ViewFilterRule #5869, moved to the face by The #5869 list-comparand rule is enforced only at the engine seam: a scalarin/nincomparand still reaches a driver, and on mingo >= 7.2.3 it escapes as a rawTypeError#9228).packages/**andexamples/**(4014 files). It searched for the analytics "two-element" sentence, the [spec]service-analytics' read-scope / Cube filter compilers still refuse$field, so a CEL field-to-field RLS rule 400s on those faces #7598{ $field }endpoint sentence ("may not be a field reference", "has the field reference"), the service-analytics 的where门把undefined值的键整个丢掉 —— 单键 where 退化成「无过滤器」,方向是加宽(#6125 五面表漏记的第六、七种读法) #6386undefinedsentence at a$betweenposition, and the[analytics] "$between"prefix. It then checked, one by one, every test in the 100 files outside this package that reachservice-analytics, for$between/ null / scalar-$inshapes. One hit: thispackages/restcase. Every other "two-element" hit is a date-range arity message, a search-term arity or a spec test. The remaining$betweenhits are drivers' own refusals or the read-scope door, none of which this PR moves.@objectstack/restsuite (built against this branch'sservice-analyticsdist/): 194 files, 3265 passed and 1 skipped.VERDICT command-exit 0. The touched file alone: 31 passed.@objectstack/service-analyticsthroughdist/, so it was run as a dist ablation. The all-arms pass was removed throughscripts/ablation-replace.mjs, the package was rebuilt, andablation-dist-preflightput the marker in 2 built files. Predicted 1; measured 1 red / 30 green:expected '[analytics] "$between" on "amount" ne…' to match /Operator "\$between" on field "amount…/. Restore: source blob == HEAD, rebuilt,--absentread the marker absent from all 6 built files, whole tree clean.pnpm --filter @objectstack/rest typecheckexits 0, but that package'stsconfigdoes not compile its test files (0 in--listFiles), so it measures nothing about this one.eslint --no-inline-configon the file: 0 errors, 0 warnings.New
src/__tests__/where-face-arms-refusal.test.ts, 101 tests. Every refusal assertscodeandstatus, plus the face's opening sentence.{ $field }shape; under$and,$or,$notand$notover$or; beside a legal operator; inside a nested relation; on a dotted member.FilterArrayspelling, which equals the face's own message.where门把undefined值的键整个丢掉 —— 单键 where 退化成「无过滤器」,方向是加宽(#6125 五面表漏记的第六、七种读法) #6386 undefined, service-analytics 的where门:字段约束里$算子与非$键混写时,非$兄弟键被静默丢掉(方向是加宽;与值无关,不是 #6386 的undefined) #6444 mixed wrapper). CONTROL: anundefinedoutside a$betweenendpoint keeps service-analytics 的where门把undefined值的键整个丢掉 —— 单键 where 退化成「无过滤器」,方向是加宽(#6125 五面表漏记的第六、七种读法) #6386's sentence.$neheld as PARITY with the face (2). The test does not pin the [finding]$newith an array comparand splits across backends: driver-sql and driver-memory refuse (400), driver-mongodb answers, formula matches every row — and both shared faces pass it #19886 defect in either direction.$in: [],$nin: [], falsy members, scalar ordering comparands,{ $field }in an ordering slot, the null predicate spellings,$contains: null, a nested scalar, and four legal$betweenranges, including a whitespace-only endpoint, which the face does not judge.engine.aggregatecall.Pins re-judged with a note, each quoting the ruling that moves it:
comparand-door-single-source.test.ts: thenullrow'swhereIncell goes fromaccepttoINVALID_FILTER/400(2026-08-31). Its six-types assertion now names that one exception.filter-normalizer-undefined-comparand.test.ts: fournull-control rows leave the group:$gt: null(analytics 的string[]值往返对字符串比较数也有损:{code: {$eq: '007'}}绑成数字7、'null'绑成真 NULL、'true'绑成1—— 文本列静默取到错行 #5526's by-construction reading, which the face records was a position "no ruling covers" until 2026-09-01),$in: [null],$nin: [null]and$between: [null, 5]. The$between: [undefined, 5]row moves to the new file (2026-09-20 namesundefined).filter-value-type-fidelity.test.ts:$gt: nullgoes from "binds NULL" to refused before anything binds. The case's own concern, that no''is bound, still holds.comparand-shape-refusal.test.ts: the$inmember list drops itsnull, and the null-carrying list is asserted refused in the face's words.cross-field-reference-refusal.test.ts(2),filter-operator-coverage.test.ts(1) andfilter-refusal-envelope.test.ts(1): a{ $field }endpoint and a one-bound$betweenkeep their verdict and now carry the face's wording.Package run (
pnpm --filter @objectstack/service-analytics test, ata1bdbf1d59; noservice-analyticsfile changed since): 119 files, 2656 tests passed.typecheckexited 0, andtsc --listFilesincludes all ten touched files. Base (before the merge oforigin/main): 117 files, 2541 tests.Ablations. Each ran from the committed fix through
scripts/ablation-replace.mjs(the anchor must hit, the mutation is verified on disk, the restore is proven by the blob matching HEAD and an emptygit diff HEAD), with the red count predicted before running. The tests import the source by relative path, so no build is on the path.native execute: expected a refusal, got rows: expected [ 'd1' ] to be undefined;draft preview: expected a refusal, got rows: expected [ 'd1', 'd4' ] to be undefined;expected 'accept' to be 'INVALID_FILTER/400'.IN, while ruling 乙 refuses the same shape at the shared face "for every driver at once" #19888's 4 and 2, since the one call now carries both. Sample:draft preview: expected a refusal, got rows: expected [ 'd1', 'd2', 'd3', 'd4' ] to be undefined.Gates.
dispatch-gates --repo objectstack-ai/objectstack --commandsderived 61 at33c73fdff4: thepackages/restedit addscheck:dispatcher-error-vocabulary, which round 1 ran from the dispatch list. The union was re-run on this head.check:dual-build-cjs-loadsandcheck:type-check-debtexited 3 (PREREQUISITE NOT MET: they need the whole workspace built, which CI does).--ran: 61 derived, 59 run, 2 NOT-MEASURED, 0 UNRUN.check-adr-0087-registration --base origin/main(one declared-breaking changeset,not-required (already-registered)),check-changeset-no-major,check:changeset-gate-self-tests,check:nul-bytes,check:where-matcher,check:test-source-alias, andcheck-issue-citationsin its board-probing mode (32 citations, all of which resolve).Lint, narrowed to the change.
eslint --no-inline-config --format jsonover the 10 touched TypeScript files: 10 files, 0 errors, 0 warnings.eslint --print-configresolves a config for each.eslint.config.mjsnever enables type-aware linting (its note near line 327), so this diff cannot move the verdict on an untouched file.Deviations from the dispatch, stated
$gt/$ne/$eq, a$betweenendpoint that is a plain object, a binary orMapcomparand, and a bigint beyond 2^53 are all accepted on the native path, bound as JSON text or as-is.{stage:{$ne:{a:1}}}served every row. TheFilterArrayspelling and the engine seam refuse each of them.undefinedis not one of these: it was already refused on both spellings (the wording differs).parseFilterASTand the engine seam run it: 48 pins red, 35 more than the shape face, across nine files.where门把undefined值的键整个丢掉 —— 单键 where 退化成「无过滤器」,方向是加宽(#6125 五面表漏记的第六、七种读法) #6386's undefined prescription, driver-sql:两类无意义比较对象仍编译成「静默空谓词」——$in/$nin的非$field对象成员,与 LIKE 族的对象比较值(String 成[object Object]) #5234's unbindable$inmember and LIKE-family sentences, service-analytics 的where门:字段约束里$算子与非$键混写时,非$兄弟键被静默丢掉(方向是加宽;与值无关,不是 #6386 的undefined) #6444's order, and [spec]service-analytics' read-scope / Cube filter compilers still refuse$field, so a CEL field-to-field RLS rule 400s on those faces #7598 / [finding]$icontainsis absent from analyticsTEXT_PATTERN_OPERATORS, so the #5234 comparand fence never covered it on thewheredoor — one operator, two answers inside one package #7693.{ $eq: {…} }account that driver-sql:两类无意义比较对象仍编译成「静默空谓词」——$in/$nin的非$field对象成员,与 LIKE 族的对象比较值(String 成[object Object]) #5234 "left open" and that [spec]service-analytics' read-scope / Cube filter compilers still refuse$field, so a CEL field-to-field RLS rule 400s on those faces #7598 made load-bearing, and [finding] service-analytics carries its own copies of the comparand-type allow-list the #7872 door now single-sources — reconcile membership and message wording to the door #8186's recorded binary admission.BigIntcrashes with a raw mingoTypeError, and four other types silently answer zero rows #7872 ruling covers them. The stop is taken on the role file's two-architectures clause, not on Section 1's ruling clause.whereskips the shared comparand-TYPE face, so a plain-object / Map / oversized-bigint comparand is bound as JSON text on the native path while the FilterArray spelling and the engine refuse 400 #20035 carries it. Nothing of it is started here.filter-normalizer.ts, the matrix'swhere*cells, the named pin files, new tests and the changeset. This PR also touches:src/preview-evaluator.ts: one import, one call and comments. This is a bounded in-place fix, and all four conditions hold. It is the same defect class. It is a mechanical call of the same exported gate, whose shape the face pins. No open PR touches the file: fix(service-analytics): the ObjectQL execute face refuses an unrunnable read scope in the withheld READ_SCOPE_COMPILE_FAILED / 500 envelope #20017, the only other PR in this package, landed before this branch mergedorigin/main. It is the same gate family, with no new verification surface. Without it, this change would have created a publish-refuses / preview-charts split for every moved arm. The claim's file surface needs this path added.comparand-shape-refusal.test.ts,cross-field-reference-refusal.test.ts,filter-operator-coverage.test.tsandfilter-refusal-envelope.test.ts: existing pins on a moved arm or its wording, re-judged with notes.packages/rest/src/analytics-filter-refusal-envelope.test.ts(thedomain:clilane), one case, test-only. The seat authorized it as this claim's own follow-through, and the claim's file surface is revised on the card. The census found no other consumer pin.not-required (already-registered filter-between-blank-endpoint-refused). The blank endpoint is the one moved arm whose transition is on the ledger. The null-member, null-endpoint and null-ordering transitions were ruled at the face with no ledger entry: their changesets declaredno-migration-prescription. This changeset carries the FROM → TO table the dispatch asked for, sono-migration-prescriptionwould be refused by the gate. The marker's own text states each arm's ledger status.origin/mainmerged in (da44a2be39): PR fix(service-analytics): the ObjectQL execute face refuses an unrunnable read scope in the withheld READ_SCOPE_COMPILE_FAILED / 500 envelope #20017 landed in this package during the run. After a reinstall and a rebuild, the package suite, the typecheck and the gate union were re-run on the final head.service-analyticssuite and so did not see thepackages/restpin on the re-worded one-bound$betweensentence; CI did. Round 2 fixed that case, ran the census, and ran the full@objectstack/restsuite.Acceptance notes
$newith a list is carried the day [finding]$newith an array comparand splits across backends: driver-sql and driver-memory refuse (400), driver-mongodb answers, formula matches every row — and both shared faces pass it #19886 stage 2 puts its refusal on the face, with no change here. The parity test holds this door to whatever the face answers.$between, a{ $field }endpoint and anundefinedendpoint.fieldLeaves' own checks for them stay as that function's invariants, annotated, and are no longer reached from the door.IN, while ruling 乙 refuses the same shape at the shared face "for every driver at once" #19888: the leaf key in the message, the full path inat.undefinedcomparand (not this card's arm; reported).evaluateAnalyticsQueryOverRowsanswers no row for{stage: undefined}/{amt: {$gt: undefined}}, while every published face refuses it 400 (service-analytics 的where门把undefined值的键整个丢掉 —— 单键 where 退化成「无过滤器」,方向是加宽(#6125 五面表漏记的第六、七种读法) #6386). The preview never ran service-analytics 的where门把undefined值的键整个丢掉 —— 单键 where 退化成「无过滤器」,方向是加宽(#6125 五面表漏记的第六、七种读法) #6386's gate. This is reachable only in-process, becauseundefinedcannot cross JSON.