Skip to content

Commit cfc3bcf

Browse files
fix(spec)!: a filter carrying a comparand the query faces refuse is refused when it is saved (#20116) (#20247)
Part of #20116 Clause-②: no ## Summary `FilterConditionSchema`, the save door behind every stored filter, now refuses every comparand slot the query faces refuse. The dataset `filter` and measure `filter` also refuse those slots inside a nested-relation condition, through #20207's walk. This closes every member the collector lists. The remainder named below is new members this run found, so this PR says `Part of`, not `Fixes`. The judge is the shared comparand-shape face itself (`assertListComparandShapes`), called read-only per slot. So the save door refuses exactly what the face refuses on every query, and passes what it passes. The two boolean flags, which that face does not judge, are refused on the predicate every flag face uses: the comparand is not a boolean. ## Measured before (`origin/main` `af32cf9a`) For every member, `FilterConditionSchema`, `DatasetSchema.filter`, a dataset measure `filter`, a dashboard widget `filter` and a report `runtimeFilter` all answered `success: true`. The query faces answered as below. Probe run from the worktree; "face" is `assertListComparandShapes`, "analytics" is `normalizeWhereComparands`. | member | face (top / `$and`) | face (nested) | analytics (top / `$and` / nested) | |:--|:--|:--|:--| | `{ stage: { $null: 'x' } }`, `$exists: 'false'`, `$null: null`, `$exists: 1` | accept (flags are not its arm) | accept | `INVALID_FILTER` / 400, every position | | `{ amount: { $gt: null } }`, `$lte: null` | 400 | accept | 400, every position | | `{ stage: { $in: 'won' } }`, `$nin: 'won'` | 400 | accept | 400, every position | | `{ stage: { $in: ['won', null] } }`, `$nin: [null]` | 400 | accept | 400, every position | | `{ amount: { $between: [null, 5] } }`, `5`, `[1]`, `[1, 2, 3]`, `['', 5]`, `[{ $field: 'a' }, 5]` | 400 | accept | 400, every position | | `{ stage: { $ne: ['won', 'lost'] } }`, `$ne: []` | 400 | accept | 400, every position | Controls answered accept on every door and face: `$null: true`, `$exists: false`, `$ne: null`, `$eq: null`, `$gt: { $field }`, `$ne: { $field }`, `$in: []`, `$nin: []`, `$between: [1, 5]`, `$between: [' ', 'M']`, `$in: [{ $field }]`, `$gt: true`. ## What changes - **`packages/spec/src/data/filter-save-door-refusals.ts` (new, not in the `data` barrel).** It holds `reportQueryFaceRefusals`, the one function both walks call. It asks the face about one slot, `{ [field]: comparand }` or `{ [field]: { [op]: comparand } }`, and reads only an `INVALID_FILTER` throw as a verdict; anything else is rethrown. Then it picks the words (below). It applies the flag rule to `$null` / `$exists`. It is a module of its own because `data/index.ts` re-exports `filter.zod.ts` whole, so an export there would be published API. It cannot import `filter.zod.ts` without a cycle, so the enforced operator slots (`FieldOperatorsSchema`) are passed in. - **`FilterConditionSchema`'s walk (`checkFilterConditionComparands`)** asks that function about every implicit comparand and every operator of a field entry, at depth 0: this node's own field entries, and every `$and` / `$or` / `$not` member through the schema's own re-parse. That is the face's reach. The walk's hand-written equality-slot checks (#19889) are now two of the face's arms, with the same sentence and the same path. - **#20207's carrier walk (`dataset.zod.ts`, renamed `refuseNestedRelationEqualityLists` → `refuseNestedRelationComparands`)** asks the same function about every entry INSIDE a nested relation. That is the analytics door's reach, which flattens a relation to dotted members. The walk still decides only where; the verdict and the words are the shared function's. Its equality-list refusals keep their sentence and path. - **Zone 2 answer 2, measured:** the face's own text cannot be reused without its location words. Its builders are module-private and put ` at PATH` mid-sentence in seven different shapes (`at P.`, `(at P)`, `(at P[i])`, `at P[i] of`, …). So the face is the JUDGE and not the text source; the text is chosen per arm as listed next. The #19889 / #20204 precedent moved the text into a shared module first, which would edit the face (⛔ in this order). ### The words, per arm (every new or changed refusal text, quoted) | arm | issue path | sentence source | |:--|:--|:--| | array in the equality slot, implicit or `$eq` | `stage`, `stage.$eq` | unchanged: `arrayEqualityComparandMessage`, shared with the face | | array under `$ne` (route A) | `stage.$ne` | `arrayInequalityComparandMessage` with the field, the face's sentence less its location | | `null` ordering comparand | `amount.$gt` | read off `FieldOperatorsSchema`'s slot for the same comparand | | `null` `$in` / `$nin` member | `stage.$in.1` | read off the slot | | `null`, blank or `{ $field }` `$between` endpoint | `amount.$between.0` | read off the slot | | non-list `$in` / `$nin` | `stage.$in` | NEW: the face's `nonListComparandError` sentence less ` at PATH`; the slot has only zod's generic wording | | `$between` that is not a pair | `amount.$between` | NEW: the face's `malformedRangeComparandError` sentence less ` at PATH` | | non-boolean `$null` / `$exists` | `stage.$null` | NEW: `driver-sql`'s first sentence word for word, then the analytics door's reason and prescription | The texts as printed (`FilterConditionSchema.safeParse`): ```text stage.$null ← { stage: { $null: "x" } } Operator "$null" on field "stage" requires a boolean comparand (true or false). Received a string ("x"). @objectstack/spec FieldOperatorsSchema declares $null as a boolean, and a non-boolean is refused rather than coerced because the backends read one in OPPOSITE directions — one as IS NULL, another as IS NOT NULL. Write the boolean itself: "$null": true matches rows whose "stage" has no value, "$null": false rows whose "stage" has a value. The filter was NOT applied. stage.$exists ← { stage: { $exists: "false" } } Operator "$exists" on field "stage" requires a boolean comparand (true or false). Received a string ("false"). @objectstack/spec FieldOperatorsSchema declares $exists as a boolean, and a non-boolean is refused rather than coerced because the backends read one in OPPOSITE directions — one as IS NULL, another as IS NOT NULL. Write the boolean itself: "$exists": true matches rows whose "stage" has a value, "$exists": false rows whose "stage" has no value. The filter was NOT applied. stage.$in ← { stage: { $in: "won" } } Operator "$in" on field "stage" requires an ARRAY of values. Received string ("won"). "$in" tests membership of a list — write ["won"] for a single value, or use "=" ($eq) to compare against it. Authoring spellings: in. The filter was NOT applied, and an unapplied filter would have returned the UNFILTERED result set. amount.$between ← { amount: { $between: 5 } } Operator "$between" on field "amount" requires a [min, max] value array. Received number (5). A range needs exactly two bounds, in order; the authoring spelling that lowers to "$between" is "between". The filter was NOT applied, and an unapplied filter would have returned the UNFILTERED result set. stage.$ne ← { stage: { $ne: ["won","lost"] } } Operator "$ne" on field "stage" requires a single comparable value, but received an array (["won","lost"]). For "none of these values" use {"$nin": […]} (authoring: nin, not_in, notin). The filter was NOT applied, and an unapplied filter would have returned the UNFILTERED result set. amount.$gt ← { amount: { $gt: null } } (the operator slot's existing sentence) null is not a valid $gt comparand. null is not ordered, and no two evaluation faces agree on what an ordering against it matches (driver-memory's live path reads two absences as equal; its reference matcher compares through JS coercion). State absence with the null predicate instead: {"$eq": null} is "has no value", {"$ne": null} is "has a value". Ruled 2026-09-01: a null ordering comparand is refused at the validation entrance. stage.$in.1 ← { stage: { $in: ["won", null] } } (the operator slot's existing sentence) null is not a valid $in member at index 1. No two backends agree on what a null in a list-comparand position matches (the SQL family answers a NULL under NOT IN unconditionally; the JS matchers split over the two readings of "no value"). State absence explicitly with the null predicate instead: {"$or": [{"$in": […]}, {"$null": true}]} is "one of […] OR has no value", and {"$null": false} is the has-a-value half. Ruled 2026-08-31: a null list member is refused at the validation entrance. amount.$between.0 ← { amount: { $between: [null, 5] } } (the operator slot's existing sentence) null is not a valid $between endpoint at index 0. (… the same null-member sentence as above …) amount.$between.0 ← { amount: { $between: ["", 5] } } (the operator slot's existing sentence) A blank value is not a valid $between endpoint at index 0 (the MIN bound). A closed interval [min, max] requires BOTH endpoints present and non-empty: an empty string is not an interval endpoint at any backend — it is compared as a value, so the range stops bounding on that side while still reading as a complete range. Write the bound you meant; and if only ONE side is genuinely bounded, that is not a range at all — drop $between and write the side you have as a scalar comparison ({"$gte": min} for a lower bound, {"$lte": max} for an upper one). Ruled 2026-09-17: a blank $between bound is refused at the validation entrance. amount.$between.0 ← { amount: { $between: [{ $field: "floor" }, 5] } } (the operator slot's existing sentence) A { "$field": … } reference is not a valid $between endpoint at index 0. No evaluation path resolves a field reference inside a list: the in-memory evaluator (matchesFilter) leaves the list unresolved and compares the raw reference OBJECT, so it silently matches nothing, and both SQL drivers refuse the position with INVALID_FILTER / 400. Write a literal value here, or move the reference to a scalar comparison operator ($eq/$ne/$gt/$gte/$lt/$lte), whose WHOLE comparand a { $field } reference may be. Ruled 2026-08-11: declared = enforced (ADR-0049). ``` A nested member on a dataset carrier prints the same sentence as its top-level form, with the leaf field named, at its own path, for example `filter.acct.stage.$in.1`. The changeset also carries a FROM → TO table for the HTTP doors. `POST /analytics/dataset/query` answers a one-bound `$between` in `selection.runtimeFilter` with `400 VALIDATION_FAILED`, located on the member, instead of `400 INVALID_FILTER` from the normalizer, and `POST /analytics/query` refuses the same shape in `where` at its request schema. It also carries rows for the producer's two spellings, `$in: [null, ""]` and `$nin: [null, ""]`. ### Changed docblocks - `checkFilterConditionComparands`: a new section, "Every slot the query faces refuse is refused on save (#20116)". The old bullet "⛔ `$ne` is not judged by this walk" is replaced; it was made false. - `inequalityComparandSchema`: the scope note "its own walk … does not judge `$ne`" now says the walk refuses the same shape through the face. - `refuseNestedRelationComparands`: the "⛔ Not judged here: `$ne` … and the face's OTHER arms inside a nested relation" paragraph is replaced by "Every slot the door refuses inside a relation (#20116)". ## What stays accepted (pinned, both doors) The null predicate (`$eq: null`, `$ne: null`), `$null` / `$exists` `true` and `false`, and a `{ $field }` reference as the whole comparand of `$eq` / `$ne` / `$gt` / `$gte` / `$lt` / `$lte`. Also a column-to-column range as two bounds, `$in: []` / `$nin: []`, `$in` with a `{ $field }` member (the face does not judge members), a whitespace or falsy `$between` endpoint, and relation traversal with no operator. On the shared schema (every carrier but the two dataset ones), any member shape INSIDE a nested relation stays accepted: neither the face nor the drivers' flag checks descend one. ## Remainder, named (why `Part of`) Every member the collector lists (`5854575239`, `5854887743`, and this card's own) is closed. This run measured two more positions of the same family: 1. **Nested-relation forms on a dashboard widget `filter` and a report `runtimeFilter`.** Both reach the analytics `where` door, which refuses every member inside a relation: `dataset-executor.ts` sends `combineFilters(compiled.filter, selection.runtimeFilter)` to it. Both still save the shape, because #20207's carrier refinement sits on the two dataset carriers only (triage record `5825670610`). The same gap holds for #20080's own equality-list shape on these two carriers. Closing it means applying the same refinement to `DashboardWidgetSchema.filter` and the report `runtimeFilter`s, which are outside this order's file surface. 2. **The comparand-TYPE face's refusals.** `normalizeFilterComparandTypes` refuses a plain-object or `Map` comparand, for example `{ stage: { $eq: { a: 1 } } }` or `{ stage: { $in: [{ a: 1 }] } }`. The analytics door refuses both, in every position, and every save door accepts both. The collector's enumeration names the shape face's tables and the flag arm, not the type face, so this PR does not move it. ## Producer census (Zone 2 answer 4) A literal-comparand `git grep -P` for each member shape, with a lit control per shape, over non-test, non-doc files: | tree | member hits that are authored filters | control hits (`$null` / `$exists` boolean, `$in: [`, `$between: [`) | |:--|:--|:--| | this repo `examples/**` at `af32cf9a` | 0 | 0 / 3 / 0 | | this repo `packages/**` at `af32cf9a` | 0 (every hit is prose, a type table or an operator map) | 87 / 230 / 40 | | objectui at the pin `f8a9d0fb05` | **1 producer**: `packages/fields/src/widgets/FilterConditionField.tsx:240-241` | 25 / 12 / 2 | | cloud `main` `48d70663ab` | 0 (one code comment) | 0 / 11 / 1 | The producer is objectui's filter-condition widget. `condToMongo` writes "is empty" as `{ [field]: { $in: [null, ''] } }` and "is not empty" as `{ [field]: { $nin: [null, ''] } }`. Both are offered by default for text, number, date, select and lookup fields. `field.form.ts` puts this widget on `relatedListFilter` and on a rollup's `summaryOperations.filter`, both `FilterConditionSchema` carriers, and it also edits `sys_sharing_rule.criteria_json`. The face has refused a `null` list member on every query since the 2026-08-31 ruling, so such a filter already fails its related list or rollup. After this PR, the Studio save is refused instead, at the slot, with the `$or` / `$null` prescription. No D2 conversion: the 2026-08-31 ruling declined to give a `null` list member any meaning, so a conversion would have to invent one. The producer fix is objectui's and is reported to the PM for routing. Blind spot: a multi-line literal or a runtime-built comparand is not matched by a line grep. ## Tests All heavy runs went through `scripts/pm/os-verify-lock.sh`. Each reading names the tree it was taken on. | run | tree | reading | |:--|:--|:--| | `pnpm --filter @objectstack/spec build`, `check:generated`, `typecheck` | `c300e80e` (final) | exit 0, exit 0, exit 0 | | `@objectstack/spec` full suite (`vitest run --maxWorkers=2`) | `c300e80e` (final) | 579 files, 16731 passed, 2 todo | | `@objectstack/spec` full suite | `70e9a610` plus the regenerated registry (`31ddfa40`) | 577 files, 16700 passed, 2 todo | | the two pin files (`filter-save-door-face-parity.test.ts`, `dataset-filter-nested-relation-list.test.ts`) | `70e9a610`, after the ablations were restored | 149 passed | | consumer: `@objectstack/service-analytics` full suite (the `where` door) | `70e9a610` | 128 files, 3022 passed | | consumer: `@objectstack/lint` full suite (`validate-chart-bindings` and the dataset readers) | `70e9a610` | 109 files, 4232 passed | | derived gates (`dispatch-gates.mjs --commands`, 87 lines) | `c300e80e` | 85 exit 0; 2 exit 3 PREREQUISITE NOT MET (`check:dual-build-cjs-loads`, `check:type-check-debt`, both need the whole-repo build); `--ran` reconciliation: 87 accounted, 0 unrun | **Patch round (review 5856977110, CI `Test Core (4/6)` at `c300e80e`).** `packages/rest/src/analytics-filter-refusal-envelope.test.ts` asserted the one-bound `$between` as `400 INVALID_FILTER` from the normalizer. `DatasetSelectionSchema.runtimeFilter` and `AnalyticsQueryRequestSchema.where` are `FilterConditionSchema`, so the route's schema door now answers first. Readings on `ab146a9a` (the tree batch F ran on; the batch header reads `dfca00f0` because the last commit, a test and changeset edit, landed before its spec step): rest full suite (`--project local`) 201 files, 3576 passed, 1 skipped; the flipped file 31 passed; service-analytics full suite 129 files, 3041 passed; spec full suite 581 files, 16770 passed, 1 todo (was 2: the `$ne` §5 todo is now a pin); spec `check:generated` exit 0; derived gates 87 derived, 85 exit 0, 2 exit 3 PREREQUISITE NOT MET (`check:dual-build-cjs-loads`, `check:type-check-debt`), `--ran` 87 accounted. Consumer sweep: a `git grep -P` for every refused member shape over all non-spec test files found 62 files. Only this one drives a request or schema door (`DatasetSchema` or `DatasetSelectionSchema` on `/analytics/dataset/query`, `AnalyticsQueryRequestSchema` on `/analytics/query`, `ObjectSchema` at registration) with a refused shape. Every other hit is an engine `where`, an RLS `scope`, a driver input or a comment. **Pin sweep.** Two published-behaviour pins flipped and were rewritten to assert the new semantics with their substance: - `filter.test.ts` "is not judged by the loose FilterConditionSchema — and neither is the #7596 shape" became "is refused by FilterConditionSchema too, at the endpoint, in the operator slot's words". It asserts the path `age.$between.1` and equality with `FieldOperatorsSchema`'s message for the same pair. - #20207's §4 control row "`$ne` carrying a list … not yet at this save door (#20116)" became a refusal test on both carriers. It asserts `INVALID_FILTER` / 400 at the analytics door, then the carrier's message equal to the door's less its location, and the `$nin` remedy. - **Patch round:** `packages/rest/src/analytics-filter-refusal-envelope.test.ts`. "a $between with one bound → 400 INVALID_FILTER" moved from the normalizer table to the `[#17551]` door table: it now asserts 400, `VALIDATION_FAILED`, exactly one `details.fields[]` entry at `selection.runtimeFilter.amount.$between`, and the face's sentence without ` at where.`. The sibling-schema control now asserts the refusal at `where.amount.$between` in the same sentence. The pass-list control is three spellings, not four. Its #20010 note moved with the row and still holds: the sentence is still the face's. - **Patch round:** `packages/spec/src/data/filter-ne-array-schema-door.test.ts` §5. Its `it.todo` ("the carrier still saves the shape") is fulfilled by this PR's `$ne` arm. It is now a real pin: path `stage.$ne`, message equal to the face's less its location, the `$nin` remedy, a dataset carrier at `filter.$or.0.stage.$ne`, and a null / scalar control. **Ablation (reverse verification), one-off, no permanent file.** Each leg went through `scripts/ablation-replace.mjs` WRAP mode from the committed tree `70e9a610`. The anchor hit exactly 1 time (x1 → x0, replacement x0 → x1), and the restore was proven by blob hash equal to HEAD with an empty `git diff HEAD`: | leg | mutation | result on the two pin files | |:--|:--|:--| | 1 | the face's verdict dropped (`if (face && false)`) | red: 95 failed / 54 passed, both files | | 2 | the flag rule dropped | red: 18 failed / 131 passed (the flag rows and the four §1 tables) | | 3 | the shared walk's reach widened into nested relations (`depth === 0` removed) | red: 30 failed / 119 passed. More diagnostics, not fewer: nested slots are refused twice, and the shared reach pins go red | | 4 | the carrier walk's operator-map arm dropped | red: 28 failed / 121 passed (nested `$eq` / §5 rows) | | restore | none | 149 passed | ## Acceptance notes - The face's docblock still names the schema door's twins as "`nullOrderingComparandMessage` (`./filter.zod.ts`)" and so on. Those builders did not move; the new module reads the same sentences off `FieldOperatorsSchema`'s slots. No edit to the face was needed or made. - `FieldOperatorsSchema.$in` / `$nin` / `$between` / `$null` / `$exists` still print zod's generic wording for a non-list, a malformed range and a non-boolean flag. `FilterConditionSchema` prints the pointed sentences above. Pointing the operator slots too is polish, and is not done here. - The four sentences read off the operator slot name the operator and the index but not the field, because the slot cannot see the field. The issue's path names it. --- _Generated by [Claude Code](https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent e462186 commit cfc3bcf

11 files changed

Lines changed: 1288 additions & 123 deletions
Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
---
2+
"@objectstack/spec": minor
3+
---
4+
5+
fix(spec)!: a filter carrying a comparand the query faces refuse is refused when it is saved (#20116)
6+
7+
**BREAKING** — an accept-set narrowing of published authoring schemas, shipped as `minor` under the repo's launch-window convention for accept-set narrowings. The save door narrows to exactly what the query faces already refuse. The hand-migration prescription is registered under protocol major 18 as `filter-query-face-comparands-refused-at-save`.
8+
9+
## What changes
10+
11+
`FilterConditionSchema` now refuses, at parse, every comparand slot the query faces refuse:
12+
13+
- a `$null` or `$exists` flag that is not a boolean — `"false"`, `"x"`, `null`, `1`;
14+
- a `null` comparand of `$gt` / `$gte` / `$lt` / `$lte`;
15+
- an `$in` or `$nin` comparand that is not a list, or a list holding `null`;
16+
- a `$between` comparand that is not a two-element list, or whose endpoint is `null`, blank (`""`) or a `{ $field }` reference;
17+
- an array under `$ne`.
18+
19+
It judges the field entries of a condition and of every `$and` / `$or` / `$not` member. The dataset `filter` and measure `filter` also judge the same slots INSIDE a nested-relation condition, through the nested-relation walk they already carry, because the analytics `where` door flattens a relation and judges its entries. The shared comparand-shape face (`assertListComparandShapes`) is called read-only as the judge for each slot, so the save door refuses exactly what that face refuses on every query. The two boolean flags, which that face does not judge, are refused on the predicate every flag face uses (`driver-sql`, `driver-memory`, `driver-mongodb`, the read-scope compiler and the analytics `where` door): the comparand is not a boolean.
20+
21+
Measured on `origin/main` `af32cf9a` before the change: a dataset `filter`, a dataset measure `filter`, a dashboard widget `filter` and a report `runtimeFilter` each parsed with `success: true` for one instance of every shape above. The comparand-shape face refused each one but the flags with `INVALID_FILTER` / 400, and the analytics `where` door refused all of them, inside a nested relation too. So such a document published clean and then failed every chart built on it.
22+
23+
Every schema that carries a `FilterCondition` refuses on parse. That covers the dataset `filter` and measure `filter`, the dashboard widget `filter` and options-source `filter`, the report and joined-report-block `runtimeFilter`, the field `relatedListFilter` and rollup `summaryOperations.filter`, the solution-blueprint summary `filter`, the analytics query `where`, the dataset selection `runtimeFilter`, the query `where` and `having`, the data-engine aggregate call's `having`, the aggregation `filter`, and the query-filter `where`. So `defineStack`, `os validate` and a save through the metadata protocol (`422 INVALID_METADATA`) refuse such a document at the slot's path, for example `filter.stage.$null` or `measures.0.filter.amount.$between.0`.
24+
25+
The words are the query face's. For an array under `$ne`, a non-list `$in` / `$nin` and a malformed `$between`, the refusal is the face's sentence without its location clause (`at where.<field>.<op>`), because the issue's path carries the location. For a `null` ordering comparand, a `null` list member or endpoint, and a blank or `{ $field }` endpoint, it is the sentence the enforced operator slot (`FieldOperatorsSchema`) already prints for the same comparand. A non-boolean flag gets the query faces' sentence: `Operator "$null" on field "stage" requires a boolean comparand (true or false).`, then the received value and the prescription.
26+
27+
Two request doors parse these carriers, and they now answer before the analytics compiler does. The REST dataset selection (its `runtimeFilter`, parsed against `DatasetSelectionSchema`) and the analytics query body (its `where`, parsed against `AnalyticsQueryRequestSchema`) answer `VALIDATION_FAILED` / 400 with the sentence at the field, for a top-level or combinator slot. Before, the compiler answered `INVALID_FILTER` / 400 for the same filter.
28+
29+
This also changes the `$ne` note of the equality-slot change earlier in this release: an array under `$ne` is now refused on save too, in the sentence `FieldOperatorsSchema.$ne` and the face print.
30+
31+
## What does NOT change
32+
33+
- **Nothing stored is rewritten, and nothing is dropped.** The parse fails and strips nothing. The read path does not re-validate stored rows, so a stored document keeps loading, and its next save is refused. Such a filter has failed every query since the runtime refusal of its shape, so the refusal is a repair.
34+
- **The shared reach is the face's, and no wider.** On every carrier but the two dataset ones, a field spec with no `$` key, such as the nested-relation condition `{ account: { region: { $in: ["a", null] } } }`, is not judged, because neither the face nor the drivers' flag checks descend one. A dashboard widget `filter` and a report `runtimeFilter` therefore still save that shape, and the analytics `where` door refuses it when they are charted.
35+
- **What the face passes still passes:** `$eq: null` and `$ne: null` (the null predicate), a `{ $field }` reference as the whole comparand of a scalar comparison, `$in: []` and `$nin: []`, a whitespace-only `$between` endpoint, and falsy endpoints such as `[0, 0]`.
36+
- **The data-engine calls' `where` option still parses.** Its type is a union whose first arm is an open record. The face refuses the shape when the call runs.
37+
- **No key, export or JSON Schema changes.** The published JSON Schema cannot state a refinement, and `FilterCondition`'s already could not state the equality-slot one.
38+
39+
## FROM → TO
40+
41+
| you wrote | write instead |
42+
|:--|:--|
43+
| `{ stage: { $null: "true" } }`, `{ stage: { $null: 1 } }` | `{ stage: { $null: true } }` ("has no value") |
44+
| `{ stage: { $exists: "false" } }`, `{ stage: { $null: null } }` | the boolean you meant: `$exists: false` is "has no value", `$null: false` is "has a value" |
45+
| `{ amount: { $gt: null } }` | `{ amount: { $eq: null } }` ("has no value") or `{ amount: { $ne: null } }` ("has a value") |
46+
| `{ stage: { $in: "won" } }` | `{ stage: { $in: ["won"] } }` or `{ stage: "won" }` |
47+
| `{ stage: { $in: ["won", null] } }` | `{ $or: [{ stage: { $in: ["won"] } }, { stage: { $null: true } }] }` |
48+
| `{ stage: { $nin: ["lost", null] } }`, meaning "has a value and is not `lost`" | `{ stage: { $nin: ["lost"], $null: false } }` |
49+
| `{ stage: { $in: [null, ""] } }`, a filter builder's "is empty" | `{ $or: [{ stage: { $null: true } }, { stage: "" }] }` |
50+
| `{ stage: { $nin: [null, ""] } }`, a filter builder's "is not empty" | `{ stage: { $null: false, $ne: "" } }` |
51+
| `{ amount: { $between: [null, 5] } }`, `{ amount: { $between: ["", 5] } }` | `{ amount: { $lte: 5 } }`, or the bound you meant |
52+
| `{ amount: { $between: [{ $field: "floor" }, 5] } }` | `{ amount: { $gte: { $field: "floor" }, $lte: 5 } }` |
53+
| `{ amount: { $between: 5 } }`, `{ amount: { $between: [1] } }` | `{ amount: { $between: [1, 5] } }` |
54+
| `{ stage: { $ne: ["won", "lost"] } }` | `{ stage: { $nin: ["won", "lost"] } }` |
55+
56+
### FROM → TO at the HTTP doors
57+
58+
Same status, different code: the refusal now comes from the route's schema door, located on the member, instead of from the analytics filter normalizer.
59+
60+
| request | before | after |
61+
|:--|:--|:--|
62+
| `POST /analytics/dataset/query` with `selection.runtimeFilter: { amount: { $between: [10] } }` | `400 INVALID_FILTER` from the analytics normalizer, in the comparand-shape face's sentence | `400 VALIDATION_FAILED`, `details.fields[]` entry `selection.runtimeFilter.amount.$between` with the sentence `Operator "$between" on field "amount" requires a [min, max] value array. Received array ([10]). …` |
63+
| the same route, any other slot above in `selection.runtimeFilter` (top level or in `$and` / `$or` / `$not`) | `400 INVALID_FILTER` | `400 VALIDATION_FAILED`, located on the slot, with that slot's sentence |
64+
| `POST /analytics/query` (`AnalyticsQueryRequestSchema`) with the same shape in `where` | `400 INVALID_FILTER` | refused by the request schema at `where.amount.$between`, answered `400 VALIDATION_FAILED` |
65+
66+
A client that branches on `INVALID_FILTER` for these shapes reads `VALIDATION_FAILED` instead. Both are 400 and both name the field.
67+
68+
## Who is affected, measured
69+
70+
A literal-comparand scan of every member shape, with a lit control per shape, over `examples/**` and the non-test `packages/**` of this repository at `af32cf9a`, the console repository at its pinned commit `f8a9d0fb05`, and the cloud repository's `main` at `48d70663ab`, found authored filters carrying one in one place. The console's filter-condition widget writes "is empty" as `{ field: { $in: [null, ""] } }` and "is not empty" as `{ field: { $nin: [null, ""] } }`. That widget edits a field's `relatedListFilter` and a rollup's `summaryOperations.filter` in the Studio field designer, and a sharing rule's criteria. Both shapes carry a `null` list member, which the face has refused on every query since the 2026-08-31 ruling, so a filter saved that way has been failing its related list or rollup since then. After this change, the Studio save is refused instead, with the `$or` / `$null` prescription. Every other hit is prose, a type table or a test fixture. Deployed datasets, dashboards and reports were NOT measured. Validating each stack, or re-saving each document, finds every instance the surface above lists.
71+
72+
Clause-②: no (narrowing) — nothing is widened. No key is added, removed or renamed, no exported symbol moves, and the operator vocabulary is unchanged. Comparand shapes that every query face already refused are now refused on save as well.
73+
74+
<!-- adr-0087: registered filter-query-face-comparands-refused-at-save -->

‎packages/rest/src/analytics-filter-refusal-envelope.test.ts‎

Lines changed: 51 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -193,19 +193,10 @@ describe('[#5352] POST /analytics/dataset/query — a filter refusal reaches the
193193
runtimeFilter: { stage: {} },
194194
message: /carries a field constraint with zero operators/,
195195
},
196-
{
197-
// [#20010] RE-WORDED, same verdict: still a real normalizer refusal,
198-
// still 400 INVALID_FILTER through this seam. PR #20032 hands every field
199-
// entry of an object-form `where` to the shared comparand-shape face
200-
// (`assertListComparandShapes`) before any node is built, and the face's
201-
// own rule answers first — a list operator takes a list, `$between` a
202-
// two-element [min, max] (#5869, moved to the face by #9228). So the
203-
// message is the face's, the same bytes the FilterArray spelling gets;
204-
// the analytics "needs a two-element" sentence is no longer reached.
205-
name: 'a $between with one bound',
206-
runtimeFilter: { amount: { $between: [10] } },
207-
message: /Operator "\$between" on field "amount" requires a \[min, max\] value array/,
208-
},
196+
// [#20116] `a $between with one bound` sat here. It no longer reaches the
197+
// normalizer: `FilterConditionSchema` now refuses every slot the shared
198+
// comparand-shape face refuses, so this route's schema door answers it
199+
// first. It moved, with its #20010 note, to the #17551 block below.
209200
{
210201
name: 'an unsupported top-level operator',
211202
runtimeFilter: { $nor: [{ stage: 'won' }] },
@@ -227,7 +218,9 @@ describe('[#5352] POST /analytics/dataset/query — a filter refusal reaches the
227218
/**
228219
* [#17551] Three spellings that used to reach the normalizer now stop one layer
229220
* earlier — at the route's schema door — and the block above no longer claims
230-
* them.
221+
* them. [#20116] A fourth joined them: the one-bound `$between`, once
222+
* `FilterConditionSchema` began refusing on save every comparand slot the
223+
* shared comparand-shape face refuses on query.
231224
*
232225
* ⚠️ This is a CODE change on a live wire surface, so it is recorded with the
233226
* measurement that justifies it rather than as a test edit. Since #17551 the
@@ -242,25 +235,41 @@ describe('[#5352] POST /analytics/dataset/query — a filter refusal reaches the
242235
* | `{ $or: [{…}, 'nope'] }` | refused at the schema | refused at the schema |
243236
* | `{ $not: 5 }` | refused at the schema | refused at the schema |
244237
* | `{ stage: {} }` | passes the schema | passes the schema |
245-
* | `{ amount: { $between: [10] } }` | passes the schema | passes the schema |
238+
* | `{ amount: { $between: [10] } }` | refused at the schema (#20116) | refused at the schema (#20116) |
246239
* | `{ $nor: [{…}] }` | passes the schema | passes the schema |
247240
* | `{ $or: [] }` | passes the schema | passes the schema |
248241
*
249242
* ⇒ the two routes now answer this field IDENTICALLY, which is the whole reason
250243
* the door exists ("one family, two postures" was the defect). The three rows
251-
* that changed changed because the dataset route used to be the LOOSER of the
252-
* two, not because anything narrowed past `FilterCondition`.
244+
* that changed under #17551 changed because the dataset route used to be the
245+
* LOOSER of the two, not because anything narrowed past `FilterCondition`. The
246+
* `$between` row changed under #20116 on BOTH routes at once, because
247+
* `FilterCondition` itself narrowed: the schema now refuses what the face
248+
* refused on every query, in the face's sentence less its location.
253249
*
254250
* ⛔ Nothing here weakens #5352's subject: the seam it exists for — a real
255251
* `AnalyticsService`, a real `normalizeAnalyticsFilterTree` refusal, and this
256252
* route's catch reading the envelope rather than a message list — is still
257253
* driven by every case left in the block above, `$sortOf` included.
258254
*/
259255
describe('[#17551] the structurally-malformed filter spellings are refused at the door', () => {
260-
const AT_THE_DOOR: Array<{ name: string; runtimeFilter: unknown }> = [
256+
const AT_THE_DOOR: Array<{ name: string; runtimeFilter: unknown; member?: string; sentence?: RegExp }> = [
261257
{ name: 'an $or that is not an array', runtimeFilter: { $or: 'won' } },
262258
{ name: 'an $or branch that is not a filter object', runtimeFilter: { $or: [{ stage: 'won' }, 'nope'] } },
263259
{ name: 'a $not of a non-object', runtimeFilter: { $not: 5 } },
260+
{
261+
// [#20010] The face's sentence has named this shape since PR #20032 handed
262+
// every field entry of an object-form `where` to the shared comparand-
263+
// shape face (`assertListComparandShapes`) — a list operator takes a
264+
// list, `$between` a two-element [min, max] (#5869, moved to the face by
265+
// #9228) — and it still does. [#20116] What moved is WHERE: the schema
266+
// door now asks that face on save and prints its sentence less the
267+
// `at where…` location, so the refusal is located on the member instead.
268+
name: 'a $between with one bound',
269+
runtimeFilter: { amount: { $between: [10] } },
270+
member: 'selection.runtimeFilter.amount.$between',
271+
sentence: /^Operator "\$between" on field "amount" requires a \[min, max\] value array\. Received array \(\[10\]\)\. A range needs exactly two bounds/,
272+
},
264273
];
265274

266275
for (const c of AT_THE_DOOR) {
@@ -274,8 +283,17 @@ describe('[#17551] the structurally-malformed filter spellings are refused at th
274283
expect(res.body.code).not.toBe('ANALYTICS_QUERY_FAILED');
275284
expect(res.body.code).toBe('VALIDATION_FAILED');
276285
// …and it says WHICH member, which the deeper refusal never did.
277-
const fields: Array<{ field: string }> = res.body.details.fields;
286+
const fields: Array<{ field: string; message: string }> = res.body.details.fields;
278287
expect(fields.map((f) => f.field).some((f) => f.startsWith('selection.runtimeFilter'))).toBe(true);
288+
if (c.member) {
289+
// [#20116] Exactly one entry, at the slot, carrying the face's sentence.
290+
const atMember = fields.filter((f) => f.field === c.member);
291+
expect(atMember, JSON.stringify(fields)).toHaveLength(1);
292+
expect(atMember[0]!.message).toMatch(c.sentence!);
293+
expect(atMember[0]!.message).not.toContain(' at where.');
294+
// The normalizer never ran: the refusal is the door's, not INVALID_FILTER.
295+
expect(res.body.code).not.toBe('INVALID_FILTER');
296+
}
279297
});
280298
}
281299

@@ -288,12 +306,24 @@ describe('[#17551] the structurally-malformed filter spellings are refused at th
288306
where: c.runtimeFilter,
289307
});
290308
expect(parsed.success, `${c.name} must be refused by the sibling schema too`).toBe(false);
309+
if (c.member) {
310+
// [#20116] …at the same slot, in the same sentence, one level up: the
311+
// sibling body carries the filter as `where`, not `selection.runtimeFilter`.
312+
const issues: Array<{ path: PropertyKey[]; message: string }> = parsed.error.issues;
313+
const slot = c.member.replace(/^selection\.runtimeFilter\./, 'where.');
314+
const atSlot = issues.filter((i) => i.path.join('.') === slot);
315+
expect(atSlot, JSON.stringify(issues.map((i) => i.path))).toHaveLength(1);
316+
expect(atSlot[0]!.message).toMatch(c.sentence!);
317+
}
291318
}
292319
});
293320

294-
it('CONTROL — the four spellings the schema PASSES still cross the seam', async () => {
321+
it('CONTROL — the three spellings the schema PASSES still cross the seam', async () => {
322+
// [#20116] `{ amount: { $between: [10] } }` left this list: the schema
323+
// refuses it now, on both routes (the AT_THE_DOOR row above, and the
324+
// sibling-schema control, which iterates that table).
295325
const { AnalyticsQueryRequestSchema } = await import('@objectstack/spec/api');
296-
const passes = [{ stage: {} }, { amount: { $between: [10] } }, { $nor: [{ stage: 'won' }] }, { $or: [] }];
326+
const passes = [{ stage: {} }, { $nor: [{ stage: 'won' }] }, { $or: [] }];
297327
for (const where of passes) {
298328
const parsed = (AnalyticsQueryRequestSchema as any).safeParse({
299329
cube: 'opportunity', measures: ['revenue'], where,

0 commit comments

Comments
 (0)