fix(spec)!: a dataset or measure filter with a list inside a nested relation is refused on save (#20080)

.changeset/19888-analytics-implicit-array.md

@@ -1,3 +1,3 @@
 ---
 "@objectstack/service-analytics": minor
 ---
@@ -27,4 +27,4 @@
 
 Each list is now handed to the shared face's equality arm before any node is built. Both spellings of one condition therefore get the same refusal, with the same wording, path and `$in` prescription. The draft-data preview runs the same gate, so a drafted chart refuses what the published chart refuses. Before, it compared each row against the list's string form.
 
-Who is affected: nothing in this repository's examples, seeds or docs authors the shape (measured over `examples/**`, `packages/**` and the fenced code in the docs). Stored datasets, dashboard widget filters, report runtime filters and measure filters in a deployment were NOT measured. In this release the authoring schema refuses the shape too, when such a document is saved (a separate change in `@objectstack/spec`, ADR-0087 entry `filter-equality-array-comparand-refused-at-save`). One position is judged only here: a list inside a nested relation, which this door flattens to a dotted member. A document carrying that still publishes, and it is refused when it is charted. The refusal names the field and the path. `$ne` with a list is not part of the ruling and is not judged here. The list operators (`$in`, `$nin`, `$between`) keep their lists, and every scalar, `null` included, compiles as before.
+Who is affected: nothing in this repository's examples, seeds or docs authors the shape (measured over `examples/**`, `packages/**` and the fenced code in the docs). Stored datasets, dashboard widget filters, report runtime filters and measure filters in a deployment were NOT measured. In this release the authoring schema refuses the shape too, when such a document is saved (a separate change in `@objectstack/spec`, ADR-0087 entry `filter-equality-array-comparand-refused-at-save`). One position is judged here and not by the shared authoring schema: a list inside a nested relation, which this door flattens to a dotted member. A dataset `filter` or measure `filter` carrying one is refused when it is saved, in the same words (a separate change in `@objectstack/spec`, ADR-0087 entry `dataset-filter-nested-relation-equality-array-refused-at-save`). Any other `where` that reaches this door with one, such as a caller `where` or a dataset selection's `runtimeFilter`, is refused only when it is charted. The refusal names the field and the path. `$ne` with a list is not part of the ruling and is not judged here. The list operators (`$in`, `$nin`, `$between`) keep their lists, and every scalar, `null` included, compiles as before.

.changeset/19889-filter-schema-door-array-equality.md

@@ -1,3 +1,3 @@
 ---
 "@objectstack/spec": minor
 ---
@@ -21,7 +21,7 @@
 ## What does NOT change
 
 - **Nothing stored is rewritten, and nothing is dropped.** The parse fails and strips nothing. The read path does not re-validate stored rows, so a stored document keeps loading, and its next save is refused. Such a filter has failed every query since the equality-slot change, so the refusal is a repair.
-- **The reach is the face's, and no wider.** A field spec with no `$` key, such as the nested-relation condition `{ account: { region: ['a'] } }`, is not judged, because the face does not judge it either. The analytics `where` door does refuse that shape when a dataset or measure filter is charted.
+- **The reach is the face's, and no wider.** A field spec with no `$` key, such as the nested-relation condition `{ account: { region: ['a'] } }`, is not judged by `FilterConditionSchema`, because the face does not judge it either. The analytics `where` door does refuse that shape, because it flattens the relation to a dotted member. So the two carriers that door charts, the dataset `filter` and the measure `filter`, refuse it on save as well, in the same words. That is a separate change in this release, ADR-0087 entry `dataset-filter-nested-relation-equality-array-refused-at-save`.
 - **The data-engine calls' `where` option still parses.** Its type is a union whose first arm is an open record. The face refuses the shape when the call runs.
 - **`$ne` carrying an array is not judged.**
 - The list operators keep their arrays, `$in: []` and `$nin: []` included. Every scalar, `null`, a `Date` and a `{ $field }` reference pass as before.

.changeset/20080-dataset-filter-nested-relation-list.md

@@ -0,0 +1,39 @@
+---
+"@objectstack/spec": minor
+---
+
+fix(spec)!: a dataset or measure filter with a list inside a nested relation is refused when it is saved, not when it is charted (#20080)
+
+**BREAKING**: an accept-set narrowing of a published authoring schema, shipped as `minor` under the repo's launch-window convention for accept-set narrowings. The hand-migration prescription is registered under protocol major 18 as `dataset-filter-nested-relation-equality-array-refused-at-save`.
+
+## What changes
+
+`DatasetSchema.filter` and `DatasetMeasureSchema.filter` now refuse an ARRAY in the equality slot of a field inside a nested-relation condition. That covers the implicit form `{ account: { region: ['a'] } }` and the explicit form `{ account: { region: { $eq: ['a'] } } }`, the empty array included, at any relation depth and under `$and` / `$or` / `$not`. So `defineStack`, `os validate` and a save through the metadata protocol (`422 INVALID_METADATA`) refuse such a dataset at the filter's path, for example `filter.account.region` or `measures.0.filter.account.region.$eq`.
+
+Before this change the dataset saved clean. The analytics `where` door, which charts both filters on every path, flattens the relation to the dotted member `account.region` and refuses the list with `INVALID_FILTER` / 400. So every chart built on the dataset failed. Measured on `origin/main` `9e7824a445`: `DatasetSchema.safeParse` answered `success: true` for both forms.
+
+The refusal is the analytics door's sentence: the field, the received list, and the two operators a list in that slot stood in for. The door adds `at where.account.region`. The schema leaves that out, because the issue's `path` already says where it is.
+
+## What does NOT change
+
+- **The shared `FilterConditionSchema` keeps its reach.** Every other schema that carries a `FilterCondition` still accepts a list inside a nested relation, because the engine reads that spec as a deep-equality comparand.
+- **Nothing stored is rewritten, and nothing is dropped.** The parse fails and strips nothing. The read path does not re-validate stored rows, so a stored dataset keeps loading, and its next save is refused. Such a filter has failed every chart, so the refusal is a repair.
+- A list outside a nested relation is refused as before, once, by `FilterConditionSchema`.
+- `$ne` carrying an array is not judged. The list operators keep their arrays, `$in: []` and `$nin: []` included. Every scalar, `null`, a `Date` and a `{ $field }` reference inside a nested relation pass as before.
+
+## FROM → TO
+
+| you wrote | write instead |
+|:--|:--|
+| `{ account: { region: ['a', 'b'] } }`, meaning "one of these values" | `{ account: { region: { $in: ['a', 'b'] } } }` |
+| `{ account: { tags: ['a'] } }`, meaning "the stored list holds `a`" on a multi-value field | `{ account: { tags: { $contains: 'a' } } }` |
+| `{ account: { region: ['a'] } }`, meaning one value | `{ account: { region: 'a' } }` |
+| `{ account: { region: { $eq: [...] } } }` | any of the rows above |
+
+## Who is affected, measured
+
+Nothing shipped in this repository carries the shape. At `9e7824a445`, the dataset and measure filters of `examples/app-crm`, `examples/app-showcase`, `examples/app-todo` and the platform objects, plus the dataset examples in `content/docs` and `skills`, carry no list inside a nested relation. Deployed datasets were NOT measured. Validating each stack, or re-saving each dataset, finds every instance.
+
+Clause-②: no (narrowing) — nothing is widened. No key is added, removed or renamed, no exported symbol moves, and the operator vocabulary is unchanged. One comparand shape in one position of two carriers, which the analytics door already refused on every chart, is now refused on save as well.
+
+<!-- adr-0087: registered dataset-filter-nested-relation-equality-array-refused-at-save -->

packages/spec/dropped-refinements.baseline.json

@@ -3,7 +3,7 @@
   "measured": {
     "zod": "4.4.3",
     "publishedSchemasWithDroppedRefinements": 207,
-    "droppedRefinementSites": 577,
+    "droppedRefinementSites": 588,
     "refinementSitesThatDidProject": 369,
     "refinementSitesWithNoJsonFormToCompare": 0
   },
@@ -56,7 +56,9 @@
         "manifest.dashboards.element.globalFilters.element",
         "manifest.dashboards.element.widgets.element",
         "manifest.datasets.element",
+        "manifest.datasets.element.filter",
         "manifest.datasets.element.include.element",
+        "manifest.datasets.element.measures.element.filter",
         "manifest.datasources.element",
         "manifest.flows.element",
         "manifest.flows.element.errorHandling",
@@ -154,7 +156,9 @@
         "data.options[1].manifest.dashboards.element.globalFilters.element",
         "data.options[1].manifest.dashboards.element.widgets.element",
         "data.options[1].manifest.datasets.element",
+        "data.options[1].manifest.datasets.element.filter",
         "data.options[1].manifest.datasets.element.include.element",
+        "data.options[1].manifest.datasets.element.measures.element.filter",
         "data.options[1].manifest.datasources.element",
         "data.options[1].manifest.flows.element",
         "data.options[1].manifest.flows.element.errorHandling",
@@ -236,7 +240,9 @@
         "options[1].manifest.dashboards.element.globalFilters.element",
         "options[1].manifest.dashboards.element.widgets.element",
         "options[1].manifest.datasets.element",
+        "options[1].manifest.datasets.element.filter",
         "options[1].manifest.datasets.element.include.element",
+        "options[1].manifest.datasets.element.measures.element.filter",
         "options[1].manifest.datasources.element",
         "options[1].manifest.flows.element",
         "options[1].manifest.flows.element.errorHandling",
@@ -279,7 +285,9 @@
         "data.packages.element.options[1].manifest.dashboards.element.globalFilters.element",
         "data.packages.element.options[1].manifest.dashboards.element.widgets.element",
         "data.packages.element.options[1].manifest.datasets.element",
+        "data.packages.element.options[1].manifest.datasets.element.filter",
         "data.packages.element.options[1].manifest.datasets.element.include.element",
+        "data.packages.element.options[1].manifest.datasets.element.measures.element.filter",
         "data.packages.element.options[1].manifest.datasources.element",
         "data.packages.element.options[1].manifest.flows.element",
         "data.packages.element.options[1].manifest.flows.element.errorHandling",
@@ -1134,12 +1142,15 @@
     "ui/Dataset": {
       "sites": [
         "",
+        "filter",
         "filter.lazy",
-        "include.element"
+        "include.element",
+        "measures.element.filter"
       ]
     },
     "ui/DatasetMeasure": {
       "sites": [
+        "filter",
         "filter.lazy"
       ]
     },

packages/spec/src/migrations/entries/semantic/18.dataset-filter-nested-relation-equality-array-refused-at-save.ts

@@ -0,0 +1,70 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+import type { SemanticMigration } from '../../types.js';
+
+// The analytics carriers' half of filter-equality-array-comparand-refused-at-save.
+// That entry refuses an equality-slot list on save wherever the shared face
+// refuses it, and names a list inside a nested-relation condition as a position
+// it deliberately leaves to execution. This one closes that position on the two
+// carriers the analytics where door charts — a dataset filter and a measure
+// filter — because that door flattens a nested relation to a dotted member and
+// refuses the list there. Recorded as its own entry because the surface is
+// narrower (two carriers, one position) and the shared schema does not move.
+export const entry: SemanticMigration = {
+  id: 'dataset-filter-nested-relation-equality-array-refused-at-save',
+  // No backticks in `surface` — build-upgrade-guide renders it inside a code
+  // span already, and a nested backtick would close it.
+  surface:
+    'ui.Dataset filter and ui.DatasetMeasure filter — an ARRAY as an EQUALITY comparand on a '
+    + 'field INSIDE a nested-relation condition, now refused when the dataset is PARSED: the '
+    + 'implicit form { account: { region: [...] } } and the explicit form '
+    + '{ account: { region: { $eq: [...] } } }, the empty array included, at any relation depth '
+    + 'and under $and / $or / $not. Every other schema that carries a FilterCondition keeps the '
+    + 'shared schema\'s reach',
+  replacement:
+    'the operator the list was standing in for, on the same field inside the same relation, '
+    + 'exactly as in filter-equality-array-comparand-refused-at-save. "One of these values" is '
+    + '$in: { account: { region: { $in: ["a", "b"] } } } (authoring spelling "in"). "The stored '
+    + 'multi-value field holds this value" is $contains with ONE member (authoring spelling '
+    + '"contains"), and an $or of those for any-of. A filter that meant a single value writes that '
+    + 'value: { account: { region: "a" } }. The list operators keep their arrays, empty lists '
+    + 'included; every scalar, null above all, is untouched; and $ne is NOT judged by this entry',
+  reason:
+    'Measured on origin/main 9e7824a445 before the change: DatasetSchema parsed a dataset whose '
+    + 'filter was { account: { region: ["a"] } }, and one whose measure filter was '
+    + '{ account: { region: { $eq: ["a"] } } }, GREEN — while the analytics where door, which '
+    + 'charts both carriers on every path (the native-SQL and ObjectQL strategies and the draft '
+    + 'preview), flattens the relation to the dotted member account.region and hands the list to '
+    + 'the shared comparand-shape face, which refuses it with INVALID_FILTER / 400. So such a '
+    + 'dataset saved clean and every chart built on it failed, for a different person, later. '
+    + 'The shared FilterConditionSchema does not descend a field spec with no $ key, because the '
+    + 'engine reads one as a deep-equality comparand; ruling A on #19889 (record 5805248669) put '
+    + 'it there and it stays there. Triage on #20080 (record 5825670610) routed the fix to the two '
+    + 'analytics carriers instead: they refine their filter with the analytics door\'s own walk '
+    + '($and / $or arrays and $not descended, other $ keys skipped, a plain object with no $ key '
+    + 'descended as a nested relation at any depth) and refuse, inside a nested relation only, '
+    + 'exactly what that door refuses there, in the face\'s words from the one builder both doors '
+    + 'import. The one difference is that the door appends the location (at '
+    + 'where.account.region) and the carrier does not, because its issue carries the location as '
+    + 'its path (filter.account.region, measures.0.filter.account.region.$eq). A list outside a '
+    + 'nested relation is the shared schema\'s refusal and is reported once. No filter changes '
+    + 'meaning: the refusal moves from chart time to save. Metadata AT REST is not rewritten and '
+    + 'this entry adds no D2 conversion, for the reason the runtime entry gives: an array on '
+    + 'equality has no single honest value. The read path does not re-validate stored rows, so a '
+    + 'stored dataset keeps loading; what changes is that re-saving it through the metadata '
+    + 'protocol (422 INVALID_METADATA), defineStack or os validate is refused at the filter\'s '
+    + 'path. Such a filter has failed every chart since the analytics door began refusing it, so '
+    + 'the refusal is a repair and not a loss. In-repo census at 9e7824a445: no dataset or '
+    + 'measure filter in examples, platform objects, docs or skills carries the shape; deployed '
+    + 'datasets were NOT measured. ADR-0021 / ADR-0087.',
+  acceptanceCriteria:
+    'Validate every stack and re-save every stored dataset: os validate or defineStack, and a '
+    + 'save through the metadata protocol, report each list in an equality slot inside a nested '
+    + 'relation by path, with the field, the received list and both remedies, so the sweep of the '
+    + 'two carriers is mechanical. Decide per filter what it meant — one of these values ($in), '
+    + 'the stored list holds a value ($contains, an $or of them for several), or one value — and '
+    + 're-check what each chart is supposed to show: the filter had been failing every chart. A '
+    + 'filter that reaches the analytics door by any other route, such as a caller where or a '
+    + 'dataset selection runtimeFilter, is still refused only when it is charted, with '
+    + 'INVALID_FILTER / 400 naming the field and the path.',
+};

packages/spec/src/migrations/entries/semantic/18.filter-equality-array-comparand-refused-at-save.ts

@@ -53,8 +53,10 @@ export const entry: SemanticMigration = {
     + 'with no $ key (a nested-relation or deep-equality condition), which the face never '
     + 'descends either. ⚠️ Three positions therefore still refuse only at execution. (1) A list '
     + 'inside a nested-relation condition, { account: { region: ["a"] } }: the analytics where '
-    + 'door flattens that to the dotted member account.region and refuses it when a dataset or '
-    + 'measure filter is charted. (2) The where option of the data-engine calls (find, count, '
+    + 'door flattens that to the dotted member account.region and refuses it when the filter is '
+    + 'charted; a dataset filter and a measure filter refuse it on save as well, which is the '
+    + 'sibling entry dataset-filter-nested-relation-equality-array-refused-at-save. (2) The where '
+    + 'option of the data-engine calls (find, count, '
     + 'update, delete, aggregate, vector find): its type is a union whose first arm is an open '
     + 'record, so it parses and the face refuses it when the call runs. (3) $ne carrying a '
     + 'list, which no ruling has decided. Two request doors parse these carriers and now answer '
@@ -79,5 +81,7 @@ export const entry: SemanticMigration = {
     + 'query. ⛔ A clean re-save is NOT a complete sweep for the three positions the reason '
     + 'names: grep nested-relation conditions and data-engine where options for a field whose '
     + 'value is a list, and exercise them, where the runtime doors refuse with INVALID_FILTER '
-    + '/ 400 naming the field and the path.',
+    + '/ 400 naming the field and the path. A dataset filter or a measure filter is the '
+    + 'exception: its nested-relation lists are refused on save too '
+    + '(dataset-filter-nested-relation-equality-array-refused-at-save).',
 };