Repository navigation
fix(objectql)!: a no-operator object beneath a relation, structured-JSON or undeclared id column is refused INVALID_FILTER / 400 on every driver (#20745) - #20781
Conversation
…tured-JSON or provisioned column at the engine door (WIP) Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…, live dialects and InMemoryDriver (WIP) Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…; spec prose, docs and changesets Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…te cell is a measured reading Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
… engine's own refusal Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 2 package(s): 21 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: ⛔ 5 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails. What this run could not see
Coarse fallback — 138 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 870fcbb63c7b6939f306f79d8f31c36188595b71 && git checkout 870fcbb63c7b6939f306f79d8f31c36188595b71
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 1bcba27d2e73242c6169d24e12031b3a9f52b44d ee17b18fdec9355e633b8f651518d2f30d98a984 && git checkout -B drift-repro 1bcba27d2e73242c6169d24e12031b3a9f52b44d && git merge --no-ff ee17b18fdec9355e633b8f651518d2f30d98a984
node scripts/docs-audit/affected-docs.mjs --json 1bcba27d2e73242c6169d24e12031b3a9f52b44d
|
Contract reviewServed-tier: PR #20781 for card #20745, branch Check-runs on the head, the newer of two readings this round (this comment posted 2026-09-30T05:31Z): 34 runs, 27 ① Derived judgmentsEvery accept-set and public-surface change the diff implies, each judged:
② Semver level
③ Boundary flagsEvery dev flag from the os-dev-report and the PR body, and both
Implemented-by: VERDICT: PASS Generated by Claude Code |
…ID_FIELD / 400 at the engine aggregate door, on every driver (objectstack-ai#20783) (objectstack-ai#20804) Fixes objectstack-ai#20783 Clause-②: no (narrowing) ## What this changes A `groupBy` entry that names a declared **structured-JSON** field (`json`, `composite`, `repeater`, `record`, `location`, `address`, `vector`) is now refused by `engine.aggregate` with `INVALID_FIELD` / 400, in the engine's words, before any driver is asked. It holds on every driver and for every caller that reaches the engine: the REST query door, a flow or hook calling the engine in-process, and the analytics strategy that lowers a cube query onto `engine.aggregate`. Both entry spellings are judged: the field name (`groupBy[0]`) and the `{ field }` object (`groupBy[0].field`), a `dateGranularity` bucket included. The words, as `POST /api/v1/data/:object/query` returns them (the route is inside the 500 characters the REST door keeps, and the REST pin asserts it there): ```text aggregate('rest_group_by_json_20783'): groupBy[0] names 'meta', a declared json field — a structured-JSON value, which the engine does not group by. The query was NOT run. Group by a field that stores one scalar value: store the part you group on in a field of its own and group by that field. A JSON document is no group key the drivers share: one merged every row into a single group, one grouped each serialized document apart, one refused the statement. ``` The thrown error carries `code: 'INVALID_FIELD'`, `status` and `httpStatus` 400, `field`, `fields` (every offending entry), `object` and `param: 'groupBy'`. **Landing site, as the order expected: `packages/objectql`'s aggregate admission. No driver file, no serialization rule.** - `packages/objectql/src/group-by-structured-json-door.ts` (new): `assertGroupByNamesNoStructuredJsonField(object, schema, groupBy)`. The class is the spec's `STRUCTURED_JSON_TYPES` (the set PR objectstack-ai#20781's JSON arm judges), never a list minted here. Not judged: an undeclared name (the engine's registry-less tolerance; the REST ingress answers an unknown name `INVALID_FIELD` first), a host with no field map, and every other type. - `packages/objectql/src/engine.ts`: one call at the entry of `aggregate`, right after `rejectCredentialAggregation` (which reads the same `groupBy` entries), so a protected field keeps that refusal's words. `aggregate` is the only engine verb that takes `groupBy`: `find` refuses the key (`ENGINE_FIND_OPTION_KEYS`). - `packages/objectql/src/number-comparand-declared-type-door.ts`: comment only. Its sentence "a `json` … groupBy … is judged now" at `having` became false for a json groupBy, which no longer reaches `having`. **Why `INVALID_FIELD`, an existing code.** The verdict is about the named field's type at a position. That is the question the REST ingress answers with `INVALID_FIELD` for an unknown `groupBy` name (`assertGroupByFieldsExist`), the search axis answers with `INVALID_FIELD` for a field whose type it cannot scan, and the engine's `assertFilterIsMaterializable` answers with `INVALID_FIELD` for a virtual field ("this verdict is about the NAME's type"). `INVALID_FILTER` is the engine's value-shape envelope and `groupBy` is not a filter; `INVALID_QUERY` is the ingress's malformed-shape code, and the entry here is well formed. ## Before, measured on `origin/main` `7a09eee1b1` Through `POST /api/v1/data/:object/query` (the real `RestServer` route over `ObjectStackProtocolImplementation` and `ObjectQL`) with `{ groupBy: [FIELD], aggregations: [{ function: 'count', alias: 'n' }] }`, and through `engine.aggregate` directly (same answers). Drivers: InMemoryDriver, SqlDriver on SQLite (better-sqlite3), and SqlDriver on a private PostgreSQL 16.13 started for this run. Three rows: `title` x, x, y; `meta` `{a:1}`, `{a:2}`, `{b:1}`; and one differing value per row under every other structured-JSON field. | `groupBy` | InMemoryDriver | SQLite | PostgreSQL 16 | |:--|:--|:--|:--| | `title` (text, the control) | 200, `x` 2 · `y` 1 | same | same | | `meta` (json, the card) | 200, **one group** `{a:1}`, `n` 3 | 200, one group per serialized document (3) | **500 `DATABASE_ERROR`** ("could not identify an equality operator for type json") | | `composite`, `repeater`, `record`, `location`, `address` | 200, one group, `n` 3 | 200, one group per serialized document | 500 | | `vector` | 200, one group per array (`[1,2]` 2 · `[3,4]` 1) | 200, one group per serialized array | 500 | | `{ field: 'meta' }` | 200, one group, `n` 3 | 200, 3 groups | 500 | | `{ field: 'meta', dateGranularity: 'month' }` | 200, one `null` bucket, `n` 3 | 200, one `null` bucket, `n` 3 | 500 ("cannot cast type json to timestamp with time zone") | | `['title', 'meta']` | 200, 2 groups | 200, 3 groups | 500 | ## After, the same run on this branch (`43ae6c1fcc`) Every structured-JSON row above answers `400 INVALID_FIELD` in the engine's words on all three drivers, naming the position (`groupBy[0]`, `groupBy[0].field`, `groupBy[1]` for the mixed entry), the field and its declared type. No read of the object runs. The `title` control answers `x` 2 · `y` 1 on all three, unchanged. The InMemoryDriver cells of this run came from a scratch script over the built packages. They are not committed: `check:driver-memory-census` refuses a new test consumer of that driver without a ruling, so the committed memory cell is the recording driver below, by construction. ## Hypotheses (zone 2): which held - **H1: held.** `aggregate` resolves `groupBy` entries against the declared field map before the driver in `rejectCredentialAggregation` (credential and `internal` fields) and, for `having`, in `aggregatedRowColumnTypes`. The refusal sits beside the first, at the verb's entry, so it runs before the per-aggregation `filter` and `having` doors and before any driver is resolved. Code: `INVALID_FIELD`, reasons above. The REST ingress also resolves `groupBy` names (`assertGroupByFieldsExist` in `metadata-protocol`), but only for the REST path. The engine is the one door every caller shares, as triage directed. - **H2: held, refined.** Every member of `STRUCTURED_JSON_TYPES` answers per driver today, and none answers one way. Six members split exactly like `json` (memory one merged group, SQLite per serialized document, PostgreSQL 500). `vector` splits differently on memory (one group per array, not one merged group) and still 500 on PostgreSQL, so it does not answer one way either. So the class is refused, not `json` alone. - **H3: held, and it was NOT already refused.** A date-bucketed `{ field, dateGranularity }` over a `json` field passed the REST ingress (a known field, a valid granularity) and answered one `null` bucket on memory and SQLite and 500 on PostgreSQL. It is refused now with the `{ field }` form's words, since no granularity makes a JSON document a date. - **H4: measured. The analytics face is partly the same door.** Measured through `AnalyticsService.query` wired with `AnalyticsServicePlugin`'s own auto-bridges (`executeAggregate` to `engine.aggregate`, `executeRawSql` to `engine.execute`), with a cube dimension `sql: 'meta'` on a `json` field: | cube query | InMemoryDriver | SQLite | PostgreSQL 16 | |:--|:--|:--|:--| | `dimensions: [meta]`, before | 200, one merged group (raw SQL unsupported, fell back to `engine.aggregate`) | 200, one group per serialized document (**native SQL**, engine not reached) | 500 `DATABASE_ERROR` (**native SQL**) | | `dimensions: [meta]`, after | **400 `INVALID_FIELD`** (this door) | unchanged | unchanged | | `timeDimensions: [{ meta, granularity: month }]`, before | 200, one `null` bucket | 200, one `null` bucket | 500 | | the same, after | **400 `INVALID_FIELD`** (the native strategy declines a granularity, so `engine.aggregate` serves it) | **400** | **400** | So the ObjectQL strategy (any cube query on a driver without raw SQL, and any bucketed time dimension) reaches this door and is folded in with no `packages/services` edit. **`NativeSQLStrategy` does not**: on SQL drivers it compiles `GROUP BY "meta"` by hand and bypasses the engine. That half is a sibling card (reported to the PM below, not filed here). The dataset compiler only checks that a dimension's field is declared (`assertDeclared`), so a dataset dimension on a `json` field compiles into the same cube dimension and splits the same way. The memory cube face (`MemoryAnalyticsService`) has **zero** production constructors (`git grep "new MemoryAnalyticsService"` outside tests: 0 hits at `7a09eee1b1`), so it reaches a driver only in tests. It was not edited and not measured for this shape. ## Producers measured (for "refuse, don't define") Zero producers group by a structured-JSON field. Every `grouping` / `groupBy` / `groupByField` / `dimensions` target in `examples/` at `7a09eee1b1` is one of `status`, `priority`, `created_at`, `account`, `stage`, `total`, `region`, `sales_region`, `progress`, `issued_on`, `industry`, `category`, `signed_on`, `health`, `completed_date`, `close_date`, and none is structured-JSON (the showcase's structured-JSON fields are `field-zoo`'s `f_*`, `account.hq`, `account.support_config` and `task.location`). The same count over the platform packages names `object_name`, `user_id`, `id`, `action`, `topic`, `provider_id`, `organization_id`, `namespace`, `kind`, `actor_id` and `phone`, plus two dynamic engine callers in `service-analytics` (see the acceptance notes). ## Tests - New `packages/objectql/src/engine-group-by-json-door.test.ts` (6 tests, recording driver, so the in-memory cell by construction). It covers every structured-JSON type with the full envelope (`code`, `status`, `httpStatus`, `field`, `fields`, `object`, `param`) and the position, and asserts zero reads. It covers the `{ field }` form, a date bucket, an entry after a scalar one and two offenders, and the REST door into `findData`. Controls: `text`, `number`, a `multiple: true` select, an image field and an undeclared name all reach the driver, and a structured-JSON field as an aggregated column is unchanged. GUARDs: the judged types equal `STRUCTURED_JSON_TYPES` over every `FieldType`, and there is no verdict without a field map, for an undeclared name, or for an entry naming no field. - New `packages/rest/src/data-group-by-json-door.test.ts`: SQLite always, PostgreSQL / MySQL where `OS_TEST_POSTGRES_URL` / `OS_TEST_MYSQL_URL` are set. Every structured-JSON type answers 400 `INVALID_FIELD` with the route in the REST body and zero reads. The object and bucket forms and the mixed entry answer the same 400. The control `title` answers `x` 2 · `y` 1 from the driver.⚠️ As with the sibling door suites, no CI job sets those URLs for `@objectstack/rest`, so the live cells run only locally. Local run with the private PostgreSQL 16.13: **6 passed (sqlite 3, live postgres 3) / 3 skipped (mysql, no URL)**. - Fixture triage in `engine-nested-object-door.test.ts` (PR objectstack-ai#20781's pin): its `having` case over `groupBy: ['meta']` pinned the branch this PR closes, because a json groupBy never reaches `having` now. It is **replaced**, not respelled. The JSON arm at `having` is still reached through a `max` of a json field (`having: { top_meta: { a: 1 } }`), which answers the same `INVALID_FILTER` whole-value words. No other fixture groups by a structured-JSON field. Measured by a grep over every test that both groups and declares a structured-JSON type; the downstream suites below stayed green. - `pnpm --filter @objectstack/objectql test` at `43ae6c1fcc`: **342 files / 6739 passed**. - `pnpm --filter @objectstack/rest test` at `43ae6c1fcc` (with the live PostgreSQL URL set): **232 files / 4516 passed / 33 skipped**. - `typecheck` for `@objectstack/objectql` and `@objectstack/rest` at `43ae6c1fcc`: exit 0, including each `check:test-typecheck` (objectql's ledger held at 40 files / 234 errors / 65 signatures; rest 0), so both new test files compile. - Downstream consumers (`...@objectstack/objectql` direction), at `cafaf885d8`: `@objectstack/service-analytics` **141 files / 3267 passed**; `@objectstack/metadata-protocol` **190 files passed, 3 skipped / 2792 passed, 19 skipped**. The other consumers are declared to CI. **Reverse verification (ablation), from the committed fix at `1eacad5296`.** It ran through `scripts/ablation-replace.mjs` in WRAP mode, trap-restored. The engine's call `assertGroupByNamesNoStructuredJsonField(object, this._registry.getObject(object), query.groupBy);` was fed `(query as { __ablated_20783__?: unknown }).__ablated_20783__` (always undefined) instead of `query.groupBy`. On disk the anchor went 1 to 0 and the marker 0 to 1, with blob `67198fc4a8fc` to `837fc9257e37`. objectql was rebuilt, and `ablation-dist-preflight` found the marker in 4 built files. - Predicted direction: red. Observed: red. - objectql pins: **3 failed / 18 passed**. Every refusal case failed. The controls, both GUARDs and the whole `engine-nested-object-door.test.ts` stayed green. - rest pins: **4 failed / 2 passed / 3 skipped**. The refusal cases failed on SQLite and live PostgreSQL, and the `title` control stayed green. - Restore leg: blob equals HEAD (`67198fc4a8fc`), `git diff HEAD` is empty, and whole-tree `git status --porcelain` is empty. After a rebuild, the `--absent` preflight found the marker absent from all 14 built files, and the tree reading was clean. Then both pin sets were green again (21 passed; 6 passed + 3 skipped). ## Gates `node scripts/pm/dispatch-gates.mjs --commands` at `43ae6c1fcc` (the merge of `origin/main` `96e724475c`, which touches none of this diff's packages) derived **65** commands over the 7 changed paths. All 65 were run on `43ae6c1fcc`, and `--ran` reconciles them: **65 derived, 65 run, 0 NOT-MEASURED, 0 UNRUN**, all exit 0. Among them: - `check:adr-0087-registration --base origin/main`: `not-required (no-migration-prescription)` accepted. - `check:changeset-no-major`, `check:empty-changeset`, `check:doc-authoring`, `check:nul-bytes`, `check:issue-citations`. - `check:engine-double-contract`, `check:where-matcher`, `check:driver-memory-census`, `check:cross-package-test-inputs`, `check:test-source-alias`, `check:type-check-coverage`, `check:type-check-debt`. - `check:query-options-erasure`. It caught a first draft of the REST pin that erased an `engine.aggregate` options bag to `any` (test surface 236 to 237). Fixed by typing it (`cafaf885d8`), and the surface is back at 236. - `check:dual-build-cjs-loads`, after a full `turbo run build` of `./packages/*` and `./packages/*/*`. Lint, narrowed and proven: `pnpm exec eslint --no-inline-config --format json` over the 6 changed `.ts` files at `43ae6c1fcc` found **6 files, 0 errors, 0 warnings**. Three facts make this narrowing a measurement: - The checked population comes from eslint's own config: `isPathIgnored` answers `false` for all 6. - The file count comes from the JSON output: 6 results. - Untouched files cannot change verdict: `parserOptions.project` and `projectService` are `null` for every file, so type-aware linting is not enabled. ## Changeset `.changeset/20783-groupby-structured-json-refused.md`: `@objectstack/objectql` `minor`, a BREAKING banner, `Clause-②: no (narrowing)`, and exactly one ADR-0087 marker, `not-required (no-migration-prescription)`, in the form PR objectstack-ai#20781's changeset uses. It carries no rewrite table and no arrow: the body states what an author sees now, why, who is affected and what is unchanged. No export or published type changes: the door module is internal, and `@objectstack/objectql`'s root and `./core` exports are unchanged. ## Acceptance notes - **Reported to the PM, not filed here (same family, measured):** - **The native-SQL analytics face** (triage's H4 sibling). A cube or dataset dimension on a structured-JSON field through `NativeSQLStrategy` answers one group per serialized document on SQLite and 500 on PostgreSQL. It never reaches this door. The fix lands in `packages/services/service-analytics`, which this claim does not touch. - **`groupBy` on a `multiple: true` select** through `POST /api/v1/data/:object/query`: memory answers one group per array, SQLite one per serialized array, and PostgreSQL 500 (json equality). Not this card's class, and a multi-value field has a plausible other meaning (a bucket per member), so it is not refused here. - **`count_distinct` over a `json` field** through the same door: memory 3, SQLite 3, PostgreSQL 500. `AGGREGATE_FIELD_TYPE_COMPATIBILITY` accepts `count_distinct` for every `FieldType` on the ground that every backend gives one answer. PostgreSQL does not for json. - `service-analytics` has two dynamic `engine.aggregate` callers: `resolveFkAttr` (`groupBy: ['id', attr]`, a cross-object dimension attribute) and the display-label pass (`groupBy: ['id', displayField]`). If that attribute or display field is structured-JSON, they now answer this 400. Before, by reading and not measured: memory and SQLite grouped under `id` first, so one row per record, and PostgreSQL would have answered the same json-equality 500. No example or platform dataset names such a dimension (the census above). - A refusal relayed through the analytics engine path names the engine's position (`groupBy[0]`), not the cube member the caller wrote (`c20783.meta`). Carrier: the native-SQL sibling card, which touches the same face. - The native-SQL analytics path returned PostgreSQL's `count` as a string (`"2"`) where SQLite returned a number, in the stand-in wiring above. Observed only; not measured through the HTTP door. --- _Generated by [Claude Code](https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…d record's column, not the refused nested form (objectstack-ai#20811) Fixes objectstack-ai#20782 Clause-②: no ## What this changes The published skill `skills/objectstack-query` taught `{ relation: { field: value } }` beneath a lookup as a working `where` form. On `main` the engine refuses that form on every driver — `INVALID_FILTER` / 400, in the words of `relationWords()` in `packages/objectql/src/no-operator-object-door.ts` — and names the route it serves: filter the related object first, then match the relation column against the ids it returns (`$in` on a single-valued column; `$contains` per id, an `$or` of those for several, on a `multiple: true` one, whose JSON column the SQL driver refuses `$in` on). The skill now states that refusal and that route at every site that taught or pointed at the form. It anticipates neither letter of the open v18 decision (objectstack-ai#20802 is not addressed here): it states today's behaviour. Six landing sites, all in `skills/objectstack-query`, measured on `origin/main` `96e72447`: | Site (at `96e72447`) | Before | After | |:--|:--|:--| | `SKILL.md:76`, Removed-key row `query.joins` | "`expand`, or a nested relation filter" | "`expand` (display), or filter the related object and `$in` its ids" | | `SKILL.md:88`, rules index | "nested relations" | "filtering by a related record" | | `SKILL.md:187-:194`, subsection | "Nested Relation Filters" with the `{ contact: { profile: { verified: true } } }` example | "Filtering by a related record": the refusal, the route, a pointer to the rule | | `SKILL.md:334` | "use a nested relation filter" | "`$in` ids from its own query" | | `SKILL.md:343`, Cross-Object row | "Filter parent by child conditions — Nested relation filter" | two rows, one per direction (below) | | `rules/filters.md:132-:152`, section | "Nested Relation Filters", two ✅ examples of the refused form | "Relation Filters": the refusal, one two-step example, the multi-valued spelling, the reverse direction | **The `:343` row was mislabelled.** "Filter parent by child conditions" names the reverse direction (a parent by its children's fields), while the form it pointed at, even had it been served, expresses only the forward one (rows by their lookup target's column). The row is now two: "Filter rows by their lookup target's column" → query the target object, then `{ lookup: { $in: ids } }` (`$contains` per id when `multiple`); "Filter parent by child conditions" → query the child with `fields: [lookup]`, then `{ id: { $in: those ids } }` on the parent. Both routes are served: the engine's own `expand` batch-loads with `{ id: { $in } }` (`engine.ts`, the "Batch-load related records using $in query" block), and the REST ingress admits `id` as a filter key (`protocol.ts` `resolveQueryFields`, `known.add('id')`). **No dotted alternative is taught.** `'owner.region'` is refused one door earlier by the objectstack-ai#8371 dotted verdict (`filter-comparand-shape.ts`, `INVALID_FIELD` / 400); a grep of the skill for a dotted `where` path finds none. **Census.** `grep -rn -i -E 'nested relation|relation filter|nested relations' skills/` on `96e72447` finds exactly the six sites; a multi-line shape grep for a `where` example nesting a no-operator object under a relation key finds the same three (`SKILL.md:193`, `rules/filters.md:138`, `:145`) and nothing else in `skills/**`. ## Paying for it inside `rules/filters.md` The file sat at 2148 / 2149 tokens. The section rewrite (881 bytes) is paid for by deleting three examples whose rule already has a home in the same package, so the file lands at 2149 / 2149 (headroom 0, ceiling untouched): | Deleted from `rules/filters.md` | Home | |:--|:--| | `## Implicit Equality (Shorthand)` (`:40-:51`) | `SKILL.md` "Implicit Equality (Shorthand)" (`:94-:101`), and the `$eq` row of this file's Operator Reference | | `### NOT` (`:93-:100`) | `SKILL.md` Logical Operators, `{ where: { $not: { status: 'closed' } } }` (`:183-:184`) | | `### Combining Logical Operators` (`:102-:113`) | `SKILL.md` "AND + OR combined" (`:173-:181`); sibling-keys-are-AND is this file's first Common Mistake | The `role: 'admin'` literal in the deleted third example moved `check:role-word`'s count for the file 8 → 7; the gate prescribes the ratchet-down ("run `--update` and commit the baseline"), and commit `b893787b` is that `--update` output: one row of `scripts/role-word-baseline.json`. That file is outside the claim's declared surface; declared here and in the report. ## `skills/**` readings (lines and tokens; tokens are the ratchet's `ceil(utf8 bytes / 4)`) | | Before (`96e72447`) | After (`b893787b`) | Δ | |:--|--:|--:|--:| | `SKILL.md` | 401 lines · 3990 tokens (ceiling 5552) | 399 lines · 4055 tokens | −2 lines · +65 tokens | | `rules/filters.md` | 251 lines · 2148 tokens (ceiling 2149) | 214 lines · 2149 tokens | −37 lines · +1 token | | Whole package `skills/objectstack-query` (6 files) | 1184 lines · 10461 tokens | 1145 lines · 10527 tokens | −39 lines · +66 tokens | Line budget (PM-set, net 0 at most): −39. No untouched line was re-wrapped; no ceiling row moved. ## Verification Gates, all at head `b893787b`, exit codes captured by redirect and verdict lines quoted from each log: the 31 families `dispatch-gates --commands --repo objectstack-ai/objectstack` derives from the change set, reconciled with `--ran` ("31 derived, 31 run, 0 NOT-MEASURED, 0 UNRUN") — every one exit 0, including `check-skills-token-ratchet` ("54 authored bundle file(s) within their ceilings") and its `--self-test` (65 cases), `check:role-word` after the ratchet-down ("OK, no new occurrences of the reserved word"), `check:corpus-claim-drift`, `check:skill-identifier-liveness`, `check:doc-authoring`, `check:nul-bytes`, `check:doc-formula-expressions` (`@objectstack/formula` and `@objectstack/lint` built first), spec `check:skill-docs` ("Skill docs in sync") and spec `check:skill-refs` ("9 generated files in sync"). `check:skill-examples` does not apply: no block in this skill carries an `os:check` marker. Pin: the skill's example validation does not cover these snippets (no `os:check` marker), so the pin is PR objectstack-ai#20781's, on `main`: `packages/objectql/src/engine-nested-object-door.test.ts` (the refusal envelope `{ code: 'INVALID_FILTER', status: 400 }` beneath `lookup`, `master_detail`, `multiple: true` lookup, `user` and `tree`, and the CONTROL that `{ owner: { $in: [...] } }`, `{ owners: { $contains: ... } }`, its `$or`, and `{ id: { $in: [...] } }` reach the driver as written) and `packages/rest/src/data-nested-object-door.test.ts` (400 over `POST /api/v1/data/:object/query`; the routes answer `d1`, `d3`). Run first-hand here: `pnpm --filter @objectstack/objectql exec vitest run --maxWorkers=2 src/engine-nested-object-door.test.ts` → "Test Files 1 passed (1), Tests 15 passed (15)". No package is touched, so no package build or test suite is owed beyond that. ## Acceptance notes - `content/docs/protocol/objectql/query-syntax.mdx` (`:602-:610`, read-only here): the "Filtering Across Relationships" callout's headline is true (neither the nested form nor a dotted path is served), but its mechanism paragraph is stale — it says `SqlDriver.applyFilters()` compiles the nested object as a single-column comparison and emits the dotted key verbatim to Knex, while on `main` both are refused at the engine before any driver (`INVALID_FILTER` / 400 for the nested form, `INVALID_FIELD` / 400 for the dotted path), and it names no served route. Noted, not filed: documentation drift under a true headline; no carrier known. - `check:pm-governed-merges` as the package script spells it is the `--self-test` alone (exit 0 here); the live sweep is CI's. - The report's `api_writes` lists every relay write of this run. ## 维护者速读(草稿) **改了什么。** 发布的查询技能包 `skills/objectstack-query` 原先把「在关联字段下直接写条件」(`{ customer: { country: 'US' } }`)当作能用的过滤写法来教。现在六处教它或指向它的句子都改成平台今天的真实行为:这种写法被引擎拒绝(`INVALID_FILTER` / 400),可行的路是先查关联对象拿到 id,再对关联字段用 `$in`(多值关联用 `$contains` 逐个 id)。规则文件里删掉了三段在 SKILL.md 已有同样规则的重复示例,用来支付这段改写;整个包净减 39 行,每个文件的 token 上限都没动。 **为什么改。** AI 作者照着技能包写,写出来的查询在内存驱动上静默返回 0 行、在 SQL 驱动上报 400;PR objectstack-ai#20781 合并后所有驱动都统一拒绝。技能包是客户项目里 AI 的教材,教错一句就是每个客户项目里的错误查询。本 PR 不预判 v18 决策卡(objectstack-ai#20802)的任一方向,只陈述今天的行为。 **风险与代价(含回滚)。** 只改文档文字与一行门禁基线(`role-word` 计数 8 → 7,门禁自己要求的下调),不改任何代码或发布包。风险在措辞:若维护者裁定 v18 支持关联过滤,这几句还要再改一次(决策卡已注明)。回滚即 revert 本 PR 的两个提交。 **席位意见。** **你要做的。** 复核六处措辞与三处删除各有归宿;批准后由席位落地(Tier H)。 --- _Generated by [Claude Code](https://claude.ai/code/session_01KTZmMfzVzjNvyaLyQ8mHvg)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…t the engine seam, the related object read as the caller, a loud cap, drivers untouched (objectstack-ai#20802) (objectstack-ai#20872) Part of objectstack-ai#20802 Clause-②: yes (widening) The engine half of ruling 5907789183 (letter A). The analytics cube read and the analytics read-scope face are the second half, after objectstack-ai#5930 step 3 (objectstack-ai#20810); `skills/objectstack-query` (objectstack-ai#20782) belongs to the skills seat, who is told after this lands. So this PR does not complete the card. ## What this does **The nested-relation form `{ relation: { field: value } }` is served in `where`.** It is lowered at the engine's filter seam that objectstack-ai#5930 step 2 built (`cfa931535`). The drivers receive `$in` / `$contains` and are not changed (ADR-0053 D-D1 item 5, D4 (b)). - **Where the lowering runs.** Stage 2 of `where` admission now has three steps: resolve the placeholders, then lower each nested-relation condition, then run the shared `lowerFilterCondition`. `ObjectQL.resolveRelateThenLowerWhere` holds that order, so a verb cannot resolve without it. `resolveWhereTokens` and `withResolvedWhere` became async to carry it. - **How the lowering works.** `lowerRelationConditions` reads the related object with the engine's own `find`: `fields: ['id']`, `limit: RELATION_FILTER_ID_CAP + 1`, and the caller's execution context. The ids it returns become `{ relation: { $in: ids } }` on a single-valued relation. On a multi-valued one they become an `$or` of one `$contains` per id, which matches on any member. That is the spec's own any-of spelling, and the SQL family refuses `$in` over the JSON column. No related record matching gives `$in: []` or `$or: []`: FALSE, never an absent predicate. - **No second walk.** A condition is found by the one filter walk the engine already runs with each column's declaration in hand, `walkCondition` in `number-comparand-declared-type-door.ts`. - At the door (stage 1), that walk admits a condition structurally, from declarations alone (`admitRelationCondition`), instead of refusing it. - The lowering calls the same walk (`mapRelationConditions`) twice: once to collect the conditions, and once, after the reads, to replace them in the same order. - The door and the lowering therefore find a condition at the same boundaries by construction. No driver-local guard exists. - **As the caller.** The related read goes through the middleware chain like any read. The related object's CRUD gate, row scope and field permissions apply, and its own doors judge the condition's comparands. A refusal from any of them is the answer, loudly. - **Bounded.** The cap is `RELATION_FILTER_ID_CAP` = 1000, one named constant, exported from `@objectstack/objectql`. Past it the filter is refused with `INVALID_FILTER` / 400. The refusal names the cap, the related object and the two-step route. The filter is never run over a cut-off list. **The accept set that widens.** It is the engine's `where` (`find`, `findOne`, `count`, `aggregate`, `update`, `delete`) plus `judgeFilter`, and the REST query doors that reach `findData` (`POST /api/v1/data/:object/query` and the `filter` / `$filter` spellings). Per relation kind: - `lookup`, `master_detail`, `user`, `tree`, single-valued: refused, now served (`$in`). - the same kinds with `multiple: true`: refused, now served (any member). **Nothing served today narrows.** - The admission runs only where the arm refused before: a no-operator object beneath a relation column at `where`. - The cap refusal is new, but it applies to a form that was refused. - Every filter without a nested-relation condition reaches the drivers by reference, byte-identical to before (pinned). **What stays refused.** Each is refused in the engine's words, before any read: - a `json` field's object comparand, and the provisioned `id` (unchanged); - a second level (a relation condition beneath the related object's own relation field); - a dotted key inside the condition; - a key the related object does not declare; - `{}`; - a related object that is not registered; - the dotted path `{ 'owner.region': 'NA' }`, still `INVALID_FIELD` (objectstack-ai#8371); - the form in an aggregation's own `filter` and in `having`. The engine evaluates both itself, and its evaluator has no member test for a stored list. Their words now say `where` serves it. **Text.** - `FilterCondition`'s docblock item 4 now states the served semantics. - The `QueryFilter` example shows the form again. - The `data-engine.mdx` example that PR objectstack-ai#20781 removed comes back, with the cut, the cap and the two-step route. **The dotted-path words, made true again (a bounded in-place fix, named here).** - Both dotted relation refusals said "a filter reaches only columns of '…' itself". This change makes that false. - The two refusals are the engine's (`filter-comparand-shape.ts`) and the query-parameter door's (`metadata-protocol` `protocol.ts`, outside the claim's declared file surface). - Both now name the nested form to write instead (`{ "owner": { "region": VALUE } }`), in the same words, and keep the shared denormalise remedy. - A conformance pin holds the two routes equal. ## Measured On this branch at `56da9b6d50` through `POST /api/v1/data/:object/query`. Owner `u1` is region NA on `d1` and `d3`, and `d4` has no owner. Before, on `origin/main` after PR objectstack-ai#20781, every relation row answered `INVALID_FILTER` / 400 on every driver. | `where` | SQLite | PostgreSQL 16.13 (live, local) | InMemoryDriver (measured, not pinned) | |:--|:--|:--|:--| | `{ owner: { region: 'NA' } }` (lookup), `boss` (master_detail) | `d1`, `d3` | `d1`, `d3` | `d1`, `d3` | | `{ owners: { region: 'NA' } }` (multiple lookup) | `d1`, `d3` | `d1`, `d3` | `d1`, `d3` (see the note) | | `{ parent: { title: 'a' } }` (tree) | `d2`, `d3` | `d2`, `d3` | `d2`, `d3` | | `{ $not: { owner: { region: 'NA' } } }` | `d2`, `d4` | `d2`, `d4` | `d2`, `d4` | | `{ $or: [{ owner: { region: 'EU' } }, { title: 'a' }] }` | `d1`, `d2` | `d1`, `d2` | `d1`, `d2` | | `{ owner: { region: 'APAC' } }` (no match) | none | none | none | | a condition matching 1001 related records | 400 `INVALID_FILTER`, the cap words | same | not measured | - **The memory note.** The in-memory driver matches `$contains` over a stored array by substring per element. That is the gap `FILTER_OPERATORS`' `$contains` docblock records for that driver. So with ids `u1` and `u10`, a multi-valued condition meaning `u1` also matches the row holding `['u10']`: measured memory `d1`, `d3`, `d5`, against SQL `d1`, `d3`. Single-valued relations are exact everywhere. - **Why memory is not pinned.** `check:driver-memory-census` refuses a new test consumer of that driver without a ruling. - **H7 (RLS), measured.** A policy `record.owner.region == 'NA'` is refused at compile: "cross-object/nested field path … is not pushdown-able". The policy is dropped to the deny sentinel, so it answers zero rows. The RLS compile seam is untouched, and no async read was added there. ## Mechanism hypotheses: which held - **H1: held, refined.** `lowerFilterCondition` is pure and synchronous, and the relation step needs the engine. So the step lives in objectql, between token resolution and the shared lowering, and each engine filter position reaches it at most once. - Served: `where` on `find`, `findOne`, `count`, `aggregate`, `update` and `delete` (the multi and by-predicate paths alike), and the judge. - Not served (refused, named): `aggregations[i].filter` and `having`. - **H2: held, with one widening of the kind list.** The door admits the form under every `REFERENCE_VALUE_TYPES` kind, not only `lookup` / `master_detail`. - Reason: `user` and `tree` point at a related object the same way, and the objectstack-ai#20745 seat answer ruled that one class gets one answer. - `user` needs `sys_user` registered; where it is not, it is refused loudly ("no object 'sys_user' is registered here"). - Every refusal the order listed is kept, as listed above. - **H3: measured "yes": the one check exists and is reused.** - A direct filter on a field the caller cannot read is refused today: `403 PERMISSION_DENIED`, the security layer's filter-oracle guard `assertReadableQueryFields`. - The related read reaches that same check, so the nested form answers the same 403, naming the field. - There is no second copy of the rule.⚠️ This is not the ruling's literal `INVALID_FILTER`; see the open question in the report. - **H4: `$in` does not mean "any member" everywhere; `$or` of `$contains` does on SQL.** - `$in` over a multi-valued lookup is refused on SQL (JSON column) and is any-member on memory. - `$contains` is membership on SQLite and PostgreSQL, and substring-per-element on memory (the note above). - The `$or` of `$contains` is the spec's declared any-of spelling, so it is the lowered form. - **H5: held, and no bound to reuse.** `expand`'s batch loader bounds nothing: it deliberately forwards no limit. The cap is a new named constant. - **H6: held.** `$and` / `$or` compose as written. `$not` over a relation condition takes the shared lowering's NULL-safe negation, so a row with no relation satisfies it. The driver input is pinned equal to the hand-written two-step route's, and `$not` + no match gives every row. An empty inner result is FALSE, never "no filter". `$nor` is not in the vocabulary. - **H7: held; out of scope.** Measured above. ## Tests (all on `56da9b6d50`) - `@objectstack/objectql` test: 345 files / 6786 passed. typecheck exit 0, `check:test-typecheck` OK. - `@objectstack/rest` test, with `OS_TEST_POSTGRES_URL` set to a local PostgreSQL 16.13: 237 files / 4706 passed / 35 skipped (MySQL cells and suites with no URL). typecheck exit 0. - `@objectstack/metadata-protocol` test: 191 files passed, 3 skipped / 2801 passed, 19 skipped. typecheck exit 0. - `@objectstack/spec` test: 578 files / 17066 passed / 1 todo. typecheck exit 0. `check:generated`: all 15 artifacts up to date (no regeneration needed; docblock only). - Downstream: `@objectstack/plugin-security` 149 files / 3227 passed, 23 skipped. `driver-memory` 65 / 1470 passed. `driver-sql` 201 files passed, 11 skipped / 3254 passed, 188 skipped. The other `...@objectstack/objectql` consumers are declared to CI. - New pins: - `packages/objectql/src/engine-nested-relation-lowering.test.ts` (13 tests, recording driver). It covers: - every relation type; - any-member on a multi-valued relation; - the empty id set; - `$and` / `$or` / `$not` / sugar, pinned equal to the two-step route's driver input; - every verb and the judge; - placeholder resolution; - the related read as the caller, and a middleware refusal surfacing with no outer read; - the cap at 1000 and at 1001; - the kept refusals, with the judge answering execution's words verbatim; - the related object's own doors; - the aggregation `filter` / `having` refusals; - a pass-through control. - `packages/rest/src/data-nested-object-door.test.ts` (rewritten): objectstack-ai#20745's table turned into rows on SQLite and live PostgreSQL, plus the two-step equivalence, the kept refusals with the route inside the 500-character REST bound, the cap pin (1001 related records refused, 1000 served) and the controls. - `packages/rest/src/data-nested-relation-permission.test.ts`: the permission pin, with the real `SecurityPlugin` on a real engine and SQLite, through the REST door. An unreadable related field is refused 403, never emptied, while a system read can filter by it. A hidden related record matches nothing. - Fixture triage for the removed refusal branch: - `engine-nested-object-door.test.ts` keeps only what still refuses. - `query-expression-conformance.test.ts`: its nested-form control now pins the served rows, and a new pin checks that both doors' dotted refusals name the same route. - `protocol-explicit-filter-field-gate.test.ts`: its GUARD still proves the name gate never descends. **Ablations, each from the committed fix.** Each ran through `scripts/ablation-replace.mjs` in WRAP mode, trap-restored. After each mutation objectql was rebuilt, and `ablation-dist-preflight` found the marker in 4 built files. - **A, the lowering disabled.** `if (sites.length === 0) return where;` became an unconditional `return where` (marker `__ablated_20802_lowering__`). Blob `9237c3dfc995` → `f9994599356f`. Predicted red, observed red: - objectql pins: 10 failed / 235 passed; - rest pins: 11 failed / 4 passed / 6 skipped. - The structural refusals and controls stayed green. - **B, the related read as the system.** `...(execCtx ? { context: execCtx } : {}),` became `isSystem: true` (marker `__ablated_20802_caller__`). Blob → `b0c1748c5b70`. Predicted red, observed red: - objectql 1 failed / 244 passed (the as-the-caller pin); - rest 2 failed / 13 passed / 6 skipped: the permission pin answered rows instead of 403, and the row-scope pin returned `d4`. - (A first run of B used a mutation that left `execCtx` unused, and its DTS step failed on TS6133; the JS carried the marker. It was re-run type-clean, and those numbers are the ones above.) - **Restore.** Blob equals HEAD `9237c3dfc995`, and `git diff HEAD` is empty. After a rebuild, the `--absent` preflight found both markers absent from all 14 built files, and the whole tree was clean. The pins were green again: 245 passed; 15 passed / 6 skipped. ## Gates `node scripts/pm/dispatch-gates.mjs --commands` at `56da9b6d50` (fresh, not stale) derived 120 commands, and all 120 were run. `--ran` with each exit code recorded: **120 derived, 120 run, 0 NOT-MEASURED, 0 UNRUN**, all exit 0. Three gates refused first with `PREREQUISITE NOT MET` (exit 3), and none of the three is counted as a failure: - `check:skill-examples` - `check:dual-build-cjs-loads` - `check:type-check-debt` They were re-run green after `turbo run build --filter='./packages/*' --filter='./packages/*/*'`. Lint, narrowed and proven: `eslint --no-inline-config --format json` over the 15 changed `.ts` files gave **15 files, 0 errors, 0 warnings**. - `isPathIgnored` is false for all 15. - The population is `eslint.config.mjs`'s `**/*.{ts,…}` block. - `parserOptions.project` / `projectService` are unset for every file. Type-aware linting is off, so this diff cannot move a verdict on an untouched file. ## Changesets - `.changeset/20802-nested-relation-filter-served.md`: `@objectstack/objectql` `minor`, `Clause-②: yes (widening)`. It says it supersedes the relation-field paragraph of the pending `20745-nested-object-door` entry, and it states the in-memory `$contains` substring caveat. - `.changeset/20802-nested-relation-prose.md`: `@objectstack/spec` `patch` (shipped JSDoc). - `.changeset/20802-dotted-relation-route.md`: `@objectstack/metadata-protocol` `patch` (refusal words). - ADR anchor: `scripts/adr-anchors/packages__objectql__src__relation-filter-lowering.ts.json` → ADR-0053. ## Acceptance notes - **The analytics half.** The cube read and the analytics read-scope face are not touched here (objectstack-ai#20810 first). Until then the analytics face still flattens the nested form to cube members, and the read scope still refuses it. - **The permission refusal's envelope.** It is `PERMISSION_DENIED` / 403, the one existing check, reused. It is not the ruling's parenthetical `INVALID_FILTER`, and it is raised to the PM as an open question. - **Read order.** The related read runs at stage 2, before the outer verb's own middleware. A caller with no read access to the queried object still gets the outer 403, but the related read has already run as that caller. It reads nothing the caller could not read directly. - **Lint face.** `@objectstack/lint`'s list-view dotted-path hint still says "Filter on a column of … itself". That is an instruction rather than a claim this change made false. Carrier: none; not changed here. - **The in-memory substring gap.** On the in-memory driver, the multi-valued any-member lowering inherits that driver's substring-per-element `$contains`. It is reported, not fixed here (no driver file). --- _Generated by [Claude Code](https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
Fixes #20745
Clause-②: no (narrowing)
What this changes
A plain object with no
$-operator key beneath a relation field (the nested-relation form), a structured-JSON field (a whole-value match), or the platform-provisionedidcolumn that the declared field map omits is now refused withINVALID_FILTER/ 400, in the engine's words, before any driver is asked. It holds on every driver, at the three positions the engine judges:where(object form andFilterArraysugar;find/findOne/count/aggregate/update/deleteand the judge-onlyjudgeFilter),aggregations[i].filter, andhaving.Landing site: the door PR #20744 opened for scalars, extended. There is no second door and no second traversal. The no-operator-object arm of the number-comparand door's walk (
walkCondition) already asked one question per field key. It now classifies the column into one of three kinds, each with its own words:packages/objectql/src/no-operator-object-door.ts:noOperatorObjectColumnKindgivesscalar(unchanged:SCALAR_FILTER_HEAD_TYPESplusMULTI_OPTION_TYPES),relation(spec'sREFERENCE_VALUE_TYPES:lookup,master_detail,user,tree),json(spec'sSTRUCTURED_JSON_TYPES), ornull(never judged).provisionedNoOperatorObjectColumncoversid/created_at/updated_atwhen the declared map omits them. The three word builders live here too. ⛔ Nothing in it walks a filter.number-comparand-declared-type-door.ts: the per-key facts carry the judged column instead of a scalar type. The walk, its positions and its boundaries are unchanged.engine.ts/having-filter.ts: comments only.packages/spec/src/data/filter.zod.ts: prose only.FilterCondition's form 4 now states that the engine refuses it and names the route. TheQueryFilterexample stops teaching it. The two "Nested relation" type comments point at the refusal. ⛔ The type and the schema are not narrowed.content/docs/kernel/contracts/data-engine.mdx: the// Nested relation filterexample is replaced by the route that works (query the related object, then$inon its ids), with one paragraph on the refusal and on multi-valued lookups.The words put the verdict and the route first. The REST door truncates a 4xx message at 500 characters (
CLIENT_MESSAGE_MAXinpackages/rest/src/error-response.ts). The first draft of these words, and the #20546 scalar words, put the route past that bound, so a REST caller never saw it. Every kind now reads: position, then verdict, thenThe filter was NOT applied., then the route, then the reasoning. The REST pins assert the route in the response body. The #20546 scalar words were rewritten in the same shape because this change made their middle sentence false: it said a nested-relation condition is something "only a relation field … can carry". Examples of the words, asengine.findthrows them:Before, measured on
origin/maina51920f5fbThe readings come through
POST /api/v1/data/:object/query(the realRestServerroute overObjectStackProtocolImplementationandObjectQL) on InMemoryDriver, SqlDriver on SQLite (better-sqlite3), and SqlDriver on a live PostgreSQL 16.13 started for this run. Three rows: owneru1(region NA) ond1andd3;meta{a:1}/{a:2}/{b:1}.where{ owner: { region: 'NA' } }(lookup, the card)d1,d3meant)INVALID_FILTER, the driver's words ("cannot be bound as a SQL parameter")master_detail, amultiple: truelookup,user,treewhere{ meta: { a: 1 } }(json, the card)d1(deep equality)where{ ship_to: { city: 'Paris' } }(address),{ spec: { k: 1 } }(composite)d1,d3(deep equality)where{ id: { a: 1 } }(the card)where{ owner: {} },{ meta: {} }aggregations[1].filter{ owner: { region: 'NA' } }aggregations[1].filter{ meta: { a: 1 } }having{ owner: { region: 'NA' } }overgroupBy: ['owner']having{ meta: { a: 1 } }overgroupBy: ['meta']DATABASE_ERROR(from the json groupBy itself, see the notes){ owner: { $in: ['u1'] } }; same undermaster_detail,userd1,d3d1,d3d1,d3{ parent: { $in: ['d1'] } }(tree)d2,d3d2,d3d2,d3{ owners: { $in: ['u1'] } }(multiple lookup)d1,d3{ owners: { $contains: 'u1' } }, and$orof$containsd1,d3d1,d3d1,d3{ 'owner.region': 'NA' }INVALID_FIELD(the dotted verdict){ meta: { $contains: 'a' } }INVALID_FILTER(the text-operator door: a JSON value is never a string){ meta: { $null: false } },{ meta: { $exists: true } }{ photo: { url: 'x' } }(image)After, the same run on this branch
Every refused row above answers
400 INVALID_FILTERin the engine's words, on all three drivers, at the path the object sits at (where.owner,where.$not.owner,aggregations[1].filter.owner,having.owner, …). No read of the object runs. The routes ($in,$contains,$null), the dotted verdict, the$contains-on-json refusal and the image control answer exactly as before.Hypotheses (zone 2): which held
REFERENCE_VALUE_TYPESbrings inuserandtreebesidelookup/master_detail, andSTRUCTURED_JSON_TYPESbrings inaddress,composite,repeater,record,locationandvectorbesidejson. See the scope note below. (b) Left unjudged: file and media types (the [finding] The FILTER axis has no DOTTED-path verdict —where: { project_id.name: 'x' }rides its head segment past both doors, where SORT refuses the same spelling (#4256) #8371 carve-out: a legacy stored value is an inline object that memory can still match) andformula(refused one door earlier,INVALID_FIELD). Triage's text covers neither.'owner.region'is served by no driver: it answers400 INVALID_FIELD(the [finding] The FILTER axis has no DOTTED-path verdict —where: { project_id.name: 'x' }rides its head segment past both doors, where SORT refuses the same spelling (#4256) #8371 dotted verdict) on all three. So the refusal names only the related-object query plus ids. One refinement:$inon ids does not work for a multi-valued lookup on SQL, where the driver refuses$inon its JSON column. So the words name$containsper id there (measuredd1,d3on all three).having, relation and JSON columns do appear: a groupBy of the field, or amin/maxof it, carries that field's type (aggregatedRowColumnTypes). A nested-relationhavingkept no group on every driver before.havinghas no field declaration to read, so its relation words name "the related object" and the$inspelling.FilterConditionwithout narrowing another form. Its index signature isany | FieldOperators | FilterCondition, which TypeScript collapses toany. So removing theFilterConditionmember is a no-op, not a separation. At the schema, the nested-relation form and a JSON object comparand are the same shape (a plain object with no$key beneath a key), and only the column's declared type tells them apart. The genericFiltertype's nested arm (a recursiveFilterover an object-typed property's own type) is a separate union member. Removing it would narrowFilterfor every object-typed property, so it is aClause-②: yes (narrowing)change for its own card. ⛔ Neither is separated here.Where this departs from the order or the ruling (named, not silently chosen)
{ id: { a: 1 } }. Triage says theidrow "answers the door's existing unknown-field verdict on every driver". The Pins ruling says every row of the card's table answers the sameINVALID_FILTER. Onorigin/mainthe door has no unknown-field verdict that refuses. Its verdict for an undeclared key is tolerance (the engine's registry-less rule, pinned byGUARD an UNKNOWN field …), which leavesidto the drivers: memory 200 with no rows, SQL 400. Both readings cannot hold at once.idis not an unknown field by the engine's own definitions:find/findOneadd it to their known set, the write gate admits it (PLATFORM_PROVISIONED_COLUMNS), and so do the REST ingress (resolveQueryFields) and the per-aggregation reference names. So this PR judges the three platform-provisioned columns by the type they store, and only when the declared map omits them. That keepsid,created_atandupdated_atin the scalar words, keeps every other undeclared key tolerated, and makes the Pins row true. Reported to the PM as an open question.json.addressandcompositewere measured with the identical split (memory deep-equal rows, SQL 400). The spec publishes one class for them, so the arm judges the class, not one member of it. This is the bounded in-place fix: it is the same defect class, the same mechanical classification as the card, and the same file under this claim, and it adds no new gate family.$containsexample for a JSON field. It is not a route: the text-operator door refuses$containsover a JSON value on every driver (measured above). The JSON words name$nulland a stored field instead.filterwith a JSON object. It was the one cell that already answered alike on every driver (count 1, the engine's own deep equality). It is refused now, so that one filter has one answer at every position. The changeset names it.engine-nested-object-door.test.ts's recording driver by construction: the arm answers before a driver is resolved. The memory readings of the routes were measured (the table above) but are not pinned in a new suite:check:driver-memory-censusrefuses a new test consumer of the in-memory driver without a maintainer ruling. It caught a first draft that put one inpackages/runtime, which was dropped.Tests
The final HEAD is
ee17b18fde, a merge oforigin/maineead9dcf40into the branch.pnpm --filter @objectstack/objectql testonee17b18fde: 339 files / 6719 passed.pnpm --filter @objectstack/rest testonee17b18fde: 231 files / 4469 passed / 71 skipped.pnpm --filter @objectstack/spec testonee17b18fde: 577 files / 17007 passed / 1 todo.test:repoone08fd6883f: spec 45 files / 794, objectql 1 / 5, rest 1 / 8.typecheckfor objectql, rest and spec onee17b18fde: exit 0, including eachcheck:test-typecheckwith its ledger held.pnpm --filter @objectstack/spec check:generated: all 15 artifacts up to date.packages/objectql/src/engine-nested-object-door.test.ts(15 tests). It covers every relation type (single and multiple, with the route per multiplicity and the related object's name), every structured-JSON type, the provisionedid,{}, every verb and the judge,$and/$or/$notand sugar, the per-aggregation filter,having(a lookup groupBy, amaxof a master-detail, a json groupBy), the three REST doors intofindData, the controls (the routes, a file field, an unknown key), and the classification GUARDs over everyFieldType.packages/rest/src/data-nested-object-door.test.ts. SQLite always runs; PostgreSQL and MySQL run whereOS_TEST_POSTGRES_URL/OS_TEST_MYSQL_URLare set. It covers every row of the card's table, and the route asserted inside the REST body (so it must land inside the 500-character bound). It also covers the per-aggregation filter andhaving, the routes answering the rows the nested form meant ($inon the related object's ids givesd1,d3;$containson a multiple lookup givesd1,d3; tree givesd2,d3), and the file control. Local run with a live PostgreSQL 16.13: 8 passed (sqlite 4, live postgres 4) / 4 skipped (mysql, no URL).@objectstack/rest, so the live cells run only locally.$key as a scalar field's filter value answers 200 with no rows on the memory driver andINVALID_FILTER400 on SQLite and PostgreSQL #20546 pins (engine-no-operator-object-door.test.ts,data-no-operator-object-door.test.ts) keep a file field as their only control. Two name-gate controls pinned "the nested-relation form still passes the doors":protocol-explicit-filter-field-gate.test.ts(Data query: an unknown field insidewhere/$filteranswers 200/0 instead of400 INVALID_FIELD— the bare-key door disagrees (#4134's uncovered sibling) #7534) andquery-expression-conformance.test.ts([finding] The FILTER axis has no DOTTED-path verdict —where: { project_id.name: 'x' }rides its head segment past both doors, where SORT refuses the same spelling (#4256) #8371). They now pin what they were for: the answer is the engine'sINVALID_FILTERin the nested-relation words, never the name gate'sINVALID_FIELD....@objectstack/objectqldirection), one08fd6883f:@objectstack/metadata-protocol: 190 files passed, 3 skipped / 2792 passed, 19 skipped.@objectstack/service-analytics: 140 / 3266 passed. Its nested-relationwhereis flattened to cube members before any engine call, and it passed unchanged.@objectstack/plugin-security: 147 files / 3202 passed, 23 skipped.Reverse verification (ablation), from the committed fix. It ran through
scripts/ablation-replace.mjsin WRAP mode, trap-restored. The walk's gateif (facts.column !== null && isNoOperatorObject(value)) {was narrowed back to the #20546 behaviour withfacts.column.kind === 'scalar' && facts.column.provisioned !== true && facts.column.type !== '__ablated_20745__'. On disk the anchor went 1 → 0 and the marker 0 → 1, with blobea19d1959255→3134e87031b1. Then objectql was rebuilt, andablation-dist-preflightfound the marker in 4 built files.$key as a scalar field's filter value answers 200 with no rows on the memory driver andINVALID_FILTER400 on SQLite and PostgreSQL #20546 scalar pin stayed green.whereand aggregate refusals failed on SQLite and live PostgreSQL. The routes, the controls and the [finding] a plain object with no$key as a scalar field's filter value answers 200 with no rows on the memory driver andINVALID_FILTER400 on SQLite and PostgreSQL #20546 file stayed green.ea19d1959255),git diff HEADis empty, and whole-treegit status --porcelainis empty. After a rebuild, the--absentpreflight found the marker absent from all 14 built files. Then both pin sets were green again (249 passed; 16 passed + 8 skipped).Gates
node scripts/pm/dispatch-gates.mjs --commandsatee17b18fde(aftergit fetch, so not stale) derived 110 commands. All 110 were run onee17b18fde.--ranreconciles them: 110 derived, 110 run, 0 NOT-MEASURED, 0 UNRUN, and all exit 0. Among them are these gates:check:adr-0087-registration --base origin/main:not-required (no-migration-prescription)accepted.check:changeset-no-major,check:empty-changeset,check:doc-authoring,check:nul-bytes.check:engine-double-contract(904 pinned),check:where-matcher(440 matchers, 0 silently wrong),check:driver-memory-census,check:cross-package-test-inputs,check:test-source-alias,check:type-check-coverage,check:type-check-debt,check:query-options-erasure.check:dual-build-cjs-loads(105 entry points, 66 packages).check:api-surface/check:docs/check:authorable-surface/check:skill-examples.Lint, narrowed and proven:
pnpm exec eslint --no-inline-config --format jsonover the 11 changed.tsfiles atee17b18fdefound 11 files, 0 errors, 0 warnings. Three facts make this narrowing a measurement:isPathIgnoredanswersfalsefor all 11.parserOptions.projectandprojectServicearenullfor every file, so type-aware linting is not enabled.Changesets
.changeset/20745-nested-object-door.md:@objectstack/objectqlminor, a BREAKING banner,Clause-②: no (narrowing), and the ADR-0087 markernot-required (no-migration-prescription), as the [finding] a plain object with no$key as a scalar field's filter value answers 200 with no rows on the memory driver andINVALID_FILTER400 on SQLite and PostgreSQL #20546 changeset has. It states what an author sees now and the route that works, per kind, with the table. It says it supersedes the "Unchanged" paragraph of the pending scalar-field entry (20546-no-operator-object-on-scalar) for relation and structured-JSON fields..changeset/20745-nested-relation-prose.md:@objectstack/specpatchfor the shipped JSDoc.@objectstack/objectql's root and./coreexports are unchanged.check:api-surfaceis green.Acceptance notes
skills/objectstack-queryteaches the nested-relation form as working:SKILL.md"Nested Relation Filters", the "Filter parent by child conditions" row and thesearchparagraph, andrules/filters.md"Nested Relation Filters". It was already untrue before this change (memory answered no rows, SQL 400). It is a governed Tier H surface outside this claim's file surface, so it is not touched here.groupByof ajsonfield answers 500DATABASE_ERRORon PostgreSQL. On InMemoryDriver it merges every row into one group (n: 3), and on SQLite it gives one group per serialized value.where: { project_id.name: 'x' }rides its head segment past both doors, where SORT refuses the same spelling (#4256) #8371 carve-out and stay unjudged.{ photo: { url: 'x' } }still answers memory 200 with no rows and SQL 400.havingthere is no field declaration, so a relation column's words say "the related object" and give the$inspelling. Ahavingover a multi-valued relation groupBy would get that single-valued spelling.referenceTargetOfnames the related object in the relation words. For a field whosereferencecarrier never went through the schema's parse (a non-string), it throws its ownTypeErrorinstead of the refusal. Parse refuses that shape at the contract door.Generated by Claude Code