Repository navigation
Commit 8d329f0
Fixes #20887
Clause-②: yes (narrowing)
The analytics half of ruling 5907789183, whose parent card is #20802
(its engine half landed as #20872, `ca5408c62`). The nested-relation
filter `{ relation: { field: value } }` now gets ONE answer on every
analytics face, and it is the engine's: the related object read as the
caller (its row scope and field permissions), capped at
`RELATION_FILTER_ID_CAP`, a multi-valued relation matching on any
member. The analytics layer holds no copy of that rule. The native-SQL
strategy declines a query carrying the form, and the engine-aggregate
strategy hands the form to the engine as written.
## Per face
The engine's answer for the same filter, computed in the same test over
the same rows, is the reference for every cell. Fixture: a ledger with
`owner` (lookup) and `owners` (multiple lookup) to an owner object; the
member cannot read `owner.secret`, and its row scope hides owners in
region `HIDDEN`. Past the cap means 1,001 matching owners. Measured with
the real `SecurityPlugin`, `ObjectQL` and `SqlDriver` (SQLite).
| face | strategy | engine rows equal | caller permissions | cap |
|---|---|---|---|---|
| cube read, `POST /api/v1/analytics/query` (`AnalyticsService.query`) |
NativeSQL composition (declines, so the engine answers) | yes: single,
multi, related row scope, `$not`, `$or` (was 500 `DATABASE_ERROR`: the
join named a table `owner` that does not exist) | 403
`PERMISSION_DENIED`, as the engine (was 500) | 400 `INVALID_FILTER`, as
the engine (was 500) |
| cube read | ObjectQL | yes (was 400 `INVALID_FIELD`, "cannot evaluate
a cross-object filter") | 403 (was 400) | 400 (was 400 cross-object) |
| dataset door, `POST /api/v1/analytics/dataset/query`, the dataset
`include`s `owner` | NativeSQL composition | yes (was: single-valued
rows via the JOIN; multi-valued 400 `DATASET_INVALID`; under `$not` the
member got `b` where the engine answers `b, d`) | 403 (was **200 with
rows a, c**: filtered by a field the caller cannot read) | 400 (was 200
with no rows) |
| dataset door | ObjectQL | yes (was 400) | 403 (was 400) | 400 (was
400) |
| a measure's own `filter` carrying the form | both | refused 400
`INVALID_FILTER`, as the engine refuses it at an aggregation's `filter`
(was: native counted it through the JOIN; ObjectQL 400 `INVALID_FIELD`)
| n/a | n/a |
| SQL echo, `POST /api/v1/analytics/sql` | both | refused 400
`INVALID_FILTER`, naming the served route (was: native printed the JOIN;
ObjectQL 400) | n/a | n/a |
| a read scope carrying the form (a host `getReadScope`) | NativeSQL |
refused 500 `READ_SCOPE_COMPILE_FAILED`, policy withheld, words now
naming the route (outcome unchanged) | n/a | n/a |
| a read scope carrying the form | ObjectQL | served, the engine's rows
(unchanged: the scope reaches the engine as written) | as the caller |
the engine's |
## Mechanism assumptions, measured
- **B1 held.** The engine's answer for the fixture, as the member: `{
owner: { region: 'NA' } }` is d1, d3; the multi-valued form is d1, d3;
`{ owner: { secret: 's1' } }` is 403 `PERMISSION_DENIED` naming `secret`
(a system caller gets d1, d3); region `HIDDEN` gives no rows (a system
caller gets d4); past the cap is 400 `INVALID_FILTER` for both
spellings; `$not` gives d2, d4; the `$or` gives d1, d2, d3; `{ owner: {}
}` and a second level are 400. `RELATION_FILTER_ID_CAP` is exported
(`packages/objectql/src/index.ts:148`), and nothing here imports it: the
analytics layer never counts ids, the engine does.
- **B2: no on every axis**, per the table. The native path joined the
related table itself: the related row scope rode in as a `WHERE`
conjunct, the field permissions did not, nothing bounded the match, and
a multi-valued relation or an undeclared join failed. The ObjectQL path
refused the form outright.
- **B3: call the engine.** `@objectstack/objectql` exports only the cap.
The lowering (`admitRelationCondition`, `lowerRelationSite`) is
module-internal, and this package has `@objectstack/objectql` as a dev
dependency only. The route that needs no export: the engine-aggregate
strategy hands the condition to `engine.aggregate` through
`executeAggregate`, with the caller's context. No export was needed, and
there is no second permission rule.
- **B4: the read scope keeps its refusal, and the words name the served
route.** `compileScopedFilterToSql` is a synchronous string builder. It
holds the caller's `ExecutionContext` for placeholders only, and no data
engine, so its compile cannot run the inner read as the caller. Routing
a read scope carrying the form to the engine instead was built and
measured, then withdrawn: on a native-only host it traded the declared
`READ_SCOPE_COMPILE_FAILED` (policy withheld) for a generic no-strategy
fault (`packages/rest/src/analytics-read-scope-refusal-envelope.test.ts`
went red). No in-repo producer emits the form in a scope: the RLS
compiler refuses a relation traversal when it compiles the policy. On
the ObjectQL path the scope reaches the engine as before, and the engine
serves it as the caller.
- **B5: `yes (narrowing)`.** Widening: the cube read (both strategies),
the ObjectQL dataset door, a multi-valued relation and a dataset without
the declared join on the native path, and a dataset's own `filter` on
the ObjectQL path all now serve the form (they answered 500 or 400).
Narrowing: on the native path the dataset door now refuses a condition
on a related field the caller cannot read (was rows), a match past the
cap (was an empty 200), and a measure filter carrying the form (was a
count). The SQL echo refuses the form. And a query combining the form
with something only the native strategy serves (a cross-object measure,
a multi-hop dimension) is refused by the engine-aggregate path.
`@objectstack/service-analytics` ships `minor` with the BREAKING banner
and an ADR-0087 `not-required (no-migration-prescription)` disposition;
`check-adr-0087-registration` and `check-changeset-no-major` pass.
- **B6: no page to update.** No hand-written `content/docs/**` page
states how the analytics read or the read scope treats the nested form.
`data-engine.mdx`, and `query-syntax.mdx` (#20906, which landed during
this work), describe the engine only.
## What changed
- `strategies/filter-normalizer.ts`: a nested-relation condition becomes
a `relation` node carrying the condition as written. It is no longer
flattened to the dotted member. `shieldNestedRelations` holds it out of
the shared lowering, because under `$not` the lowering guarded the
relation column, and this package's engine hand-off spells that guard
`$ne: null`, which `driver-sql` refuses over a multi-valued JSON column.
Measured: the multi-valued `$not` pin went red before the shield, and
the engine guards what it lowers the condition to itself.
`findNestedRelationCondition` is the routing detector.
- `strategies/native-sql-strategy.ts`: `canHandle` declines when the
`where`, the dataset's own `filter` or a requested measure's `filter`
carries the form. This is the mechanism of the cross-field decline
(maintainer ruling 2026-08-12, Q1 = B). Its compiler refuses a
`relation` node bare, as routing drift.
- `strategies/objectql-strategy.ts`: the condition goes to the engine as
its own conjunct, under the key the author wrote. The display-SQL echo
declines it.
- `read-scope-sql.ts`: the nested-relation form's refusal has its own
words, naming the route. An empty or mixed value object keeps the old
words.
- `analytics-service.ts`: the no-strategy error names the
nested-relation decline.
- The mixed-wrapper refusal no longer says a nested member "compiles to
the dotted member".
## Pins, red first (`568727629`)
- `packages/rest/src/analytics-nested-relation-filter.test.ts`: both
compositions, the cube read and the dataset door through its route,
against the engine's answer. It was red 10 of 10 on the base, and is 10
of 10 green now.
-
`packages/services/service-analytics/src/__tests__/nested-relation-engine-handoff.test.ts`:
the native decline per producer, the ObjectQL hand-off as written, the
compile backstop and the read-scope words. It was 7 red with 2 controls
green on the base, and is 9 of 9 green now.
## Ablations, predicted before running, at `5bb764181`
Each ablation mutated the committed file through
`scripts/ablation-replace.mjs` (anchor hit once, blob moved), rebuilt
`@objectstack/service-analytics`, and passed `ablation-dist-preflight`
(the marker present in 2 built files). It then ran both pin files, plus
`where-door-shared-lowering-seam.test.ts` in the unit run. The restore
leg proved the blob equal to HEAD and `git diff HEAD` empty, rebuilt,
and found the marker absent from all 6 built files. Every observed count
equals its prediction.
| ablation | face it guards | unit (27) | route pins (10) |
|---|---|---|---|
| A1 the native decline removed | cube read and dataset door, native | 4
red | 4 red (native: rows, refusals, measure filter, echo) |
| A2 the hand-off drops the condition | both strategies' rows,
permission, cap | 2 red | 6 red |
| A3 the aggregate call forwards no caller context | caller permissions
| 0 | 4 red: the member then saw `d` (a hidden owner's row), and the
unreadable field answered rows |
| A4 the read-scope route words | read scope, native | 1 red | 1 red |
| A6 the lowering shield removed | multi-valued `$not` | 1 red | 2 red |
| A7 the echo refusal removed | SQL echo | 0 | 2 red |
A first round at `dca1af7cb` matched its own predictions too, including
A5, the read-scope decline arm, which B4's correction removed from the
code.
## Pins re-judged
These pins recorded the flattening this change removes, so each was
re-judged:
- respelled to the dotted cube member where the pin was about the
traversal: `filter-normalizer-not-null-safe`,
`icontains-text-comparand-refusal`;
- re-expected as a `relation` node where the pin was about acceptance:
`where-equality-slot-list-refusal`, `where-face-arms-refusal`,
`where-type-face-refusal`, `filter-normalizer-mixed-wrapper`'s
pure-shape block;
- replaced where the pin held the removed branch: mixed-wrapper row 6
and its `guardFieldEntry` recursion row (the engine refuses that inner
wrapper, `INVALID_FILTER` / 400, measured),
`where-door-shared-lowering-seam`, `infer-cube-relation-traversal`,
`infer-cube-where-spelling-parity`, and `where-source-field-gate`, which
now judges the relation field `owner` as a column of the queried object;
- re-worded to the read-scope refusal's new words: `read-scope-sql`,
`read-scope-not-null-safe`, `read-scope-undefined-comparand`, and
`read-scope-refusal-envelope`, which gains row 17 because the
nested-relation form now has a throw site of its own.
## Verification
- `@objectstack/service-analytics`: `test` 146 files, 3334 passed;
`typecheck` exit 0, with 146 of 146 test files in the tsc program
(`--listFiles`). Both at `4d383dac0`, after merging `main`.
- `@objectstack/rest`: the full `local` project, 239 files, 4662 passed
and 106 skipped, at `5bb764181`. The merge brought no rest or analytics
change. At `4d383dac0`, the new pin, the read-scope envelope pin and the
engine half's permission pin: 3 files, 21 passed. `typecheck` passes,
including the test layer (`check:test-typecheck` OK).
- Consumer sweep, narrowed to the files that load this package:
`@objectstack/runtime` `analytics-*` plus
`cross-field-refusal-operand-withhold`, 5 files, 38 passed and 4
skipped; `@objectstack/client` `analytics-automation-json-erasure`, 7
passed.
- Gates at `4d383dac0`: `dispatch-gates --commands` derived 62. All 62
were run, plus the 4 roster families (`check-changeset-fixed`,
`check:authz-resolver`, `check:error-code-casing`,
`check:filter-alias-parity`), all exit 0. `dispatch-gates --ran`: 62
derived, 62 run, 0 NOT-MEASURED, 0 UNRUN. `check:dual-build-cjs-loads`
and `check:type-check-debt` first exited 3 (prerequisite not met) and
were re-run green after `turbo run build` over `./packages/*`.
- Lint, narrowed and proved at `4d383dac0`. The population is the 21
changed `.ts` files, none ignored by eslint's own config
(`isPathIgnored` false for all 21). `eslint --no-inline-config --format
json` over them gives 21 files, 0 errors, 0 warnings.
`parserOptions.project` and `projectService` are unset for all 21, so no
type-aware lint runs and no untouched file's verdict can move.
- NOT MEASURED: a live PostgreSQL cell (the dialect axis of the lowering
is the engine's, pinned by #20872's `data-nested-object-door.test.ts`;
this file's axis is the analytics faces), the dogfood and integration
lanes, and the whole-workspace typecheck. All are left to CI.
## Acceptance notes
- The ObjectQL path's cross-object refusal still says "Run this query on
a native-SQL driver". A query the form routed away from the native
strategy can meet those words on a SQL deployment.
- The engine's cap refusal names the position the engine received. The
strategy ANDs the condition in, so the words read `where.$and[0].owner`
where the author wrote `where.owner`.
- `packages/types/src/error-leak.test.ts` keeps a hand-written stand-in
of the read-scope refusal shapes. Its nested-relation line is the old
wording. It is a heuristic fixture, not a pin of this module, and it
stays green.
- `MemoryAnalyticsService` (driver-memory's cube face, #20859's
position) is not touched, and its answer for the form is not measured
here. The draft preview refuses the form as an operator it cannot
evaluate, unchanged.
## Patch rounds (the seat's append from the dev's reports `5917211340`,
`5917807320` and `5918233769`; the dev writes a body only once)
### Patch round 1
`Test Core (3/6)` went red on `4d383dac0`, in `packages/client`'s
`envelope-caller-census.test.ts`: 2 of its tests failed. Reproduced
locally: the client suite fails 1 file and 2 tests at `4d383dac0`, and
passes 50 files and 641 tests at the merge base `9509ea106`.
**Root cause.** The census walks the whole workspace for call sites of
`analytics.query(` and requires a hand-ledger row for each one. This
PR's new pin,
`packages/rest/src/analytics-nested-relation-filter.test.ts`, calls the
real `AnalyticsService`'s `analytics.query` five times. Those are
producer reads, the census's `NOT_SDK` class, and the ledger has no row
for them. The failing assertions are the §3 key comparison (the one
extra key is that file, `analytics.query`, `service`, 5) and the §2
producer-receiver count (1 expected, 6 found).
**What it is not.** It is not a product defect. It is not a client pin
of the nested-relation form or of the read-scope wording either.
**The fix, pending the seat.** It lives in
`packages/client/src/envelope-caller-census.test.ts`, which is outside
this card's claim surface. It adds one `NOT_SDK` ledger row with a count
of 5, and moves the two producer-read counts from 1 to 6. Measured on a
scratch copy of that file: 20 of 20 tests passed. The copy was restored
byte-identical, and nothing was committed.
### Patch round 2
The seat authorised the census remedy, round 1's option A, for one file:
`packages/client/src/envelope-caller-census.test.ts`.
- **main moved.** `975b2481c` (#20808) touched
`packages/services/service-analytics`, so `origin/main` was merged into
the branch as `1fdaff7e5` (no rebase). The merge was clean, with no
regeneration pending.
- **The census change is its own commit, `7eb2ecf20`.**
- It adds one `NOT_SDK` ledger row for
`packages/rest/src/analytics-nested-relation-filter.test.ts`
(`analytics.query`, `service`, count 5).
- The producer-receiver count goes from 1 to 6, and the set of two files
is asserted.
- `verdictTotal('NOT_SDK')` goes from 1 to 6, and its test title changes
with it.
- Nothing else in that file changed.
- **Measured at `7eb2ecf20`.** Each exit code was captured before any
pipe.
- `pnpm --filter @objectstack/client test`: exit 0, 50 files and 641
tests passed. The census file run alone passed 20 of 20.
- `pnpm --filter @objectstack/client typecheck`: exit 0. `tsc --noEmit`
passed, and `check:test-typecheck` answered OK.
- `@objectstack/service-analytics` test: exit 0, 146 files and 3335
tests passed. Its typecheck: exit 0.
- The three rest files (`analytics-nested-relation-filter`,
`analytics-read-scope-refusal-envelope`,
`data-nested-relation-permission`): exit 0, 3 files and 21 tests passed.
- ESLint over the 22 changed `.ts` files: 0 errors and 0 warnings. The
config ignores none of them and lints none type-aware, so this diff
cannot move a verdict on an untouched file.
- Gates, re-derived: 63 derived and 63 run, 0 not measured, plus the 4
roster families. All exit 0 except one.
- **The one red is `check:cross-package-test-inputs`.**
- Cause: the new ledger row spells the rest pin's path as a literal, and
`@objectstack/client`'s declared cross-package input globs do not cover
it. The gate is green at `1fdaff7e5`, the commit before.
- The gate's own remedy: declare that one file in
`scripts/cross-package-test-inputs.mjs`, and mirror it in `turbo.json`'s
`@objectstack/client#test` inputs.
- Measured on the working tree: that gate and `check-ci-filter-parity`
both exit 0. The two files were then restored byte-identical.
- Both files lie outside the authorised surface, so the remedy waits for
the seat.
### Patch round 3
The seat authorised the gate's own remedy for
`check:cross-package-test-inputs`, in two files.
- **main.** No commit since `975b2481c` touched this card's surface, the
census or either of the two files, so there was no merge. The commits
checked were `def279a39`, `4d0b9cd54` and `d78a0bda0`.
- **The declaration is its own commit, `2881f478c`.**
- `scripts/cross-package-test-inputs.mjs`: in `@objectstack/client`'s
entry, one per-file glob,
`packages/rest/src/analytics-nested-relation-filter.test.ts`, with a
3-line comment.
- `turbo.json`:
`$TURBO_ROOT$/packages/rest/src/analytics-nested-relation-filter.test.ts`
in `@objectstack/client#test`'s inputs. The line before it gains the
comma JSON requires.
- Nothing else changed in either file.
- **Measured at `2881f478c`.**
- Gates, re-derived: 81 derived and 81 run, 0 not measured. Also run:
the 11 roster families whose roster lies under a path this diff touches,
`check-ci-filter-parity --self-test` and `check:select-shard-packages`.
All 93 commands exit 0.
- `check:cross-package-test-inputs` (with `--self-test`) is green: "OK:
29 package(s) read outside themselves, all declared".
- `check-ci-filter-parity` is green: "all 188 declared cross-package
glob(s) (135 unique) are covered".
- `check:turbo-task-graph` is green.
- `pnpm --filter @objectstack/client test`, as the control: exit 0, 50
files and 641 tests passed, the same as at `7eb2ecf20`. The declaration
moved no verdict.
- Layer A at work: `--union-into`, given a diff of the rest pin alone,
now pulls `@objectstack/client` into the run (8 packages). At
`7eb2ecf20` it did not (7 packages). No other package changed.
- Turbo hashes, from `--dry=json` before and after, over build, test,
test:repo and typecheck (303 tasks):
- The global hash is unchanged, and no build or typecheck hash moved.
- 7 test hashes moved. `@objectstack/client#test` moved through
`turbo.json`: its task definition changed, and the rest pin is a new
input.
- The other 6 moved only because the content of
`scripts/cross-package-test-inputs.mjs`, an input they declare, changed.
They are `cli#test`, `plugin-auth#test`, `vitest-filter-preflight#test`,
`objectql#test:repo`, `runtime#test:repo` and `spec#test:repo`.
- ESLint over the 23 changed `.ts` and `.mjs` files: 0 errors and 0
warnings. The config ignores none of them and lints none type-aware.
### Patch round 4 (the seat's append from the dev's report `5922062971`)
`main` was merged (no rebase) to take in four landings in
`service-analytics`:
- PR #20931: the field-read gate at the door.
- PR #20955: the queryable-field gate.
- PR #20954: `plugin-security`'s comparand guard.
- PR #20962: relationship-path objects in the admitted and scoped set.
**The merge, `6b6bffb3e`.** It is clean at the text level, in
`analytics-service.ts` and in `native-sql-strategy.ts`. Every line
either side added is present in the merged files, checked line by line.
**What the landed gate and object set do with the `relation` node.**
This was measured on the merged tree, in the shipped composition (the
real `SecurityPlugin` over `ObjectQL` on SQLite), under both strategies.
- **The gate judges the relation field.** `collectFilterLeaves` yields
the nested form's relation field as its member: `{ owner: { region: 'NA'
} }` gives `owner`, with operator `relation`. It does the same under
`$not` and inside `$or`. So the field gate judges the relation field on
the base object.
- A caller who may not read `owner` is refused by the gate: 403
`PERMISSION_DENIED`, in the engine's own words, with no engine call
made.
- **The related object does not enter `queryObjects`.** The security
service is asked only about the base object.
- **The engine guards the related object instead.** The nested form is
served on the ObjectQL path, where the engine reads the related object
as the caller. Each of these is refused with the same code, status and
words as `engine.find`, and never answered:
- a related field the caller cannot read: 403;
- a masked related field (#20935): 403;
- a related object the caller cannot read at all: 403.
- **Before this branch, the answer was a refusal.** On `main` alone,
even a readable nested condition was refused 403, "reading "owner" is
not permitted", because the flattened `owner.region` named the relation
field as an object to admit. With this branch, the answer is the
engine's.
**Measured at `6b6bffb3e`.** Every run was under the shared lock, with
each exit code captured before any pipe.
- `@objectstack/service-analytics`: tests exit 0 (149 files, 3434
tests), and typecheck exits 0.
- The five rest route pins pass 61 of 61:
- `analytics-nested-relation-filter`: 10
- `analytics-read-scope-refusal-envelope`: 8
- `data-nested-relation-permission`: 3
- `analytics-field-permission-gate`: 12
- `analytics-relationship-path-admission`: 28
- The client census passes 20 of 20.
- Gates, re-derived and run as one locked sequential script: 81 derived,
81 run, 0 not measured. Also run: the 11 roster families and 2 extras.
All 93 commands exit 0.
- ESLint over the 23 changed `.ts` and `.mjs` files: 0 errors and 0
warnings.
**Acceptance notes.**
- **A dotted path on an inferred cube is still refused.** `{
'owner.region': 'NA' }` is refused 403, "reading "owner" is not
permitted". The hop object is taken from the alias, because an inferred
cube declares no join. This is the same on `origin/main`, and it is
outside this card; it went to the seat as a finding.
- **A host read scope does not reach the related object.** The related
object is not in `queryObjects`, so a host-supplied `getReadScope` is
not asked about it. In the shipped composition that provider is the
security service's `getReadFilter`, the same row scope the engine
applies when it reads the related object as the caller. That case is
pinned in `analytics-nested-relation-filter` ("the related row scope").
---
_Generated by [Claude
Code](https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent dfe5a08 commit 8d329f0
25 files changed
Lines changed: 1145 additions & 202 deletions
File tree
- .changeset
- packages
- client/src
- rest/src
- services/service-analytics/src
- __tests__
- strategies
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
473 | 473 | | |
474 | 474 | | |
475 | 475 | | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
476 | 482 | | |
477 | 483 | | |
478 | 484 | | |
| |||
670 | 676 | | |
671 | 677 | | |
672 | 678 | | |
673 | | - | |
674 | | - | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
675 | 684 | | |
676 | 685 | | |
677 | 686 | | |
| |||
716 | 725 | | |
717 | 726 | | |
718 | 727 | | |
719 | | - | |
| 728 | + | |
720 | 729 | | |
721 | 730 | | |
722 | | - | |
| 731 | + | |
723 | 732 | | |
724 | 733 | | |
725 | 734 | | |
| |||
0 commit comments