Repository navigation
Commit 4727fcb
fix(service-analytics)!: refuse a JSON-stored dimension or count_distinct over a relationship path the cube declares no join for, located through the one hop resolver (#21247)
Fixes #21232
Clause-②: no (narrowing)
## What changed
The structured-JSON door (`structured-json-dimension-door.ts`) now
locates a dotted path's column through the one hop resolver, the way
every other reader in the package does.
- **`columnOf`** asks `columnObjectOf` (`hop-object.ts`, consumed
unchanged) with the host's hop reference. The answer is the cube's
declared join at that path, else the relationship field's declared
`reference`, else the alias. That is the object both strategies join and
read for the path. Before, `columnOf` read `cube.joins` alone and stood
down on a path the cube declares no join for. ⛔ No second resolver: the
hop walk is `hop-object.ts`'s, as for the measure side in PR #21230.
- **`assertNoStructuredJsonDimension`** takes one more argument,
`referenceOf` (the `HopReference` type from `hop-object.ts`). Its one
caller, `AnalyticsService.assertDimensionsGroupScalarColumns`, passes
`this.hopReference`. That is the same function the field gate, the
admitted and scoped set, and both strategies resolve a hop with. The
function is internal to the package (not exported from `index.ts`).
- The refusal words and the envelope are unchanged. A member over an
undeclared-join path now gets the refusal a member over a declared join
already got: `INVALID_FIELD` / 400, `member`, `param`, `cube`, `field`
(the path), `object` (the object the lookup declares as its target).
## Measured: `POST /api/v1/analytics/query` and `/sql` on the real
dispatcher route
Setup: `AnalyticsServicePlugin` over a real `ObjectQL` engine and
`SqlDriver`, a signed-in caller, the real `dispatcher-plugin` mount,
SQLite in memory and a private PostgreSQL 16.14. A configured cube over
`os21232_deal` declares a join for `account` (`hq` `json`, `name`
`text`) and none for `owner`. `owner` is a lookup whose `reference` is
`os21232_person` (`prefs` `json`, `labels` `tags`, `email` `text`).
Before: `origin/main` at `3a7b6eb0`. After: this branch's
`service-analytics` build. There are 80 cells (2 drivers x 2 faces x 2
doors x 10 members). 32 changed and 48 are byte-identical. The scratch
probe was deleted.
| member | face | SQLite before → after | PostgreSQL 16.14 before →
after |
|:--|:--|:--|:--|
| dimension `owner.prefs` (json) / `owner.labels` (tags) | native | 200,
one group per serialized value → **400 `INVALID_FIELD`** | **500
`DATABASE_ERROR`** → **400** |
| the same | ObjectQL | 400 `INVALID_FIELD` from the engine,
`groupBy[1]` → **400 from this door, naming the member** | the same |
| `count_distinct` over `owner.prefs` / `owner.labels` | native | 200,
`2` / `2` → **400** | **500** → **400** |
| the same | ObjectQL | 400, the cross-object refusal (no `field` /
`object`) → **400 from this door** | the same |
| any of the above on `/sql` (dry run) | both | 200 (native, and the
ObjectQL dimension) → **400** | the same |
| control: the same members over `account.hq` (declared join) | both |
400 `INVALID_FIELD`, this door → unchanged | unchanged |
| control: `owner.email` dimension | both | 200 → unchanged | unchanged
|
| control: `owner.email` / `account.name` count_distinct | native /
ObjectQL | 200 / 400 cross-object refusal → unchanged | unchanged |
## Pins
**New file:**
`packages/services/service-analytics/src/__tests__/json-stored-door-undeclared-join.test.ts`.
It uses the plugin's own composition over a real engine, a SQLite cell
and a PostgreSQL cell (a named skip without `OS_TEST_POSTGRES_URL`), and
both faces. It has 10 tests, 5 per cell.
- A dimension and a `count_distinct` over `owner.prefs` / `owner.labels`
are refused `INVALID_FIELD` / 400 on both faces, with nothing read. The
checked fields are `code`, `status`, `member`, `param`, `cube`, `field`
(the path) and `object` (`os21232_person`), and the raw-SQL and
engine-aggregate counters stay at 0. The two faces' envelopes must be
equal. Neither face's own refusal carries this envelope, so equality
shows the door answered.
- An ad-hoc query's inferred cube declares no join at all. A dotted
dimension on it (`owner.prefs`) is refused the same way.
- The dry-run door refuses what the query door refuses.
- Controls: the same members over the declared join `account.hq` get the
same refusal (`object` the joined object). The scalar `owner.email`
dimension is served on both faces (one group per owner), and its
`count_distinct` answers `3`.
**Unit file** `dimension-structured-json-door.test.ts`: a new block with
5 tests. The relationship field's declared reference names the object,
on both faces (`crm_account`, not the alias). The reference wins over an
alias that names a described object (the door stands down and the
statement joins `"crm_account"`). A host with no reference resolves the
alias, and both doors refuse there. Control: the referenced object's
text column is served.
## Ablations
The tests import the subject by relative path
(`../analytics-service.js`, `../plugin.js`), so each run reads `src` and
there is no `dist` leg. Every mutation went through
`scripts/ablation-replace.mjs` in WRAP mode, with an outer `trap`
restore on `EXIT INT TERM` against the absolute path. Predictions were
written before each run. They ran from committed `08ea135b`, and `git
diff 08ea135 b6e6418 -- packages/services/service-analytics` is empty.
The PostgreSQL cell was live.
| ablation | mutation | predicted | observed |
|:--|:--|:--|:--|
| A1 | `columnOf` gets back the joins-only resolution (the removed code,
byte for byte) | 9 red: the undeclared-join, ad-hoc and dry-run tests on
each cell, the two reference-tier face tests and the alias-tier test |
**9 failed / 19 passed** |
| A2 | the call site passes `undefined` in place of `this.hopReference`
| 9 red: the same six live tests, the two reference-tier face tests, and
"the reference, not the alias" (the alias tier stays green) | **9 failed
/ 19 passed** |
The prediction's total (30) was an arithmetic slip: 28 tests ran. Each
mutation landed: anchor 1 → 0, and the blob changed (A1 `04bf4095e712` →
`db237bdc284e`, A2 `16d63d721b6e` → `5d257e0a59d3`). Each was restored
and proven: the blob equals the HEAD blob, `git diff HEAD` is empty, and
porcelain shows 0. An earlier pair of runs at `50d5511f`, before the
ad-hoc pin existed, gave 7 failed / 19 passed for each, as predicted
then.
## Fixture triage
The full `service-analytics` suite turned up exactly one fixture that
pinned the removed stand-down: `dimension-structured-json-door.test.ts`,
"a dotted path the cube declares no join for is a synthetic traversal …
not judged". It pinned the branch this PR deletes, so it was replaced,
not respelled. Its stand-down now has an honest reason, a host that
describes nothing on the object the hop reaches (`describes: null`), and
the new block above pins the judged tiers.
Consumer radius: the analytics fixtures with dotted members in
`packages/rest` (8 files), `packages/runtime` (4) and
`packages/driver-memory` (3) were run against this branch's build. They
gave 107 passed / 3 skipped, 24 / 10 and 239 / 0, with no failures.
## Other readers of `cube.joins` in `service-analytics` (dispatch Zone
2, item 2)
No other reader resolves a dotted path through `cube.joins` alone. Every
path-splitting reader goes through `resolvePathHops` / `columnObjectOf`.
Six sites enumerate `cube.joins` without splitting a path. They are
listed for the seat and not touched here.
- `strategies/native-sql-strategy.ts` `qualifyAndRegisterJoin`,
`canJoin`: it qualifies a bare base column only when the cube declares a
join. **Measured** at `b6e64185` (this file is byte-identical to
`origin/main`). Take a configured cube with no declared join and the
dimensions `note` + `owner.email`, where the lookup's target also
declares `note`. The native face answers **500 `DATABASE_ERROR`** on
SQLite ("ambiguous column name: note") and PostgreSQL (42702). The
ObjectQL face answers 200. This is reported to the seat as a finding and
is not handled in this PR.
- `strategies/native-sql-strategy.ts` `canHandle`, the federated-object
decline: it asks `isExternalObject` of the declared join targets only,
not of an object reached through a path with no declared join. Reach not
measured.
- `native-sql-strategy.ts` (three sites) and `analytics-service.ts`
`cubeObjects`: these read the declared joins as a fallback for a context
built without `readScopedObjects`, or beside `namedQueryFields`, which
adds the path-reached objects. No defect was found.
## Docs
`git grep -nE "declares no join|declared join|structured-JSON|structured
JSON|count_distinct"` over `content/docs/**`, excluding `releases/` and
`references/`, gave 32 hits. None says the door stands down on a path
without a declared join, and none speaks about a `count_distinct` over a
related field's JSON-stored column. Positive control: the pattern hits
`content/docs/deployment/validating-metadata.mdx:228`, the
dataset-dimension door sentence. That page speaks about datasets, whose
dotted fields must traverse a declared `include` (a declared join), and
it stays true. No page edited.
## Deviation from the declared file surface
The claim names `structured-json-dimension-door.ts` (`columnOf`) and its
tests. Routing `columnOf` through the resolver's reference tier, which
is the triage direction, needs the host's `HopReference`, and only the
door's one caller holds it. So `analytics-service.ts` changes by one
argument (`this.hopReference`) and three docblock lines in
`assertDimensionsGroupScalarColumns`, the door's caller. Nothing else in
that file moved. A2 above pins that line.
## Verification at `b6e64185`
The branch head `fafbf053` carries a tree byte-identical to `b6e64185`
(`git diff b6e6418 fafbf05` is empty), so every reading below holds
for it. `b6e64185` has `origin/main` (`ef96c9ed`) merged in. Install and
a full build were refreshed after the merge, and `pnpm --filter
@objectstack/spec check:generated` reports all 15 artifacts up to date.
- `pnpm --filter @objectstack/service-analytics test`: 164 files, 3758
passed / 56 skipped (the live-PostgreSQL cells), 0 failed. `typecheck`
(`tsc --noEmit`): exit 0.
- PostgreSQL 16.14 live (`OS_TEST_POSTGRES_URL`):
`json-stored-door-undeclared-join`, `json-stored-door-live-drivers`,
`cube-measure-relationship-path-type`, `dimension-structured-json-door`
and `multi-value-json-stored-door` gave 72 passed / 0 skipped. `runtime`
`analytics-json-dimension-door` and
`analytics-cube-measure-field-type-door` gave 20 passed / 0 skipped (run
at `9a32e950`, whose analytics source is the same).
- Gates: `node scripts/pm/dispatch-gates.mjs --repo
objectstack-ai/objectstack --commands` derives 62 commands, and all 62
exit 0 at `b6e64185`. `--ran` reconciliation: "62 derived, 62 run, 0
NOT-MEASURED, 0 UNRUN", with every exit code recorded.
`check:adr-0087-registration` was red once, on the changeset's "FROM →
TO" label, which the gate reads as a rewrite prescription. The section
describes behaviour, not a rewrite, so it was relabelled "Before and
after". The gate is now green with `not-required
(no-migration-prescription)`, the disposition the door's earlier entries
carry.
- Lint, a declared narrowing: `eslint --no-inline-config --format json`
over the 4 touched `.ts` files at `b6e64185` reports 4 files, 0 errors
and 0 warnings. ① All 4 are inside the population `eslint.config.mjs`
lints (`packages/**/*.{ts,tsx,mts,cts}`; none ignored), and the
changeset `.md` is in no `files` glob. ② The count of 4 is read from the
JSON output. ③ The config enables no type-aware linting
(`--print-config` gives `parserOptions` `{ecmaVersion:'latest',
sourceType:'module'}`, with no `project`), so this diff cannot move a
verdict on an untouched file. The repo-wide `pnpm lint` is left to CI.
## Acceptance notes
- **Carrier note for release compilation (not filed).** The two pending
release notes for this door, `20807-analytics-json-dimension-refused.md`
and `20912-analytics-multi-value-distinct-refused.md`, list a dotted
path the cube declares no join for as Unchanged. This PR makes that
clause untrue, and its own changeset states the reversal. When the
CHANGELOG is assembled, the release compiler should drop that clause
from both entries. They cannot be corrected from this PR: editing
another PR's pending changeset is a foreign-changeset edit that stays
refused until a person confirms it (#17712).
- The ObjectQL face's own refusals of these members (the engine's
`groupBy[1]`, the cross-object measure refusal) are now unreachable for
them, because the door answers first. Neither refusal changes.
- No route-level pin was added. The dispatcher relays this door's
envelope generically, and
`packages/runtime/src/analytics-json-dimension-door.test.ts` already
pins that relay for this door. The route-level readings above were taken
through the real route.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 4b59a38 commit 4727fcb
5 files changed
Lines changed: 491 additions & 23 deletions
File tree
- .changeset
- packages/services/service-analytics/src
- __tests__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
Lines changed: 66 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
23 | 27 | | |
24 | 28 | | |
25 | 29 | | |
| |||
99 | 103 | | |
100 | 104 | | |
101 | 105 | | |
102 | | - | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
103 | 116 | | |
104 | 117 | | |
105 | 118 | | |
| |||
117 | 130 | | |
118 | 131 | | |
119 | 132 | | |
120 | | - | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
121 | 137 | | |
122 | 138 | | |
123 | 139 | | |
| |||
218 | 234 | | |
219 | 235 | | |
220 | 236 | | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
221 | 283 | | |
222 | 284 | | |
223 | 285 | | |
| |||
233 | 295 | | |
234 | 296 | | |
235 | 297 | | |
236 | | - | |
237 | | - | |
| 298 | + | |
| 299 | + | |
238 | 300 | | |
239 | 301 | | |
240 | 302 | | |
| |||
0 commit comments