Repository navigation
Commit 58a77db
Fixes #20987
Clause-②: yes (narrowing)
## What changed
The analytics read scope and the native analytics `where` now answer
`$contains` / `$notContains` on a field declared multi-valued or
JSON-stored by MEMBERSHIP, as the contract
(`FILTER_OPERATORS.$contains`) and `driver-sql` already do. On a scalar
text column both faces keep the substring test. There is one per-dialect
implementation, now in `@objectstack/core`.
- **Move (`@objectstack/core`, `@objectstack/driver-sql`).**
`jsonMembershipCandidates` and `jsonMembershipPredicate` leave
`sql-driver.ts`, where they were module-private, for
`packages/core/src/utils/json-membership-sql.ts`. They are exported from
the core root beside `compensatedSum`. The predicate is now
placeholder-agnostic: it emits the column and each value through caller
callbacks, left to right, and returns the SQL text.
`SqlDriver.applyJsonMembership` adapts it with knex's identifier
binding. No other region of `sql-driver.ts` moves.
- **Read scope (`read-scope-sql.ts`).** The `$contains` and
`$notContains` arms ask a new `membershipMatch` helper first. On a field
the caller declares multi-valued or JSON-stored
(`ReadScopeCompileOptions.declaredValueShape`, asked through the spec
predicates `driver-sql` asks: `isMultiValueField` and
`STRUCTURED_JSON_TYPES`), it emits the membership construct.
`$notContains` keeps its NULL rule (`col IS NULL OR NOT (...)`). On the
`'unknown'` dialect such a field is refused `READ_SCOPE_COMPILE_FAILED`
/ 500 before anything binds.
- **Native `where` (`native-sql-strategy.ts`, the `buildFilterClause`
text branch only).** The same fork, through a new adapter
`contains-membership-sql.ts`. On the `'unknown'` dialect such a field is
refused `INVALID_FILTER` / 400, which is the `$empty` where-leaf's
refusal.
- `filter-normalizer.ts`, `objectql-strategy.ts`, the `execute` region
and `text-match-sql.ts` are untouched.
## Per face, by class
| face | dialect | field class | before | after |
|:--|:--|:--|:--|:--|
| read scope | SQLite | multi-valued / JSON-stored | admits a row
outside the policy (the `u1` / `["u10"]` class) | membership: equals the
data door |
| read scope | PostgreSQL | multi-valued / JSON-stored | 500 for every
query under the policy | membership |
| read scope | MySQL | multi-valued / JSON-stored | substring construct
| membership construct (rendering only, NOT MEASURED) |
| read scope | unknown | multi-valued / JSON-stored | substring
construct | refused, `READ_SCOPE_COMPILE_FAILED` / 500 |
| native `where` | SQLite | multi-valued / JSON-stored | `$contains`
over-counts, `$notContains` under-counts | membership, exact complements
|
| native `where` | PostgreSQL | multi-valued / JSON-stored | 500 |
membership |
| native `where` | unknown | multi-valued / JSON-stored | substring
construct | refused, `INVALID_FILTER` / 400 |
| both | every dialect | scalar text | substring | substring (unchanged)
|
## Move proof
- **Extraction.** The moved region (`sql-driver.ts` `:3787-3971` at base
`d1f8ce8658`, 185 lines) was diffed against the moved region of
`json-membership-sql.ts` at `d898a2cc5b` (181 lines). 164 lines are
byte-identical. The 21 changed lines fall into two groups:
- The placeholder plumbing (16 lines): the two `export` keywords, the
signature (`dialect` / `emit` / `value`, returning the SQL text), the
dropped bindings array and its pushes, the five placeholder sites now
calling `emit.column()` / `emit.value(...)`, and the one docblock
sentence that described the knex plumbing.
- Five citation lines: two cards the docblocks cite answer 404 on the
board now, and `check-issue-citations` judges moved text as added. Each
was re-anchored to its landing commit, `e04a0aff2` or `82cb69fed`, in
their own commit.
The rest of both docblocks moved verbatim.
- **Pin.** `sql-driver-20987-json-membership-move.test.ts` compiles
`$contains` / `$notContains` on a JSON column through the driver's real
filter emitter, offline, for SQLite, PostgreSQL and MySQL clients and an
unmodelled client. It covers seven comparand shapes: one candidate, two
candidates, a canonicalised number, a JS number, a JSON-escaped string,
and a string `Number()` accepts but JSON refuses. Each statement and its
bindings are asserted against the ones captured from the pre-move
driver. It ran 51/51 green at base `d1f8ce8658`, before the move, and is
green after.
- **Dump.** A 60-cell dump of the same emitter (4 clients, 7 shapes,
both operators, and a scalar control) was taken at the base and again at
`d898a2cc5b`. The two are byte-identical.
## Pins (red first)
The first commit carries the pins alone; the fix follows.
- `service-analytics/src/__tests__/contains-membership.test.ts`: 24 red
/ 6 green at base. It runs the read scope on the native face, the
`/analytics/sql` echo and the ObjectQL face, and the `where` on both
strategies, all executed on SQLite. It also checks the construct per
dialect (PostgreSQL and MySQL rendering) and the `'unknown'` refusals,
with scalar-text controls.
- `qa/dogfood/test/analytics-contains-membership.dogfood.test.ts`: a
door pin through a real row policy written in the policy language, with
a scalar-text twin, on `sqlite-wasm` (native) and `memory` (ObjectQL).
Base: 2 red (SQLite), 8 green.
- `rest/src/analytics-contains-membership-door.test.ts`: the dataset
door over a real `SqlDriver`, read scope and `runtimeFilter`, with a
live-PostgreSQL cell behind `OS_TEST_POSTGRES_URL` (a named skip in CI).
Base: 3 red / 2 green on SQLite.
- `core/src/utils/json-membership-sql.test.ts`: the moved function's own
contract (11 tests).
The second commit narrows two ObjectQL-face assertions to the cells the
engine reaches. The engine's text-operator door refuses a
`STRUCTURED_JSON_TYPES` field (the #15661 ruling). The fourth commit
restores the strategy's `$notContains` assertion, because #20918 landed
in between.
## Ablation, one per face
Each ablation was run through `scripts/ablation-replace.mjs`, which
wraps the run, checks the anchor count, and restores by absolute path
with blob == HEAD. Each mutation was rebuilt into `dist/`, and
`ablation-dist-preflight.mjs` showed the marker in `dist/` before the
run. Predictions were written down before each run. Readings are at
`327d0bf142`:
| face removed | service-analytics pin | rest pin (SQLite) | rest pin
(live PostgreSQL) | dogfood pin |
|:--|:--|:--|:--|:--|
| read scope's membership arm | 14 red / 16 green, exactly the
read-scope cells | 2 red (both read-scope legs) | 2 red (500) | SQLite
read-scope leg red |
| `where`'s membership arm | 10 red / 20 green, exactly the
native-`where` cells | 1 red (`runtimeFilter`) | 1 red (500) | SQLite
`where` leg red |
**Restore leg.** Both sources were proven blob == HEAD. A rebuild
followed, both markers were shown absent from `dist/`, and `git status
--porcelain` was empty.
## Local PostgreSQL run
A private PostgreSQL 16.13 cluster was started for the run, used,
stopped, and its data directory removed. At the final head:
- the rest pin passed 10/10 (SQLite and PostgreSQL);
- `driver-sql`'s JSON-column membership suite, its multi-valued boolean
membership suite and this move pin passed 104, with 2 MySQL cells
skipped.
With either arm ablated, the PostgreSQL cell answers 500
`DATABASE_ERROR` (SQLSTATE 42883).
## Verification
**Gates.** Run at `d898a2cc5b` as one sequential script under the shared
lock:
- 75 commands, all exit 0. They are the 71 that `dispatch-gates.mjs
--commands` derives from this diff, plus the 4 roster families whose
roster sits under these paths (`check-changeset-fixed`,
`check:authz-resolver`, `check:error-code-casing`,
`check:filter-alias-parity`).
- `--ran` reconciliation: 71 derived, 71 run, 0 NOT MEASURED, each with
its exit code recorded.
- One correction came out of the first sweep: the citation re-anchor
commit above.
**Suites and typechecks.** Run at `c4c507d044`. Its one successor,
`d898a2cc5b`, changes comment lines only.
- `@objectstack/driver-sql`: 204 files, 3333 passed, 188 skipped (live
cells).
- `@objectstack/core`: 64 files, 1838 passed.
- `@objectstack/service-analytics`: 155 files, 3528 passed, 10 skipped.
- `typecheck` exits 0 for `core`, `driver-sql`, `service-analytics`,
`rest` and `dogfood`.
**Door pins at `c4c507d044`.**
- rest pin: 10/10 (SQLite and live PostgreSQL).
- dogfood pin: 8 passed. The 2 red were the `memory` legs; patch round 1
turns them green (10/10 at `d30ab8f65f`, see the section below).
**ESLint.** The narrowed run covers the 11 TypeScript files of this
diff, a count read from `--format json`: 0 errors, 0 warnings. The
config enables no type-aware rules (`eslint.config.mjs`: no
`parserOptions.project`), so this diff cannot move the verdict on any
untouched file. The full `pnpm lint` is CI's.
**NOT MEASURED:**
- MySQL: no server in this container.
- CI's path-scheduled jobs and the type-check lanes `dispatch-gates`
names outside its command list.
## Acceptance notes
- **Open question for the seat:** decided A (`5927023075`); see *Patch
round 1* below.
- **Q3 (ruled):** `driver-memory`'s copy of the SQLite construct
(`memory-analytics.ts`, its echo) is not converged here. It can now read
`@objectstack/core`'s function: `domain:engine`'s convergence item
5922891789.
- **Q4 (ruled):** the ObjectQL strategy's `/analytics/sql` echo of a
`where` still prints the substring construct while that strategy's
execution answers membership. The echo serves no rows. Its read-scope
echo now prints membership, because the read-scope compiler is shared,
and it refuses on `'unknown'`.
- **MySQL: NOT MEASURED.** No server in this container. The constructs
are asserted as text only. The read-scope compiler quotes identifiers
with `"` on every dialect, which is pre-existing and unchanged here.
- **Engine door vs. these faces on `STRUCTURED_JSON_TYPES`.** The engine
refuses every text operator over a `json`-class field (#15661). These
two faces now answer membership there, following the ruled population.
Before this change they answered substring. The faces disagreed before
and still disagree, so a reader should not take membership on that class
as engine-confirmed.
- **Residue, not visible to analytics:**
- a single-value `file` / `image` column whose JSON storage depends on
the deployment (`driver-sql`'s `mediaColumnIsJson`);
- the driver-internal `object` / `array` aliases;
- a host that wires no field metadata.
All three keep the text-match family, as `driver-sql` does for a table
it was never told about.
- **Stale gate tree:** the gate list was derived at the final head,
which was behind `origin/main` by commits that touch none of this diff's
files (`ci.yml` and SDUI manifest scripts among the gate inputs). CI
judges the merge ref.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H)_
## Patch round 1: the unknown-dialect route (the seat's decision
5927023075, option A)
_Appended by the `domain:services` seat (#6021) from the patch-round
report `5928649759`. The dev writes a PR body once._
**What changed.** `NativeSQLStrategy.canHandle` gained one decline and
the helper `jsonConstructOnUnknownDialectIn` beside it. Nothing else in
`native-sql-strategy.ts` moved: not `execute`, the shaping point,
`AGGREGATE_SQL` or `buildFieldMeta`. Two import lines were added.
The decline fires when a query would need a JSON function on an object
whose dialect is `'unknown'`. That means one of these, on a declared
field:
- `$contains` / `$notContains` on a multi-valued or JSON-stored field;
- `$empty` on a multi-valued field.
The judgement is made per object, from the declared value shape and the
dialect the emitters ask. It reads the filters the strategy would
compile:
- the `where`;
- the dataset's own `filter`;
- each requested measure's `filter`;
- the read scope of every object the statement scopes.
The query then routes to the ObjectQL strategy, whose engine answers.
With no ObjectQL bridge, nothing answers and the query is refused,
fail-closed. The compile-time refusals stay as the backstop. The
`$empty` latent defect on non-SQL drivers is closed by the same decline.
**Pins, red first** (commit `8f48d53bdf`, on the merge of `main` at
`6703bfdf05`):
- `contains-membership.test.ts` gained a describe for a host with no
dialect answer, whose raw-SQL bridge refuses as a non-SQL driver's does.
It runs 14 tests. Before the fix, 10 were red, each with the
compile-time refusal:
- the policy face: `$contains` / `$notContains` on the two multi-valued
classes, and `$empty`;
- the `where` face: the same five.
- The other 4 stay green before and after:
- the no-bridge refusal on each face, because the compile-time backstop
also refuses;
- a scalar-text control on each face, which is not declined: the native
strategy is tried once and falls back as before.
- The dogfood door pin's two `memory` legs were red before the fix.
**Fix** `d30ab8f65f`. Readings at `d30ab8f65f`:
- `contains-membership.test.ts`: 44/44.
- dogfood door pin: 10/10, the `memory` legs included.
**Ablation (the decline removed).** Done with `ablation-replace.mjs` in
WRAP mode, with the marker shown in `dist/` before the run. The
prediction was written first, and the run matched it exactly:
- the 10 route pins went red; 34 stayed green;
- the dogfood pin's two `memory` legs went red; 8 stayed green.
The restore was proven blob == HEAD (`d276c0303c`). After a rebuild, the
marker was absent from `dist/` and `git status --porcelain` was empty.
**Suites at `d30ab8f65f`** (after the merge of `main`):
- `service-analytics`: 156 files, 3570 passed, 10 skipped.
- `core`: 72 files, 2084 passed.
- `driver-sql`: 205 files, 3336 passed, 188 skipped.
**Door pins at `d30ab8f65f`.** Both door pins passed:
- rest: 10/10, SQLite and a private live PostgreSQL 16.13 cluster,
started, stopped and removed again;
- dogfood: 10/10.
**Typecheck** exits 0 for `core`, `driver-sql`, `service-analytics`,
`rest` and `dogfood`.
**Gates at `d30ab8f65f`.** The run was one locked sequential script of
75 commands: the 71 derived, plus the 4 roster families under these
paths. All 75 exit 0.
- `check:dual-build-cjs-loads` first answered PREREQUISITE NOT MET,
because 8 unrelated packages had no `dist/` in the fresh worktree. After
building them it exits 0, re-measured at the same head.
- `--ran` reconciliation: 71/71, every exit code recorded, 0 NOT
MEASURED.
- The narrowed ESLint run over the diff's 11 TypeScript files: 0 errors,
0 warnings.
**Acceptance notes, added:**
- On a host whose raw-SQL bridge runs SQL but which answers no dialect,
and which has no ObjectQL bridge, such a query used to get the declared
compile-time refusal (`READ_SCOPE_COMPILE_FAILED` / `INVALID_FILTER`).
It now gets the service's "no strategy" refusal. Both refuse, and
neither runs a statement.
- The plugin's default composition wires both bridges, so this reaches
only a hand-built host.
- The unit pins hold the fail-closed half on both faces.
- #21080 (queued) also edits `canHandle`. It goes after this PR, or
merges `main` after it.
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent 3572916 commit 58a77db
12 files changed
Lines changed: 1830 additions & 190 deletions
File tree
- .changeset
- packages
- core/src
- utils
- drivers/driver-sql/src
- qa/dogfood/test
- rest/src
- services/service-analytics/src
- __tests__
- strategies
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
56 | 56 | | |
57 | 57 | | |
58 | 58 | | |
| 59 | + | |
59 | 60 | | |
60 | 61 | | |
61 | 62 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
0 commit comments