Repository navigation
feat(spec,objectql,plugin-security): one shared filter lowering, run once at the engine and RLS seams (#5930 step 2) - #20794
Conversation
…ded) Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…y filter position Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…, typed by the declared datetime columns Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…d driver input Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…ng export Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 3 package(s): 19 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: ⛔ 6 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails. What this run could not see
Coarse fallback — 141 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 4f8720eea10306a776162b9fc1c68eaca689729c && git checkout 4f8720eea10306a776162b9fc1c68eaca689729c
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 8acdae9d8f0fc71cabe1671e7ec622213cc4f546 36e5ce8828a20a32ba415108d429c6f2141ca738 && git checkout -B drift-repro 8acdae9d8f0fc71cabe1671e7ec622213cc4f546 && git merge --no-ff 36e5ce8828a20a32ba415108d429c6f2141ca738
node scripts/docs-audit/affected-docs.mjs --json 8acdae9d8f0fc71cabe1671e7ec622213cc4f546
|
Contract reviewServed-tier: PR #20794 ( ① Derived judgmentsConformance to the amended D-D1, item by item (1–7, 9):
Published surfaces the diff implies:
Review faces, sentence by sentence: the changeset, the module TSDoc, the anchor and the PR body are true against the code as read — the rule list; the copy-on-write / idempotent / never-refuses / provenance / closed-vocabulary contract ( Check-runs on the head, as read (not waited for): success — Auto Label, filter, Check Changeset, Check PR Size, Governed Surface Queue Guard, the three card / branch / single-writer guards, Spec property liveness, Flag docs affected by code changes, Check Documentation Links, Type Check · source gates, Type Check · debt ledger, Dogfood Regression Gate (1/3), and the Vercel status; skipped (rostered) — Console Pin Gate, Build Docs, Packed-tarball smoke; in_progress — Build Core, Test Core (1/6 through 6/6), Temporal Conformance (live PG + MySQL), Dogfood Regression Gate (2/3 and 3/3), Dogfood Verify CLI, Lint & Repo Gates, Type Check · consumer gates, Type Check · workspace. None failing. Their conclusions are the gate verdicts; the seat owns convergence. ② Semver level
③ Boundary flags
Implemented-by: VERDICT: FAIL One item: ② Generated by Claude Code |
…Guard.datetime widens a published accept set Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
Contract reviewServed-tier: Delta review of PR #20794 ( ① Derived judgmentsThe FAIL item is fixed as prescribed — right. The delta is one commit, No sentence of the changeset needed to change with the level — right. Read in full at the head: no sentence names a level. The two sentences the widening touches already name it — " The addendum agrees with the diff. 5905899813 reports Check-runs on the head, as they stood when read (this record posted 2026-09-30T07:04Z; not waited for): success — Auto Label, filter, Check Changeset, Check PR Size, Check Documentation Links, Flag docs affected by code changes, Governed Surface Queue Guard, Spec property liveness, No other open PR may claim the same issue, No other open PR may claim the same single-writer path, Part-of PR must not also close its card, The card this PR closes must claim this branch (12); skipped, rostered — Build Docs, Console Pin Gate, Packed-tarball smoke (3); in_progress — Build Core, Test Core (1/6 through 6/6), Temporal Conformance (live PG + MySQL), Dogfood Regression Gate (1/3 through 3/3), Dogfood Verify CLI, Lint & Repo Gates, Type Check · source gates, Type Check · debt ledger, Type Check · consumer gates, Type Check · workspace (17); the Vercel status pending. 32 check-runs, none failing. Their conclusions are the gate verdicts; the seat owns convergence. Three that were ② Semver level
③ Boundary flags
Implemented-by: VERDICT: PASS The one FAIL item of 5905611995 is fixed as prescribed and nothing else moved; every other judgment of that record carries over to this head unchanged. Generated by Claude Code |
…sys_migration DATABASE_ERROR (objectstack-ai#20768) (objectstack-ai#20818) Fixes objectstack-ai#20768 Clause-②: no On the first boot of a new database, the SQL driver printed one `[sql-driver] DATABASE_ERROR ... no such table: sys_migration` line on its warn channel. The read behind it is the engine's migration-gate read. The driver's own pre-DDL question reaches that read before the schema pass has created any table. The driver now asks that question inside an async scope. Inside the scope, one class of refusal goes to `debug` instead of `warn`: a missing table, recognised by the shared `isMissingTableError` over the envelope's declared target. Every other refusal still warns, and so does a missing table read anywhere else. No gate's answer changes. ## Reproduction on the base `objectstack dev --database file:NEW.sqlite -p PORT` on `examples/app-crm`, base `96e724475c` (every package in the app's closure built), stdout and stderr captured separately: | run | `DATABASE_ERROR` lines | channel | |:--|:--|:--| | first boot of a new file | **1** | stderr (the driver's default `console.warn` sink) | | second boot of the same file | 0 | none | The line, verbatim: ```text [sql-driver] DATABASE_ERROR — the backend refused a read on 'sys_migration' (SQLITE_ERROR). The dialect message below is kept server-side: it carries the compiled statement, and on the dialects that inline them the bound literals too (objectstack-ai#7929, objectstack-ai#8931): select * from `sys_migration` where `id` = 'adr-0104-file-references' limit 1 - no such table: sys_migration ``` ## The mechanism: which dispatch hypotheses held **H1 held.** For one boot, I added a temporary stack-trace line to the built `packages/objectql/dist` (restored afterwards, `cmp` byte-identical on all four bundles, marker count 0). The first gate read at boot is `ObjectQL.readMigrationFlagVerified`, reached through `readFileReferencesFlagRow` and `haveFileColumnsMoved`. It is called by the closure that `registerDriver` hands the driver, which `SqlDriver.resolveFileColumnsMoved` asks at the start of `SqlDriver.initObjects`, from the schema pass's first `syncSchema`, before any DDL. The other gate readers run after the schema pass has created `sys_migration`, so on the first boot they read without error: `isValueShapesMigrationVerified` from write hooks, and `announceOpenMigrationGates` at `kernel:bootstrapped`. **H2 falsified.** The engine cannot tell whether the table exists without asking the backend: - The driver's registered-table set answers "registered in this process", not "exists". On a second boot the table exists but is not registered yet when the question is asked. Answering "not moved" there without a read would change the answer on a deployment whose columns have moved, which is the failure the resolver exists to prevent. - The schema pass has no plan at that moment: the question is asked before the first `hasTable`. - For an absent table, `conclusive` must be `false`. The same pass creates the table moments later, and a memoized "not verified" would freeze a whole boot's posture from a moment when nothing could answer. That is what the refused read already returns, so it is unchanged. - A catalog probe (`hasTable`) would give "not asked", but it needs a new public driver method or a new `IDataDriver` member. That enlarges a public surface, against this card's `Clause-②: no`, so this PR does not add one. **H3 held.** `SqlDriver.backendStatementFault` writes the line before the engine sees the error. The engine's `findOne` path does not log, and its catch in `readMigrationFlagVerified` answers `{ verified: false, conclusive: false, columnsMoved: false }` silently. So the demotion is in the driver, keyed on `isMissingTableError`, and limited to this read: the async chain of the driver's own question. **H4: measured at `c5ae3a6f8e`.** The second boot prints nothing, as on the base. `os migrate plan --database-url file:X` on the database the boots created prints no `DATABASE_ERROR`. On a path that does not exist yet, the plan printed 7 lines on the base, 2 of them on `sys_migration`, and prints 6 now. The one `sys_migration` line left is the `adr-0104-value-shapes` read from `announceOpenMigrationGates`, outside the driver's question. The other 5 are `sys_metadata` (4) and `sys_metadata_activation` (1). They are not this card. See the acceptance notes. ## Landing site: `packages/drivers/driver-sql`, not `packages/objectql/src/engine.ts` The dispatch expected the engine, with the driver only if a demotion was needed. As H2 and H3 show, the demotion is needed, and the line's producer is the driver. The premature read is the driver's own pre-DDL question. `engine.ts` is unchanged. ## What changed `packages/drivers/driver-sql/src/sql-driver.ts`: - A module-level `AsyncLocalStorage` (`PRE_DDL_QUESTION_SCOPE`). `resolveFileColumnsMoved` runs the resolver inside it, and nothing else does. It is module-level rather than per instance because the question's read goes to whichever driver serves the ledger, which on a multi-datasource composition can be another instance in the same async chain. - `backendStatementFault` composes the envelope first. If the scope is active and `isMissingTableError(envelope, object)` holds, it writes the same dialect text to `this.logger.debug?.(...)` and returns the envelope. Otherwise it warns as before. The throw, the envelope, and what the resolver hears are unchanged. - The logger shape gains an optional `debug` channel. The default sink has none. - `isMissingTableError` is imported from its home, `@objectstack/types`. `@objectstack/metadata/errors` re-exports the same symbol, and driver-sql cannot depend on `@objectstack/metadata`. No second message regex. `.changeset/20768-first-boot-migration-gate-read.md`: `@objectstack/driver-sql` patch. `@objectstack/runtime` gains only a test file, which its `files[]` (`dist`, `README.md`, `CHANGELOG.md`) does not ship, so it gets no changeset entry. ## After: this branch at `c5ae3a6f8e`, built | run | `DATABASE_ERROR` lines | `sys_migration` mentions | server ready | |:--|:--|:--|:--| | `objectstack dev --database file:NEW.sqlite` (`examples/app-crm`), first boot | 0 | 0 | yes | | second boot of that file | 0 | 0 | yes | | `os migrate plan` on that file | 0 | 0 | exit 0 | Normalised for timestamps, port and file path, the first-boot stderr differs from the base run in exactly one line: the removed one. Both ledgers end in the same state after the first boot: both flags verified by the fresh-datastore attestation, and `columns_moved_at` null. ## Tests - **`packages/drivers/driver-sql/src/sql-driver-20768-pre-ddl-question-missing-table.test.ts`** (new, SQLite on a new temp file). The resolver is a stand-in that reads the ledger the way the engine does. - ① First boot: no warn line, one `debug` line (`no such table`). The resolver gets the envelope (`code: 'DATABASE_ERROR'`, `status: 500`), and the arm stays "not moved". - ② Same result when the ledger is served by a second driver instance. - ③ Second boot: a `columns_moved_at` row written between the boots makes the arm "moved", so the read reached the row. Nothing is logged. - Controls, each still `warn`: ④ a malformed read on an existing table inside the question (40,000 bound variables, SQLite refuses the statement); ⑤ a view over a dropped table inside the question (a missing table named by another relation); ⑥ a missing table read outside the question. - **`packages/runtime/src/first-boot-migration-gate-read.integration.test.ts`** (new). A real `ObjectQL` engine over a real `SqlDriver` on a new file, `SysMigration` registered, and `engine.syncSchemas()`, which is the same `syncSchema`, `initObjects`, resolver, gate-read chain. - First and second boot: no `sys_migration` `DATABASE_ERROR` on warn, and one demoted line on the first boot only. - Gate answers asserted on both boots: not moved and not verified on the first; verified after a row is written between boots. - Control: a malformed read on an existing table still warns, and so does a missing table read after the boot. It counts only its own reads' lines. - Suites at `c5ae3a6f8e`: - driver-sql: 201 files passed, 11 skipped (the live PG and MySQL cells, not provisioned here); 3254 tests passed, 188 skipped. - runtime `--project local`: 293 files, 4207 tests passed, 1 skipped. - Typecheck green for driver-sql and runtime. Runtime's `check:test-typecheck` debt is unchanged, and `tsc --listFiles` shows both new tests inside their packages' programs. ## Reverse verification (from the committed fix) `packages/drivers/driver-sql/src/sql-driver.ts` was restored to its base blob with `git restore --source=96e724475c`: on-disk blob `2462eccd` = base, marker count 0. driver-sql was rebuilt, and `ablation-dist-preflight --absent PRE_DDL_QUESTION_SCOPE` found the marker absent from all 6 built files. - Driver pin: **red** on ① and ② (`expected [ Array(1) ] to deeply equal []`). ③ to ⑥ stayed green. - Runtime pin: the boot test **red** (same message). The control stayed green. Restore: `git checkout HEAD -- packages/drivers/driver-sql/src/sql-driver.ts`. Blob `a3647fd1` = HEAD, and `git diff HEAD` was empty. After a rebuild, the preflight found the marker in 2 built files and a clean tree, and both pins were green again. The first run of this verification also turned the runtime control red: it had counted the boot's own `sys_migration` line together with its own. It now counts only its own reads, and the second run above is from that commit. ## Gates `node scripts/pm/dispatch-gates.mjs --commands` at `c5ae3a6f8e` derives 63 commands, and all 63 ran on that head with exit 0. `--ran` reconciliation: "63 derived famil(ies) accounted for — 63 run, 0 NOT-MEASURED (a DERIVED zero — all 63 recorded an exit code and none of them is 3)". On the first pass, `check:dual-build-cjs-loads` answered PREREQUISITE NOT MET (9 packages had no `dist/`). I built those 9 and re-ran it, and it exited 0. Lint, narrowed: `eslint --no-inline-config --format json` over the 3 touched `.ts` files counted 3 files, 0 errors, 0 warnings. The population is the config's `packages/**/*.{ts,tsx,mts,cts}` and `**/*.{ts,...}` objects. The config never enables type-aware linting (no `parserOptions.project`), so this diff cannot change the verdict on an untouched file. The branch merged `main` twice. The second merge brought in objectstack-ai#20794, the serial constraint. Neither merge touched this diff's files, apart from the tracker-number wording in the same `DATABASE_ERROR` line, which this branch now carries as `main` spells it. ## Acceptance notes - `os migrate plan` against a database that does not exist yet still prints 6 `DATABASE_ERROR` lines at `c5ae3a6f8e`: `sys_metadata` 4, `sys_metadata_activation` 1, and `sys_migration` 1. The last is the `adr-0104-value-shapes` read by `announceOpenMigrationGates` at `kernel:bootstrapped`, which on a deferred-DDL plan runs over tables the plan never creates. It is the same false-alarm family, on a different door, and is reported to the seat. It is not fixed here. - "Demoted, not deleted" holds only where a host injects a logger that has `debug`. No production composition hands `SqlDriver` a logger today, so with the default sink the line is dropped. The engine's catch records nothing either. - `PRE_DDL_QUESTION_SCOPE` covers any read in the resolver's async chain, including one a hook starts inside it. That is intended: every such read is issued for the question, before the schema pass has run. --- _Generated by [Claude Code](https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…the analytics seams and the cube face's new door (objectstack-ai#5930 step 3) (objectstack-ai#20857) Fixes objectstack-ai#20810 Clause-②: yes (narrowing) objectstack-ai#5930 step 3, under ADR-0053 D-D1 as amended by objectstack-ai#20754: the one shared `FilterCondition → FilterCondition` lowering (`lowerFilterCondition`, `@objectstack/spec/data`, landed in step 2 as objectstack-ai#20794) now runs at the three analytics seams, after the comparand doors and after filter-token resolution, and the two faces that could not compile its output now can. `packages/spec/src/**`, the interim window arms and objectstack-ai#20807's position are untouched. ## Named gap: three pins outside the claim's file surface `packages/drivers/driver-memory/src/memory-driver-filter-logic-conformance.test.ts` holds three pins of the cube face's PRE-change vocabulary and shape, and it is outside the claim's `memory-analytics*.test.ts` surface, so this branch does not edit it. They are red here, and the `driver-memory` Test Core shard is red until they move: - "every combinator case is refused rather than dropped" asserts every `$or` case refuses. The face compiles `$or` now; the same file's case-by-case invariant (agree with `find()` or refuse) stays green over every `$or` case. - "every operator this face DECLARES compiles to a predicate that agrees with find()" needs a probe for `$null`, which joined the face's table. - the `$notContains` pipeline-dump row pins the un-lowered `$match`; the lowering's NULL escape now wraps it. The three-row patch (15 changed lines) is prepared and was verified on a copy of the file: 125 of 125 tests pass. It lands once the file is added to the surface. ## Per seam | Seam | Position at `bbe03f406` | Pin | Ablation (lowering removed at that seam) | |---|---|---|---| | analytics `where` door, F10 (`where` → tree, both strategies) | `normalizeWhereComparands` `filter-normalizer.ts:2218`, reached through `lowerAnalyticsWhere` `:2244`; the lowering sits in `normalizeAnalyticsFilterTree` `:2356`, the one compile entry both spellings reach | `where-door-shared-lowering-seam.test.ts`, 18 rows | 6 of 18 red: the tree rows, the ObjectQL hand-off and the echo | | draft-preview door, F11 | `preview-evaluator.ts:675`, right after its `normalizeWhereComparands` call | same file (preview block) | 3 of 18 red: the three NULL-escape rows | | read scope | entry of `compileScopedFilterToSql` `read-scope-sql.ts:747`, after placeholder resolution `:764`, before `compileNode` `:769` | `read-scope-shared-lowering-seam.test.ts`, 12 rows | 5 of 12 red: the caller-filter rows, the RLS bound as written, the SQL row and the date-macro row | | cube face F5, the new door | `normalizeFilters` `memory-analytics.ts:1585`, ahead of its gate `:1592` | `memory-analytics-shared-lowering-door.test.ts`, 13 rows | doors removed: 4 of 13 red (objectstack-ai#20734's table); lowering removed: 2 of 13 red | | the where door's nested-relation spelling (below) | `normalizeAnalyticsFilterTree` | the nested-relation rows in two files | 2 red | Every ablation mutated the committed file through `scripts/ablation-replace.mjs` (anchor hit once, blob moved), ran the pins, and restored: blob equal to HEAD and `git diff HEAD` empty, each time. The pins import their subjects by relative path, so vitest reads `src/` and no build sits between the mutation and the reading. ## What each seam does - **Read scope.** `compileScopedFilterToSql` lowers the scope right after its placeholders resolve. Column-type scope (item 7): the `declaredValueShape` option both consumers already pass, so a declared `datetime` column is rewritten and a `date`, `time` or text column compiles byte-identical; no declarations handed in reads no column as `datetime`. The shared comparand doors still judge the scope as written after compilation (objectstack-ai#20018's order); the lowering never refuses, so no verdict moves. - **Analytics `where` door (F10).** `normalizeAnalyticsFilterTree` lowers what the door admitted before `buildNode` reads it. Its column-type reader is now a REQUIRED argument: both strategies pass `declaredDatetimeLowering` (a member is `datetime` when its column is declared so, through the `declaredFieldType` hook, asked of the same target each strategy compiles against); a context without the hook, and the member-only readers (`assertWhereFields`, the cross-object view), pass `NO_DATETIME_COLUMNS`. `lowerAnalyticsWhere` stays un-lowered: its other readers (ad-hoc cube dimension minting, the routing detectors) read the authored condition. - **Nested relations at the `where` door.** The shared lowering has no reading of the nested-relation spelling (`{ account: { region: 'NA' } }`: accepted by the schema, refused by the engine, flattened only by this door). Inside a `$not` it read `account` as a column and guarded it. The door now spells nested relations as the dotted members `fieldLeaves` has always compiled them to, before the lowering reads the condition, so the guard lands on `account.region`. - **Draft preview (F11).** Lowered after its `normalizeWhereComparands` with `NO_DATETIME_COLUMNS`: drafted rows carry no schema, and a type-blind rewrite would move one cell away from the typed drivers (`$lte` on the last supported day over a non-temporal value that sorts above it); its own `lteBound` copy keeps the whole-day rule until its deletion card. It now evaluates `$null`. - **Cube face (F5).** `normalizeFilters` runs `assertListComparandShapes`, then `normalizeFilterComparandTypes` (its return is the condition read from there on), then the lowering (type-blind: the face reaches declared types only as a storage-form conversion, and its own copy and `find()` are type-blind already), then its own gate on the lowered condition. It compiles `$or` (a disjunction entry both exits render, with the objectstack-ai#5322 identities) and `$null`; `$not`, `$startsWith`, `$endsWith` and `$empty` stay refused. ## Every corrected answer Read scope (`compileScopedFilterToSql`: the NativeSQL statement's scope and the `/analytics/sql` echo's), each now `SqlDriver.find`'s rows on the same filter: - `{ signed_at: { $lte: '2026-07-28' } }` on a declared `datetime` over rows at 10:00Z on 07-27, 07-28, 07-29 and a row with no value: was 07-27, now 07-27 and 07-28 (`< '2026-07-29'`). objectstack-ai#20733's caller-filter half. - `{ signed_at: { $between: ['2026-07-28', '2026-07-28'] } }`: was no row, now 07-28. - a `{today}` upper bound is widened as the day it resolves to. Analytics `where`, ObjectQL path: a bare-day `$lte` on a `datetime` member reaches the engine as `$lt` the next day, and the `/analytics/sql` echo prints that half-open bound where it printed `<=` the named day. Rows unchanged. Draft preview: `$null` answered (was refused 400). A row with no value now satisfies `$ne`, `$nin` and the negation of an equality when the comparand is the text `"null"` or `"undefined"`; the face compared those as text against the missing value. Cube face, measured on the published `MemoryAnalyticsService.query` at the base blob and at this head: | `where` (fixture: `d` is `v1`, `v2`, null, absent) | base | this head | `find()` | |---|---|---|---| | `{d: undefined}` / `{d: {$eq: undefined}}` | 3, 4 | refused 400 | 3, 4 | | `{d: {$ne: undefined}}` | 1, 2 | refused 400 | 1, 2 | | `{d: {$in: ['v1', undefined]}}` | 1 | refused 400 | 1 | | `{d: {$in: ['v1', null]}}` | 1, 3, 4 | refused 400 | 1, 3, 4 | | `{d: {$nin: ['v1', null]}}` | 2 | refused 400 | 2 | | `{d: {$gt: null}}` | none | refused 400 | none | | `{d: {$in: 'v1'}}` | 1 | refused 400 | uncoded throw | | `{d: {$eq: {a: 1}}}` | none | refused 400 | none | | `{d: {$ne: {a: 1}}}` | all four | refused 400 | all four | | `{d: new Map()}` | all four | refused 400 | none | | `{n: {$gt: 2n ** 60n}}` | uncoded throw | refused 400 | uncoded throw | | `{n: {$gt: 2n}}` | uncoded throw | 3, 4 | uncoded throw | | `{d: {$between: ['v1', 'v2']}}` | refused 400 | 1, 2 | 1, 2 | | `{d: {$null: true}}` | refused 400 | 3, 4 | 3, 4 | | `{$or: [{d: 'v1'}, {n: 4}]}` | refused 400 | 1, 4 | 1, 4 | | `{d: {$ne: 'v1'}}`, `{$not: {d: 'v1'}}` | unchanged | unchanged | — | (`find()` here is the driver called directly, without the engine seam whose doors it relies on.) Every "refused 400" is the ADR-0112 `INVALID_FILTER` envelope every other analytics face already answers. ## Clause-②, measured - **Widening:** the cube face accepts `$or`, `$null` and `$between` (each refused at the base, each now `find()`'s rows), and the draft preview accepts `$null`. `ANALYTICS_FILTER_CAPABILITIES` names `$null` and `$or`. - **Narrowing:** the cube face refuses the comparand shapes in the table above that it used to answer. Hence `yes (narrowing)`. `@objectstack/driver-memory` ships `minor` with the BREAKING banner, the migration and an ADR-0087 `not-required (no-migration-prescription)` disposition; `@objectstack/service-analytics` ships `minor` (`Clause-②: yes`, widening only: the read scope and the `where` door refuse nothing new). ## Mechanism assumptions, measured - **A1 held**, with one refinement: the F10 call sits in `normalizeAnalyticsFilterTree` rather than inside `normalizeWhereComparands`, because the array spelling reaches F10 through `parseFilterAST` and never through `normalizeWhereComparands`, and because `lowerAnalyticsWhere`'s other readers want the authored condition. F11 lowers right after its own `normalizeWhereComparands` call. - **A2 measured.** `where` door: tokens resolve upstream on every path that reaches it (`AnalyticsService.resolveQueryTokens` from `query()` and `generateSql()`, the per-request dataset-scope getter, `DatasetExecutor.resolveSelectionTokens` ahead of the preview). Read scope: placeholder resolution is the compiler's first act, and the doors run after `compileNode` on the scope as written (objectstack-ai#20018), so the lowering sits between resolution and compilation. F5: no token resolution exists on that face. - **A3 proved, not assumed.** With the real `SecurityPlugin` and the real read-scope compiler on SQLite (a one-off run, not committed): `getReadFilter` returned `{"$and":[{"signed_at":{"$lt":"2026-07-29"}},{"due_on":{"$lte":"2026-07-28"}}]}`, the read scope compiled `("t"."signed_at" < ? AND "t"."due_on" <= ?)` and answered c27 and c28, equal to `SqlDriver.find`. The committed pin holds the RLS half in both spellings the read scope can be handed (lowered by the RLS seam, and as written), so its answer no longer depends on which arrives. - **A4 measured.** F5 compiled 12 operators with `$and` alone; F11 10 operators with `$and`/`$or`/`$not`. The lowering emits `$and`, `$or`, `$lt`, `$gte`, `$lte`, `$null`. Widened: F5 `$or` + `$null`; F11 `$null`. Nothing wider. - **A5 held.** No window arm is touched (NativeSQL, the preview's window loop, F5's `timeDimensions`). - **A6** as the table above: objectstack-ai#20733's rows (both halves, the last day, the declared `date` control) and objectstack-ai#20734's table, plus a pin and an ablation per seam. - **A7** above. - **A8, coupling, re-measured:** objectstack-ai#20280 landed on `main` as `05a7547c9` while this branch was open. Its datetime year floor lives in the engine's temporal comparand door (`objectql` `temporal-comparand-door.ts`) and in `@objectstack/core`'s temporal storage form; in `packages/spec/src/data` it changed one comment. The two spec doors the cube face now runs are not changed by it. What the two share is the storage-form conversion both of the face's exits apply to a comparand (the driver's `filterComparandStorageForm`, which reads `@objectstack/core`): objectstack-ai#20280 changed that conversion, and this branch neither touches nor pre-empts it. Nothing in this PR's file surface changed on `main`, so the branch is not re-merged; this PR's CI runs on the merge ref with objectstack-ai#20280 in it, and the merge queue adjudicates the rest. ## Evidence (head `92a449c2d`, `main` merged at `c90f9fb6e`) - `@objectstack/service-analytics`: 143 files, 3297 tests passed; `typecheck` green (baseline at `bbe03f406`: 141 files, 3267). - `@objectstack/driver-memory`: 66 files, 1482 passed, 3 failed (the named gap above); `typecheck` green (baseline 65 files, 1470). - Existing shape pins moved to the lowered structure; no asserted row count moved. The row-level conformance suites (native-SQL filter-logic and temporal, read-scope conformance, preview temporal) are unchanged and green. - Gates: `node scripts/pm/dispatch-gates.mjs --commands` derived 63 families at this head; all 63 were run with their exit codes recorded, and `--ran` reports 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN (a derived zero). `check:dual-build-cjs-loads` and `check:type-check-debt` first answered `PREREQUISITE NOT MET` and were re-run green after `turbo run build --filter='./packages/*' --filter='./packages/*/*'`. The six roster families under this card's directories (`check-changeset-fixed`, `check:authz-resolver`, `check:error-code-casing`, `check:filter-alias-parity`, `check:object-def-param-keys`, `check:tenant-chokepoint`) are green too. - Downstream analytics suites on the rebuilt dists: `@objectstack/rest` `analytics-*` (10 files, 150 tests), `@objectstack/runtime` `analytics-*` (3 files, 23), `@objectstack/dogfood` `analytics-adhoc-query-isolation` (24): green. - Lint, a narrowed run: the population is the `eslint.config.mjs` block `files: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']`; `eslint --no-inline-config --format json` over the 38 changed source files reports 38 files, 0 errors, 0 warnings at `92a449c2d`; the config enables no type-aware linting (every `parserOptions` is `ecmaVersion` / `sourceType`), so this diff cannot move a verdict on an untouched file. - NOT MEASURED: live PostgreSQL / MySQL (CI's temporal job), a real mongod. ## Acceptance notes - **Double guards.** Each face's own interim NULL-polarity copy still guards what the lowering already guarded, so the SQL, the trees and the ObjectQL hand-off for `$ne`, `$nin`, `$notContains` and a `$not` operand carry the guard twice. The same rows; the deletion cards remove the inner copy and update these pins. - **For the step-4 deletion cards.** The draft preview reads no member as `datetime`; the `where` door with no `declaredFieldType` hook, and the read scope with no `declaredValueShape`, read none. Once a face's own bound copy is deleted, that path gets no whole-day bound unless its card gives it a typed reader. - **The read scope's temporal coercion (ADR-0053 D-A1, objectstack-ai#20733's other half).** The read scope binds the lowered calendar string as written, as it bound the authored one. Measured correct on SQLite ISO text; PostgreSQL / MySQL NOT MEASURED. Carrier: none. - **The shared lowering and the nested-relation spelling.** The spec module reads a nested-relation spec under a `$not` as a column constraint. No face meets that after this change (the analytics door spells it dotted; the engine, the read scope and the preview refuse the spelling; the cube face refuses `$not`). Noted for the module's owner; carrier: none. ## Patch round 1 (appended by the `domain:services` seat) **The named gap is closed.** The claim's surface gained `packages/drivers/driver-memory/src/memory-driver-filter-logic-conformance.test.ts` (amendment `5911719808`), and `edb3e2de4` applies the prepared 15-line patch to it and to nothing else: - every `$not` case must still refuse; the `$or` cases moved to the answered column, which the file's case-by-case invariant holds to `find()`'s rows; - `$null` joins the declared-operator probe roster; - the `$notContains` pipeline dump shows the lowering's NULL escape. Evidence on head `edb3e2de4`: - `@objectstack/driver-memory`: 66 files, 1485 passed (the three pins green, nothing else moved); `typecheck` green. - `@objectstack/service-analytics`: 143 files, 3297 passed. - Gates: the same 63 derived families, all exit 0; `--ran` reports 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN; the six roster families green. - `main` was not re-merged: none of the five commits after `c90f9fb6e` touches this PR's files. --- _Generated by [Claude Code](https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
Part of #5930 — step 2 of ruling 5902355785 (the seam lowering in the engine / RLS seams). Steps 3 (the analytics-face seams and the F5 / F11 vocabulary) and 4 (the per-face deletions) remain, so the card stays open.
Clause-②: yes
What this does
This implements ADR-0053 D-D1 as amended on 2026-09-30, items 1–7 and 9. The bare-day upper bound, the
$betweensplit and the NULL-polarity guards are applied once, by one sharedFilterCondition → FilterConditionlowering. It runs at the engine and RLS seams, after the comparand doors and after filter-token resolution. Drivers receive the lowered filter.@objectstack/spec/data: new pure modulefilter-lowering.ts, exportinglowerFilterCondition(filter, options?)andFilterLoweringOptions. It is exported from the./datasubpath only, never the root entry (the ruling's D3). The rules:$betweenbecomes$gtemin and$ltemax. A range holding a{ $field }or a non-pair is left whole, for the face that refuses it.$lteon a bareYYYY-MM-DDbecomes$ltnextUtcCalendarDay(day), in the calendar-string domain, never a storage form (D-A1).UNBOUNDED_ABOVEturns a lone$lteinto{ $null: false }, and a$betweenkeeps its minimum. Instants andDates are never widened.$neof a value,$ninand$notContainsget{ $or: [{ f: { $null: true } }, { f: op }] }(非否定路径上的$ne/$nin/$notContains:driver-sql 排除 NULL 行,driver-memory / formula 返回它们(#5146 只裁定了$not) #5298). Every leaf of a$notoperand is made total ($not的语义在 driver-sql 与 driver-memory / formula 之间分叉:NULL 行的去留相反,$not: {}一个是 TRUE 一个是 FALSE #5146).options.isDatetimeColumnscopes rules 1–2 on a typed seam (item 7).@objectstack/objectqlengine.ts: one stage function,resolveThenLowerWhere(resolve, then lower), is the only way any filter position resolves. That coversfind,findOneandcount(resolveWhereTokens);updateanddelete(withResolvedWhere); andaggregate'swhere, eachaggregations[i].filterandhaving.resolveWhereTokensandwithResolvedWherenow require the lowering options, so no verb can resolve without lowering. The judge (judgeWhereAdmission) runs the same stage. The type reader forwhereandaggregations[i].filteris the object's declaredtype === 'datetime', the same testSqlDriverindexesdatetimeFieldsby. Forhavingit is the aggregated row's column types (aggregatedRowColumnTypes, wheremax(datetime)isdatetime).@objectstack/plugin-security:judgeCompiledComparands(the RLS compile seam, servingusingandcheck) lowers every compiled policy filter right after the two faces.RlsFieldGuardgains an optionaldatetimeset.SecurityPluginfills it from the same declaration pass as the field-name set (loadObjectFieldNames) and hands it in at both compile sites. A guard without types reads no column asdatetime.scripts/adr-anchors/packages__spec__src__data__filter-lowering.ts.json: pins ADR-0053 to the module (Prime Directive [WIP] Add Chinese version of the documentation #13)..changeset/5930-shared-filter-lowering.md:@objectstack/specminor,@objectstack/objectqland@objectstack/plugin-securitypatch. It cites ADR-0053 D-D1 (amended).No face copy is deleted, no driver file is touched, and the analytics
where/ preview door, the read scope and the memory cube face are untouched (step 3).The stop line was reached: one evaluator's answers move, on rows with no value
The acceptance is answer invariance. Measured, the answers of every driver face stay the same. The engine's own in-process evaluator for
aggregations[i].filterandhaving(F8,having-filter.ts) moves, and only on rows or groups with no value. Before this change F8 was the only face that disagreed with the others on those rows. After it, all faces agree.A/B probe, not committed. Each face answers the filter as written and the lowered filter, on sqlite
SqlDriver(F1),InMemoryDriver(F3),matchesFilterCondition(F7) andmatchesAggregationFilter(F8):FILTER_LOGIC_CASESoverFILTER_LOGIC_ROWSTEMPORAL_CASES(plus the resolvedtokenFilters) overTEMPORAL_ROWS$not, plus$between/$ne/$nin/$notContainsprobesAll 14 moved cells are a
$betweenon adatetimecolumn with a row whose value is null. F8 kept that row in the range (7 cells) and dropped it under$not(7 cells). F1, F3 and F7 exclude it from the range and keep it under$not, which is the #5146 / #5298 reading. A second probe found the same class on a number column:{ $not: { amount: { $lt: 5 } } }dropped a nullamountin F8, because JS comparesnull < 5as true. Everywhere else the null row is kept. That probe covered the per-aggregation filter andhaving, one cell each.The decision on whether to keep the two aggregate seams wired is in the report on #5930, with the four-axis frame. This PR carries option A (keep them). Dropping them (option B) removes the two
aggregatehunks and their two pin rows.Mechanism hypotheses — which held
lowerWhereFilterArrayand tokens resolve after it on every verb. One helper (resolveThenLowerWhere) holds "resolve, then lower", and every verb has its own pin.record.signed_on <= '{today}'compiles to{ signed_on: { $lte: '{today}' } }, passes both faces, and reachesusing's drivers andcheck'smatchesFilterConditionverbatim. Nothing resolves a placeholder on either RLS clause. The RLS lowering therefore runs after the faces (item 3) and reads'{today}'as a non-day string it leaves as written, the same as every face does today. This is pinned.RlsFieldGuardcarried names only, but the types are in the same declarationloadObjectFieldNamesreads. (b) No in-repo or example policy compares any column against a bare day or a date token: 72 non-testusing/checkpredicate lines, all==,in,== null,!= nullor1 == 1. So no real policy's rows or admitted writes change under either reading. The seam takes the typed reading, because the type-blind one would moveusinganswers on SQL for a non-datetimecolumn (a text column holding day-prefixed strings, and$lte '9999-12-31'on text) in constructible policies.mainat085ca6bc1chasTest Core(6/6),Temporal Conformance (live PG + MySQL)andDogfood Regression Gategreen.$and,$or,$lt,$gte,$lteand$null. The unit table pins that closure overFILTER_LOGIC_CASES,TEMPORAL_CASESand every row. F1, F2, F3, F6, F7 and F8 already compile those.$nottotaliser are the SQL copies' tables cell for cell. The copies stay.Evidence (all on head
9ca3698b67)packages/spec/src/data/filter-lowering.test.ts: 46 tests. The rule table, the item-7 scope, idempotence over the table and both case sets, vocabulary closure, copy-on-write, provenance, and pass-through.packages/objectql/src/engine-shared-filter-lowering-seam.test.ts: 11 tests, one per verb and position, plus{today}resolved-then-widened, the typed scope, the last supported day, copy-on-write and the judge.packages/plugins/plugin-security/src/rls-shared-lowering-seam.test.ts: 14 tests. Both clauses throughRLSCompiler, plusSecurityPlugin.getReadFilterandcomputeWriteCheckFilterfed the declareddatetimeset.rls-compiled-comparand-facesgets the same treatment.rls-empty-membership-polarity'snot inshape is updated, and its admitted-row count stays 3.typecheckis green for spec, objectql, plugin-security, the five drivers and formula.check:driver-conformanceis OK (50 cells).scripts/ablation-replace.mjsand restored (blob equals HEAD,git diff HEADempty):resolveThenLowerWherewithout the lowering: 8 of 11 seam pins red, across every verb and all threeaggregatepositions.judgeCompiledComparandswithout the lowering: 8 of 14 red.SecurityPluginwithout thedatetimehand-off: the 2 plugin-level pins red.node scripts/pm/dispatch-gates.mjs --commandsderived 103 families; all 103 were run with exit 0, and--ranreports 0 NOT-MEASURED and 0 UNRUN. Three gates (check:dual-build-cjs-loads,check:i18n,check:type-check-debt) first refused withPREREQUISITE NOT MET. They were re-run green afterturbo run build --filter='./packages/*' --filter='./packages/*/*'.check:generatedshows 15 of 15 up to date after regeneratingapi-surface/andexport-origins/.eslint.config.mjsblockfiles: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}'].eslint --no-inline-config --format jsonover the 13 changed source files reports 13 files, 0 errors and 0 warnings.parserOptionsisecmaVersion/sourceTypeonly), so this diff cannot move a verdict on an untouched file.Acceptance notes
datetimewhen the object's declaration is missing (a registry-less host, or a guard without types). They do not apply the rule type-blind. This keeps every driver's answer where it was (SqlDriveralso widens nothing without a declaration). The step-4 deletion cards should re-read this: once a face's copy is gone, a declaration-less path gets no whole-day bound.security-plugin.tschanges in two places: thedatetimeset is read in the existing declaration pass, and it is handed in at the two compile sites. The ADR anchor is one new JSON file. Neither adds a seam.checkwith a date token.os validaterefuses a{placeholder}in a read-scopeusingclause (validate-rls-predicate-enforceability.tsjudges it through the engine). Acheckclause is not judged there, and nothing resolves the token at run time. Acheckofrecord.signed_on <= '{today}'therefore compares against the literal text, and every ISO value sorts below{. Public-door reach is not measured. Carrier: none.usingthroughgetReadFilter, so from this PR on it receives the RLS seam's lowered policy. For a policy with a bare-day$lteon adatetimecolumn (none in-repo), it now keeps the whole day, which is [finding] service-analytics read scope: compileScopedFilterToSql applies no whole-day upper bound and binds a temporal comparand as written, so an RLS $lte on a bare day drops the rest of that day in NativeSQL analytics #20733's direction. [finding] service-analytics read scope: compileScopedFilterToSql applies no whole-day upper bound and binds a temporal comparand as written, so an RLS $lte on a bare day drops the rest of that day in NativeSQL analytics #20733 itself (the scope's own bound on a caller's filter) is step 3 and is not addressed here.engine.tsandfilter.zod.tsprose, and whichever lands second mergesmain.Generated by Claude Code