Skip to content

Commit cfa9315

Browse files
feat(spec,objectql,plugin-security): one shared filter lowering, run once at the engine and RLS seams (#5930 step 2) (#20794)
Part of #5930 — step 2 of ruling 5902355785 (the seam lowering in the engine / RLS seams). Steps 3 (the analytics-face seams and the F5 / F11 vocabulary) and 4 (the per-face deletions) remain, so the card stays open. Clause-②: yes ## What this does This implements ADR-0053 D-D1 as amended on 2026-09-30, items 1–7 and 9. The bare-day upper bound, the `$between` split and the NULL-polarity guards are applied once, by one shared `FilterCondition → FilterCondition` lowering. It runs at the engine and RLS seams, after the comparand doors and after filter-token resolution. Drivers receive the lowered filter. - **`@objectstack/spec/data`**: new pure module `filter-lowering.ts`, exporting `lowerFilterCondition(filter, options?)` and `FilterLoweringOptions`. It is exported from the `./data` subpath only, never the root entry (the ruling's D3). The rules: 1. `$between` becomes `$gte` min and `$lte` max. A range holding a `{ $field }` or a non-pair is left whole, for the face that refuses it. 2. A `$lte` on a bare `YYYY-MM-DD` becomes `$lt` `nextUtcCalendarDay(day)`, in the calendar-string domain, never a storage form (D-A1). `UNBOUNDED_ABOVE` turns a lone `$lte` into `{ $null: false }`, and a `$between` keeps its minimum. Instants and `Date`s are never widened. 3. NULL polarity, cell for cell from the four hand copies' tables. `$ne` of a value, `$nin` and `$notContains` get `{ $or: [{ f: { $null: true } }, { f: op }] }` (#5298). Every leaf of a `$not` operand is made total (#5146). - The module is copy-on-write, so a filter it does not change comes back as the same reference. It is idempotent, and it recognises its own guards and the same shapes written by an author. It never refuses. It carries the #8220 provenance mark onto every node it replaces. `options.isDatetimeColumn` scopes rules 1–2 on a typed seam (item 7). - **`@objectstack/objectql` `engine.ts`**: one stage function, `resolveThenLowerWhere` (resolve, then lower), is the only way any filter position resolves. That covers `find`, `findOne` and `count` (`resolveWhereTokens`); `update` and `delete` (`withResolvedWhere`); and `aggregate`'s `where`, each `aggregations[i].filter` and `having`. `resolveWhereTokens` and `withResolvedWhere` now require the lowering options, so no verb can resolve without lowering. The judge (`judgeWhereAdmission`) runs the same stage. The type reader for `where` and `aggregations[i].filter` is the object's declared `type === 'datetime'`, the same test `SqlDriver` indexes `datetimeFields` by. For `having` it is the aggregated row's column types (`aggregatedRowColumnTypes`, where `max(datetime)` is `datetime`). - **`@objectstack/plugin-security`**: `judgeCompiledComparands` (the RLS compile seam, serving `using` and `check`) lowers every compiled policy filter right after the two faces. `RlsFieldGuard` gains an optional `datetime` set. `SecurityPlugin` fills it from the same declaration pass as the field-name set (`loadObjectFieldNames`) and hands it in at both compile sites. A guard without types reads no column as `datetime`. - **`scripts/adr-anchors/packages__spec__src__data__filter-lowering.ts.json`**: pins ADR-0053 to the module (Prime Directive #13). - Changeset `.changeset/5930-shared-filter-lowering.md`: `@objectstack/spec` minor, `@objectstack/objectql` and `@objectstack/plugin-security` patch. It cites ADR-0053 D-D1 (amended). No face copy is deleted, no driver file is touched, and the analytics `where` / preview door, the read scope and the memory cube face are untouched (step 3). ## The stop line was reached: one evaluator's answers move, on rows with no value The acceptance is answer invariance. Measured, the answers of every driver face stay the same. The engine's own in-process evaluator for `aggregations[i].filter` and `having` (F8, `having-filter.ts`) moves, and only on rows or groups with no value. Before this change F8 was the only face that disagreed with the others on those rows. After it, all faces agree. A/B probe, not committed. Each face answers the filter as written and the lowered filter, on sqlite `SqlDriver` (F1), `InMemoryDriver` (F3), `matchesFilterCondition` (F7) and `matchesAggregationFilter` (F8): | Case set | Filters | Cells | Moved | Filters where the faces disagree, before → after | |---|---|---|---|---| | `FILTER_LOGIC_CASES` over `FILTER_LOGIC_ROWS` | 36 | 144 | 0 | 0 → 0 | | `TEMPORAL_CASES` (plus the resolved `tokenFilter`s) over `TEMPORAL_ROWS` | 32 | 128 | 0 | 0 → 0 | | the same plus a row with no value, each filter also under `$not`, plus `$between` / `$ne` / `$nin` / `$notContains` probes | 70 | 280 | **14, all F8** | **14 → 0** | All 14 moved cells are a `$between` on a `datetime` column with a row whose value is null. F8 kept that row in the range (7 cells) and dropped it under `$not` (7 cells). F1, F3 and F7 exclude it from the range and keep it under `$not`, which is the #5146 / #5298 reading. A second probe found the same class on a number column: `{ $not: { amount: { $lt: 5 } } }` dropped a null `amount` in F8, because JS compares `null < 5` as true. Everywhere else the null row is kept. That probe covered the per-aggregation filter and `having`, one cell each. The decision on whether to keep the two aggregate seams wired is in the report on #5930, with the four-axis frame. This PR carries option A (keep them). Dropping them (option B) removes the two `aggregate` hunks and their two pin rows. ## Mechanism hypotheses — which held - **H1 held**: the doors run in `lowerWhereFilterArray` and tokens resolve after it on every verb. One helper (`resolveThenLowerWhere`) holds "resolve, then lower", and every verb has its own pin. - **H2 — measured: yes, a compiled policy CAN carry an unresolved date token.** `record.signed_on <= '{today}'` compiles to `{ signed_on: { $lte: '{today}' } }`, passes both faces, and reaches `using`'s drivers and `check`'s `matchesFilterCondition` verbatim. Nothing resolves a placeholder on either RLS clause. The RLS lowering therefore runs after the faces (item 3) and reads `'{today}'` as a non-day string it leaves as written, the same as every face does today. This is pinned. - **H3 — measured.** (a) The RLS seam could not read declared types: `RlsFieldGuard` carried names only, but the types are in the same declaration `loadObjectFieldNames` reads. (b) No in-repo or example policy compares any column against a bare day or a date token: 72 non-test `using`/`check` predicate lines, all `==`, `in`, `== null`, `!= null` or `1 == 1`. So no real policy's rows or admitted writes change under either reading. The seam takes the typed reading, because the type-blind one would move `using` answers on SQL for a non-`datetime` column (a text column holding day-prefixed strings, and `$lte '9999-12-31'` on text) in constructible policies. - **H4 held on F1/F3/F7; F8 is the stop-line item above.** Face suites are green after the change. Before: `main` at `085ca6bc1c` has `Test Core` (6/6), `Temporal Conformance (live PG + MySQL)` and `Dogfood Regression Gate` green. - **H5 held**: the output introduces only `$and`, `$or`, `$lt`, `$gte`, `$lte` and `$null`. The unit table pins that closure over `FILTER_LOGIC_CASES`, `TEMPORAL_CASES` and every row. F1, F2, F3, F6, F7 and F8 already compile those. - **H6 held**: the three polarity functions and the `$not` totaliser are the SQL copies' tables cell for cell. The copies stay. ## Evidence (all on head `9ca3698b67`) - New pins: - `packages/spec/src/data/filter-lowering.test.ts`: 46 tests. The rule table, the item-7 scope, idempotence over the table and both case sets, vocabulary closure, copy-on-write, provenance, and pass-through. - `packages/objectql/src/engine-shared-filter-lowering-seam.test.ts`: 11 tests, one per verb and position, plus `{today}` resolved-then-widened, the typed scope, the last supported day, copy-on-write and the judge. - `packages/plugins/plugin-security/src/rls-shared-lowering-seam.test.ts`: 14 tests. Both clauses through `RLSCompiler`, plus `SecurityPlugin.getReadFilter` and `computeWriteCheckFilter` fed the declared `datetime` set. - Existing shape pins updated to expect the lowered driver input. No asserted row count changed. The door suites for number comparands, text operators and filter arrays now compare against the lowering of the door's output. `rls-compiled-comparand-faces` gets the same treatment. `rls-empty-membership-polarity`'s `not in` shape is updated, and its admitted-row count stays 3. - Suites after the change: - spec: 578 files, 17065 passed. - objectql: 341 files, 6729 passed. - plugin-security: 148 files, 3216 passed. - driver-sql: 200 files passed and 11 skipped (live PG/MySQL cells). - driver-memory: 65 files, 1470 passed. - driver-turso: 80 files, 2195 passed. - driver-sqlite-wasm: 36 files, 675 passed. - driver-mongodb: 29 files passed; 5 skipped, because they need a real mongod. - formula: 42 files, 1240 passed. - `typecheck` is green for spec, objectql, plugin-security, the five drivers and formula. `check:driver-conformance` is OK (50 cells). - Ablations, each with the seam committed and then mutated on disk through `scripts/ablation-replace.mjs` and restored (blob equals HEAD, `git diff HEAD` empty): - engine `resolveThenLowerWhere` without the lowering: 8 of 11 seam pins red, across every verb and all three `aggregate` positions. - RLS `judgeCompiledComparands` without the lowering: 8 of 14 red. - `SecurityPlugin` without the `datetime` hand-off: the 2 plugin-level pins red. - The pins that stay green are the pass-through rows, which hold with or without the lowering. - Gates: `node scripts/pm/dispatch-gates.mjs --commands` derived 103 families; all 103 were run with exit 0, and `--ran` reports 0 NOT-MEASURED and 0 UNRUN. Three gates (`check:dual-build-cjs-loads`, `check:i18n`, `check:type-check-debt`) first refused with `PREREQUISITE NOT MET`. They were re-run green after `turbo run build --filter='./packages/*' --filter='./packages/*/*'`. `check:generated` shows 15 of 15 up to date after regenerating `api-surface/` and `export-origins/`. - Lint, as a narrowed run: - The population is the `eslint.config.mjs` block `files: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']`. - `eslint --no-inline-config --format json` over the 13 changed source files reports 13 files, 0 errors and 0 warnings. - The config enables no type-aware linting (every `parserOptions` is `ecmaVersion` / `sourceType` only), so this diff cannot move a verdict on an untouched file. - NOT MEASURED locally: live PostgreSQL / MySQL (the CI job covers them), a real mongod, and Turso remote against a live server. - Changed lines: 1318 (+1289 / −29, 17 files). ## Acceptance notes - **Item 7, when declarations are absent at run time.** Both seams read NO column as `datetime` when the object's declaration is missing (a registry-less host, or a guard without types). They do not apply the rule type-blind. This keeps every driver's answer where it was (`SqlDriver` also widens nothing without a declaration). The step-4 deletion cards should re-read this: once a face's copy is gone, a declaration-less path gets no whole-day bound. - **Two sibling files outside the declared file surface.** `security-plugin.ts` changes in two places: the `datetime` set is read in the existing declaration pass, and it is handed in at the two compile sites. The ADR anchor is one new JSON file. Neither adds a seam. - **RLS `check` with a date token.** `os validate` refuses a `{placeholder}` in a read-scope `using` clause (`validate-rls-predicate-enforceability.ts` judges it through the engine). A `check` clause is not judged there, and nothing resolves the token at run time. A `check` of `record.signed_on <= '{today}'` therefore compares against the literal text, and every ISO value sorts below `{`. Public-door reach is not measured. Carrier: none. - **The analytics read scope.** It reads `using` through `getReadFilter`, so from this PR on it receives the RLS seam's lowered policy. For a policy with a bare-day `$lte` on a `datetime` column (none in-repo), it now keeps the whole day, which is #20733's direction. #20733 itself (the scope's own bound on a caller's filter) is step 3 and is not addressed here. - Serial: #20766 landed before this PR and was merged in cleanly. #20745 has not landed. It edits other regions of `engine.ts` and `filter.zod.ts` prose, and whichever lands second merges `main`. --- _Generated by [Claude Code](https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent df67985 commit cfa9315

17 files changed

Lines changed: 1289 additions & 29 deletions
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
---
2+
'@objectstack/spec': minor
3+
'@objectstack/objectql': patch
4+
'@objectstack/plugin-security': minor
5+
---
6+
7+
feat(spec, objectql, plugin-security): one shared filter lowering, run once at the engine and RLS seams (ADR-0053 D-D1, amended)
8+
9+
Clause-②: yes
10+
11+
`@objectstack/spec/data` exports `lowerFilterCondition(filter, options?)` and its `FilterLoweringOptions` type. It is not exported from the package root entry. It is a pure `FilterCondition → FilterCondition` rewrite that applies three rules once:
12+
13+
- `$between` becomes `$gte` its minimum and `$lte` its maximum.
14+
- A `$lte` whose comparand is a bare `YYYY-MM-DD` day becomes `$lt` the next day, in the calendar-string domain. On the last supported day (`9999-12-31`) a lone `$lte` becomes `{ $null: false }`, and a `$between` keeps only its minimum.
15+
- The NULL-polarity guards the drivers already compile. A `$ne` of a value, a `$nin` or a `$notContains` holds for a row with no value. Every leaf of a `$not` operand is made total.
16+
17+
The rewrite is copy-on-write, idempotent and never refuses. A node it rewrites keeps its filter-subtree provenance mark. With `options.isDatetimeColumn` (a typed seam), the first two rules change only a declared `datetime` column. Without it they apply to every column.
18+
19+
As ADR-0053 D-D1 (amended 2026-09-30) requires, the seams now run it once, after the comparand doors and after filter-token resolution:
20+
21+
- **`@objectstack/objectql`** runs it on every filter position, typed by the object's declared fields. That covers `where` on `find`, `findOne`, `count`, `update` and `delete`, and `aggregate`'s `where`, `aggregations[i].filter` and `having`. `having` is typed by the aggregated row's columns, so `max` of a `datetime` field counts as a `datetime`. Drivers receive the lowered filter. A date macro such as `{today}` is resolved before the lowering reads it.
22+
- **`@objectstack/plugin-security`** runs it on every compiled RLS policy filter (`using` and `check`), right after the two comparand faces. `SecurityPlugin` now hands the compile seam the object's declared `datetime` columns (`RlsFieldGuard.datetime`). A guard without that set treats no column as `datetime`.
23+
24+
Row answers stay the same on every driver. Each driver keeps its own copy of these rules, and every copy gives the same answer on lowered input. One result changes. The engine evaluates `aggregate`'s `aggregations[i].filter` and `having` itself, and that evaluator now treats a row or group with no value the way every driver's `where` already does. It no longer counts such a row in a `$between` on a `datetime` column. It now keeps such a row under a `$not` over an ordering such as `$lt`.
25+
26+
Nothing is removed or renamed, and there is nothing to migrate.

‎packages/objectql/src/engine-filter-array-lowering.test.ts‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -673,8 +673,11 @@ describe('Door 2 lowers FilterArray to FilterCondition before the driver (#5158)
673673
await engine.find('deal', asFilterArrayQuery([['stage', 'in', ['won', 'lost']]]));
674674
expect(lastWhere()).toEqual({ stage: { $in: ['won', 'lost'] } });
675675

676+
// [ADR-0053 D-D1, amended — #5930] `$nin` reaches the driver with the
677+
// shared lowering's NULL escape around it, the #5298 reading every face
678+
// already gives it; the list itself is untouched.
676679
await engine.find('deal', asFilterArrayQuery([['stage', 'not_in', ['lost']]]));
677-
expect(lastWhere()).toEqual({ stage: { $nin: ['lost'] } });
680+
expect(lastWhere()).toEqual({ $and: [{ $or: [{ stage: { $null: true } }, { stage: { $nin: ['lost'] } }] }] });
678681

679682
await engine.find('deal', asFilterArrayQuery([['amount', 'between', [5, 25]]]));
680683
expect(lastWhere()).toEqual({ amount: { $between: [5, 25] } });
@@ -686,7 +689,8 @@ describe('Door 2 lowers FilterArray to FilterCondition before the driver (#5158)
686689
await engine.find('deal', { where: { stage: { $in: [] } } });
687690
expect(lastWhere()).toEqual({ stage: { $in: [] } });
688691
await engine.find('deal', { where: { stage: { $nin: [] } } });
689-
expect(lastWhere()).toEqual({ stage: { $nin: [] } });
692+
// [ADR-0053 D-D1, amended — #5930] …inside the shared lowering's NULL escape.
693+
expect(lastWhere()).toEqual({ $and: [{ $or: [{ stage: { $null: true } }, { stage: { $nin: [] } }] }] });
690694
});
691695

692696
it('the gate does not re-judge list MEMBERS — that is #5234, on another face', async () => {
@@ -728,8 +732,10 @@ describe('Door 2 lowers FilterArray to FilterCondition before the driver (#5158)
728732
});
729733

730734
it('a scalar on a NON-collection operator is untouched', async () => {
735+
// [ADR-0053 D-D1, amended — #5930] `$ne` reaches the driver inside the
736+
// shared lowering's NULL escape; the scalar comparand is untouched.
731737
await engine.find('deal', asFilterArrayQuery([['stage', '!=', 'won']]));
732-
expect(lastWhere()).toEqual({ stage: { $ne: 'won' } });
738+
expect(lastWhere()).toEqual({ $and: [{ $or: [{ stage: { $null: true } }, { stage: { $ne: 'won' } }] }] });
733739
// String bounds on a range comparison stay legal, and since #5685 the
734740
// declaration agrees: `FieldOperatorsSchema` now declares `$gt` as
735741
// number|Date|string|FieldReference, matching the ISO strings the showcase

‎packages/objectql/src/engine-number-comparand-declared-type-door.test.ts‎

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ import {
5252
NUMBER_COMPARAND_DOOR_FIXTURE_OBJECT,
5353
NUMBER_COMPARAND_DOOR_LIST_OPERATORS,
5454
NUMBER_COMPARAND_DOOR_SCALAR_OPERATORS,
55+
lowerFilterCondition,
5556
type EngineAggregateOptions,
5657
type EngineQueryOptions,
5758
type FilterCondition,
@@ -68,6 +69,19 @@ import {
6869

6970
const OBJECT = NUMBER_COMPARAND_DOOR_FIXTURE_OBJECT;
7071

72+
/**
73+
* [ADR-0053 D-D1, amended — #5930] What a driver receives is the door's output
74+
* after the engine's shared lowering (the NULL-polarity guards on `$ne` / `$nin`
75+
* / `$notContains`, the whole-day rule on a declared `datetime`), which runs
76+
* after this door on every verb. The door adds nothing beyond that, so a pin on
77+
* the driver's input compares against the same lowering of the door's answer.
78+
*/
79+
const lowered = (where: unknown): unknown =>
80+
lowerFilterCondition(where, {
81+
isDatetimeColumn: (column) =>
82+
(NUMBER_COMPARAND_DOOR_FIXTURE.fields as Record<string, { type?: string } | undefined>)[column]?.type === 'datetime',
83+
});
84+
7185
interface SeenRead { ast: any }
7286

7387
/** Minimal recording driver — the same witness shape as the sibling door suites. */
@@ -195,7 +209,7 @@ describe('[#20351] the number-comparand declared-type door at the engine collect
195209
const asWritten = JSON.stringify(filter);
196210
await expect(engine.find(OBJECT, { where: filter }), c.name).resolves.toBeDefined();
197211
expect(reads, `${c.name}: the driver must have been read`).toHaveLength(1);
198-
expect(reads[0]?.ast?.where, `${c.name}: the driver must receive the number`).toEqual(c.expectedFilter());
212+
expect(reads[0]?.ast?.where, `${c.name}: the driver must receive the number`).toEqual(lowered(c.expectedFilter()));
199213
// Copy-on-write: the filter belongs to the caller (view metadata, flow config).
200214
expect(JSON.stringify(filter), `${c.name}: the caller's filter must not be edited`).toBe(asWritten);
201215
}
@@ -207,7 +221,7 @@ describe('[#20351] the number-comparand declared-type door at the engine collect
207221
const filter = c.filter();
208222
await expect(engine.find(OBJECT, { where: filter }), c.name).resolves.toBeDefined();
209223
expect(reads, `${c.name}: the driver must have been read`).toHaveLength(1);
210-
expect(reads[0]?.ast?.where, `${c.name}: the filter must reach the driver unchanged`).toEqual(filter);
224+
expect(reads[0]?.ast?.where, `${c.name}: the filter must reach the driver unchanged`).toEqual(lowered(filter));
211225
}
212226
});
213227

@@ -291,7 +305,7 @@ describe('[#20351] the number-comparand declared-type door at the engine collect
291305
expect(reads).toHaveLength(0);
292306
}
293307
await engine.find(OBJECT, { where: { $or: [{ f_text: 'a' }, { $not: { f_number: { $in: ['12', 5] } } }] } });
294-
expect(reads[0]?.ast?.where).toEqual({ $or: [{ f_text: 'a' }, { $not: { f_number: { $in: [12, 5] } } }] });
308+
expect(reads[0]?.ast?.where).toEqual(lowered({ $or: [{ f_text: 'a' }, { $not: { f_number: { $in: [12, 5] } } }] }));
295309
});
296310

297311
it('refuses a {placeholder} against a number field UNRESOLVED — before the token resolver, in the door\'s words', async () => {
Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [ADR-0053 D-D1, amended 2026-09-30 — #5930] The engine's placement of the
5+
* shared `FilterCondition → FilterCondition` lowering (`lowerFilterCondition`,
6+
* `@objectstack/spec/data`): once per filter position, AFTER the comparand doors
7+
* and AFTER filter-token resolution (the amendment's item 3), on every verb that
8+
* takes a filter — `find`, `findOne`, `count`, `update`, `delete`, and
9+
* `aggregate`'s three positions (`where`, `aggregations[i].filter`, `having`).
10+
*
11+
* The witness is what leaves the engine: the recording driver's `where` for the
12+
* verbs a driver compiles, and the operation context a middleware reads for the
13+
* two positions the engine evaluates itself. Row answers are the faces' suites'
14+
* business; these pins say WHERE the rule runs, so reverting any one seam call
15+
* turns its pin red.
16+
*
17+
* Fixture: `closed_at` is the declared `datetime` the whole-day rule is for,
18+
* `due_on` a `date` the typed seam must leave byte-identical (item 7).
19+
*/
20+
21+
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
22+
import type { EngineAggregateOptions, EngineQueryOptions, FilterCondition } from '@objectstack/spec/data';
23+
import { ObjectQL } from './engine.js';
24+
25+
const OBJECT = 'lowering_probe';
26+
27+
const SCHEMA = {
28+
name: OBJECT,
29+
label: 'Lowering probe',
30+
fields: {
31+
id: { name: 'id', type: 'text' },
32+
stage: { name: 'stage', type: 'text' },
33+
amount: { name: 'amount', type: 'number' },
34+
closed_at: { name: 'closed_at', type: 'datetime' },
35+
due_on: { name: 'due_on', type: 'date' },
36+
},
37+
};
38+
39+
interface Seen { verb: string; ast: any }
40+
41+
function makeRecordingDriver() {
42+
const seen: Seen[] = [];
43+
const driver: any = {
44+
name: 'recording', version: '0.0.0', supports: {},
45+
async connect() {}, async disconnect() {}, async checkHealth() { return true; }, async execute() { return null; },
46+
async find(_o: string, ast: any) { seen.push({ verb: 'find', ast }); return []; },
47+
async findOne(_o: string, ast: any) { seen.push({ verb: 'findOne', ast }); return null; },
48+
async count(_o: string, ast: any) { seen.push({ verb: 'count', ast }); return 0; },
49+
async create(_o: string, data: Record<string, unknown>) { return { ...data, id: 'r1' }; },
50+
async update(_o: string, id: string, data: Record<string, unknown>) { return { ...data, id }; },
51+
async updateMany(_o: string, ast: any) { seen.push({ verb: 'updateMany', ast }); return 0; },
52+
async delete() { return true; },
53+
async deleteMany(_o: string, ast: any) { seen.push({ verb: 'deleteMany', ast }); return 0; },
54+
async beginTransaction() { return { commit: async () => {}, rollback: async () => {} }; },
55+
async commit() {}, async rollback() {},
56+
};
57+
return { driver, seen };
58+
}
59+
60+
/** A bare-day upper bound on the datetime column, and what the lowering makes of it. */
61+
const WHOLE_DAY_IN = { closed_at: { $lte: '2026-07-28' } };
62+
const WHOLE_DAY_OUT = { closed_at: { $lt: '2026-07-29' } };
63+
/** A negative-polarity leaf, and its NULL escape. */
64+
const NEGATIVE_IN = { stage: { $ne: 'won' } };
65+
const NEGATIVE_OUT = { $and: [{ $or: [{ stage: { $null: true } }, { stage: { $ne: 'won' } }] }] };
66+
67+
describe('[ADR-0053 D-D1 amended — #5930] the engine runs the shared lowering once per filter position', () => {
68+
let engine: ObjectQL;
69+
let seen: Seen[];
70+
71+
beforeEach(async () => {
72+
const rec = makeRecordingDriver();
73+
seen = rec.seen;
74+
engine = new ObjectQL();
75+
engine.registerDriver(rec.driver, true);
76+
await engine.init();
77+
engine.registry.registerObject(SCHEMA as any, 'test');
78+
seen.length = 0;
79+
});
80+
81+
afterEach(() => {
82+
vi.useRealTimers();
83+
});
84+
85+
const lastWhere = (verb: string) => [...seen].reverse().find((s) => s.verb === verb)?.ast?.where;
86+
87+
it('find: the driver receives the lowered where — whole-day bound and NULL escape', async () => {
88+
await engine.find(OBJECT, { where: WHOLE_DAY_IN });
89+
expect(lastWhere('find')).toEqual(WHOLE_DAY_OUT);
90+
await engine.find(OBJECT, { where: NEGATIVE_IN });
91+
expect(lastWhere('find')).toEqual(NEGATIVE_OUT);
92+
});
93+
94+
it('findOne: the driver receives the lowered where', async () => {
95+
await engine.findOne(OBJECT, { where: WHOLE_DAY_IN });
96+
expect(lastWhere('findOne') ?? lastWhere('find')).toEqual(WHOLE_DAY_OUT);
97+
});
98+
99+
it('count: the driver receives the lowered where', async () => {
100+
await engine.count(OBJECT, { where: NEGATIVE_IN });
101+
expect(lastWhere('count')).toEqual(NEGATIVE_OUT);
102+
});
103+
104+
it('update (multi): the driver receives the lowered where', async () => {
105+
await engine.update(OBJECT, { stage: 'x' }, { where: WHOLE_DAY_IN, multi: true } as any);
106+
expect(lastWhere('updateMany')).toEqual(WHOLE_DAY_OUT);
107+
});
108+
109+
it('delete (multi): the driver receives the lowered where', async () => {
110+
await engine.delete(OBJECT, { where: NEGATIVE_IN, multi: true } as any);
111+
expect(lastWhere('deleteMany')).toEqual(NEGATIVE_OUT);
112+
});
113+
114+
it('aggregate: `where`, `aggregations[i].filter` and `having` are each lowered, before the middleware chain', async () => {
115+
let atMiddleware: any;
116+
engine.registerMiddleware(async (opCtx: any, next: any) => {
117+
if (opCtx.operation === 'aggregate') atMiddleware = structuredClone(opCtx.ast);
118+
return next();
119+
});
120+
await engine.aggregate(OBJECT, {
121+
where: WHOLE_DAY_IN,
122+
groupBy: ['stage'],
123+
aggregations: [
124+
{ function: 'count', alias: 'n' },
125+
{ function: 'count', alias: 'open_n', filter: NEGATIVE_IN },
126+
{ function: 'max', field: 'closed_at', alias: 'last_closed' },
127+
],
128+
having: { last_closed: { $between: ['2026-07-01', '2026-07-28'] } },
129+
} as EngineAggregateOptions);
130+
expect(atMiddleware.where).toEqual(WHOLE_DAY_OUT);
131+
expect(atMiddleware.aggregations[0].filter).toBeUndefined();
132+
expect(atMiddleware.aggregations[1].filter).toEqual(NEGATIVE_OUT);
133+
// `max(closed_at)` is a `datetime` column of the aggregated row, so the
134+
// whole-day rule reaches it; the aggregated row's type is what `having` reads.
135+
expect(atMiddleware.having).toEqual({ last_closed: { $gte: '2026-07-01', $lt: '2026-07-29' } });
136+
// The rows path asks the driver for rows with the lowered `where` too.
137+
expect(lastWhere('find')).toEqual(WHOLE_DAY_OUT);
138+
});
139+
140+
it('item 3: the lowering runs AFTER token resolution — `{today}` resolves to the day, then widens', async () => {
141+
vi.useFakeTimers({ toFake: ['Date'] });
142+
vi.setSystemTime(new Date('2026-03-10T12:00:00.000Z'));
143+
await engine.find(OBJECT, { where: { closed_at: { $lte: '{today}' } } } as EngineQueryOptions);
144+
expect(lastWhere('find')).toEqual({ closed_at: { $lt: '2026-03-11' } });
145+
await engine.update(OBJECT, { stage: 'x' }, { where: { closed_at: { $lte: '{today}' } }, multi: true } as any);
146+
expect(lastWhere('updateMany')).toEqual({ closed_at: { $lt: '2026-03-11' } });
147+
});
148+
149+
it('item 7: a typed seam leaves a `date` column, and a `$between` on a number, byte-identical', async () => {
150+
const where = { due_on: { $lte: '2026-07-28' }, amount: { $between: [5, 25] } };
151+
await engine.find(OBJECT, { where });
152+
expect(lastWhere('find')).toBe(where);
153+
});
154+
155+
it('the last supported day: `$lte` keeps only { $null: false }', async () => {
156+
await engine.find(OBJECT, { where: { closed_at: { $lte: '9999-12-31' } } });
157+
expect(lastWhere('find')).toEqual({ closed_at: { $null: false } });
158+
});
159+
160+
it('copy-on-write: the caller\'s filter object is never edited', async () => {
161+
const where: FilterCondition = { closed_at: { $lte: '2026-07-28' }, stage: { $ne: 'won' } };
162+
const asWritten = JSON.stringify(where);
163+
await engine.find(OBJECT, { where });
164+
await engine.update(OBJECT, { stage: 'x' }, { where, multi: true } as any);
165+
expect(JSON.stringify(where)).toBe(asWritten);
166+
});
167+
168+
it('the judge runs the same stage and gains no verdict from it', () => {
169+
expect(engine.judgeFilter(OBJECT, { closed_at: { $between: ['2026-07-01', '2026-07-28'] } })).toEqual({ ok: true });
170+
expect(engine.judgeFilter(OBJECT, { $not: { stage: { $ne: 'won' } } })).toEqual({ ok: true });
171+
});
172+
});

‎packages/objectql/src/engine-text-operator-declared-type-door.test.ts‎

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,7 @@ import {
6868
TEXT_OPERATOR_DOOR_FIXTURE,
6969
TEXT_OPERATOR_DOOR_FIXTURE_OBJECT,
7070
TEXT_OPERATOR_DOOR_TYPE_CLASSES,
71+
lowerFilterCondition,
7172
type FilterTextCase,
7273
type FilterTextRowsCase,
7374
type TextOperatorDoorCase,
@@ -81,6 +82,19 @@ import { findTextOperatorOverNonTextField } from './text-operator-declared-type-
8182

8283
const OBJECT = TEXT_OPERATOR_DOOR_FIXTURE_OBJECT;
8384

85+
/**
86+
* [ADR-0053 D-D1, amended — #5930] What a driver receives is the door's output
87+
* after the engine's shared lowering (the NULL-polarity guards on `$ne` / `$nin`
88+
* / `$notContains`, the whole-day rule on a declared `datetime`), which runs
89+
* after this door on every verb. The door rewrites nothing, so a pin on the
90+
* driver's input compares against the same lowering of the caller's filter.
91+
*/
92+
const lowered = (where: unknown): unknown =>
93+
lowerFilterCondition(where, {
94+
isDatetimeColumn: (column) =>
95+
(TEXT_OPERATOR_DOOR_FIXTURE.fields as Record<string, { type?: string } | undefined>)[column]?.type === 'datetime',
96+
});
97+
8498
interface SeenRead { ast: any }
8599

86100
/** Minimal recording driver — the same witness shape as the #7872 door suite. */
@@ -187,7 +201,7 @@ describe('[#15773] the text-operator declared-type door at the engine collection
187201
const filter = c.filter();
188202
await expect(engine.find(OBJECT, { where: filter }), c.name).resolves.toBeDefined();
189203
expect(reads, `${c.name}: the driver must have been read`).toHaveLength(1);
190-
expect(reads[0]?.ast?.where, `${c.name}: the filter must reach the driver unchanged`).toEqual(filter);
204+
expect(reads[0]?.ast?.where, `${c.name}: the filter must reach the driver unchanged`).toEqual(lowered(filter));
191205
}
192206
});
193207

@@ -197,7 +211,7 @@ describe('[#15773] the text-operator declared-type door at the engine collection
197211
const filter = c.filter();
198212
await expect(engine.find(OBJECT, { where: filter }), c.name).resolves.toBeDefined();
199213
expect(reads, `${c.name}: the driver must have been read`).toHaveLength(1);
200-
expect(reads[0]?.ast?.where, `${c.name}: the filter must reach the driver unchanged`).toEqual(filter);
214+
expect(reads[0]?.ast?.where, `${c.name}: the filter must reach the driver unchanged`).toEqual(lowered(filter));
201215
}
202216
});
203217

0 commit comments

Comments
 (0)