Repository navigation
Commit cfa9315
Part of #5930 — step 2 of ruling 5902355785 (the seam lowering in the
engine / RLS seams). Steps 3 (the analytics-face seams and the F5 / F11
vocabulary) and 4 (the per-face deletions) remain, so the card stays
open.
Clause-②: yes
## What this does
This implements ADR-0053 D-D1 as amended on 2026-09-30, items 1–7 and 9.
The bare-day upper bound, the `$between` split and the NULL-polarity
guards are applied once, by one shared `FilterCondition →
FilterCondition` lowering. It runs at the engine and RLS seams, after
the comparand doors and after filter-token resolution. Drivers receive
the lowered filter.
- **`@objectstack/spec/data`**: new pure module `filter-lowering.ts`,
exporting `lowerFilterCondition(filter, options?)` and
`FilterLoweringOptions`. It is exported from the `./data` subpath only,
never the root entry (the ruling's D3). The rules:
1. `$between` becomes `$gte` min and `$lte` max. A range holding a `{
$field }` or a non-pair is left whole, for the face that refuses it.
2. A `$lte` on a bare `YYYY-MM-DD` becomes `$lt`
`nextUtcCalendarDay(day)`, in the calendar-string domain, never a
storage form (D-A1). `UNBOUNDED_ABOVE` turns a lone `$lte` into `{
$null: false }`, and a `$between` keeps its minimum. Instants and
`Date`s are never widened.
3. NULL polarity, cell for cell from the four hand copies' tables. `$ne`
of a value, `$nin` and `$notContains` get `{ $or: [{ f: { $null: true }
}, { f: op }] }` (#5298). Every leaf of a `$not` operand is made total
(#5146).
- The module is copy-on-write, so a filter it does not change comes back
as the same reference. It is idempotent, and it recognises its own
guards and the same shapes written by an author. It never refuses. It
carries the #8220 provenance mark onto every node it replaces.
`options.isDatetimeColumn` scopes rules 1–2 on a typed seam (item 7).
- **`@objectstack/objectql` `engine.ts`**: one stage function,
`resolveThenLowerWhere` (resolve, then lower), is the only way any
filter position resolves. That covers `find`, `findOne` and `count`
(`resolveWhereTokens`); `update` and `delete` (`withResolvedWhere`); and
`aggregate`'s `where`, each `aggregations[i].filter` and `having`.
`resolveWhereTokens` and `withResolvedWhere` now require the lowering
options, so no verb can resolve without lowering. The judge
(`judgeWhereAdmission`) runs the same stage. The type reader for `where`
and `aggregations[i].filter` is the object's declared `type ===
'datetime'`, the same test `SqlDriver` indexes `datetimeFields` by. For
`having` it is the aggregated row's column types
(`aggregatedRowColumnTypes`, where `max(datetime)` is `datetime`).
- **`@objectstack/plugin-security`**: `judgeCompiledComparands` (the RLS
compile seam, serving `using` and `check`) lowers every compiled policy
filter right after the two faces. `RlsFieldGuard` gains an optional
`datetime` set. `SecurityPlugin` fills it from the same declaration pass
as the field-name set (`loadObjectFieldNames`) and hands it in at both
compile sites. A guard without types reads no column as `datetime`.
-
**`scripts/adr-anchors/packages__spec__src__data__filter-lowering.ts.json`**:
pins ADR-0053 to the module (Prime Directive #13).
- Changeset `.changeset/5930-shared-filter-lowering.md`:
`@objectstack/spec` minor, `@objectstack/objectql` and
`@objectstack/plugin-security` patch. It cites ADR-0053 D-D1 (amended).
No face copy is deleted, no driver file is touched, and the analytics
`where` / preview door, the read scope and the memory cube face are
untouched (step 3).
## The stop line was reached: one evaluator's answers move, on rows with
no value
The acceptance is answer invariance. Measured, the answers of every
driver face stay the same. The engine's own in-process evaluator for
`aggregations[i].filter` and `having` (F8, `having-filter.ts`) moves,
and only on rows or groups with no value. Before this change F8 was the
only face that disagreed with the others on those rows. After it, all
faces agree.
A/B probe, not committed. Each face answers the filter as written and
the lowered filter, on sqlite `SqlDriver` (F1), `InMemoryDriver` (F3),
`matchesFilterCondition` (F7) and `matchesAggregationFilter` (F8):
| Case set | Filters | Cells | Moved | Filters where the faces disagree,
before → after |
|---|---|---|---|---|
| `FILTER_LOGIC_CASES` over `FILTER_LOGIC_ROWS` | 36 | 144 | 0 | 0 → 0 |
| `TEMPORAL_CASES` (plus the resolved `tokenFilter`s) over
`TEMPORAL_ROWS` | 32 | 128 | 0 | 0 → 0 |
| the same plus a row with no value, each filter also under `$not`, plus
`$between` / `$ne` / `$nin` / `$notContains` probes | 70 | 280 | **14,
all F8** | **14 → 0** |
All 14 moved cells are a `$between` on a `datetime` column with a row
whose value is null. F8 kept that row in the range (7 cells) and dropped
it under `$not` (7 cells). F1, F3 and F7 exclude it from the range and
keep it under `$not`, which is the #5146 / #5298 reading. A second probe
found the same class on a number column: `{ $not: { amount: { $lt: 5 } }
}` dropped a null `amount` in F8, because JS compares `null < 5` as
true. Everywhere else the null row is kept. That probe covered the
per-aggregation filter and `having`, one cell each.
The decision on whether to keep the two aggregate seams wired is in the
report on #5930, with the four-axis frame. This PR carries option A
(keep them). Dropping them (option B) removes the two `aggregate` hunks
and their two pin rows.
## Mechanism hypotheses — which held
- **H1 held**: the doors run in `lowerWhereFilterArray` and tokens
resolve after it on every verb. One helper (`resolveThenLowerWhere`)
holds "resolve, then lower", and every verb has its own pin.
- **H2 — measured: yes, a compiled policy CAN carry an unresolved date
token.** `record.signed_on <= '{today}'` compiles to `{ signed_on: {
$lte: '{today}' } }`, passes both faces, and reaches `using`'s drivers
and `check`'s `matchesFilterCondition` verbatim. Nothing resolves a
placeholder on either RLS clause. The RLS lowering therefore runs after
the faces (item 3) and reads `'{today}'` as a non-day string it leaves
as written, the same as every face does today. This is pinned.
- **H3 — measured.** (a) The RLS seam could not read declared types:
`RlsFieldGuard` carried names only, but the types are in the same
declaration `loadObjectFieldNames` reads. (b) No in-repo or example
policy compares any column against a bare day or a date token: 72
non-test `using`/`check` predicate lines, all `==`, `in`, `== null`, `!=
null` or `1 == 1`. So no real policy's rows or admitted writes change
under either reading. The seam takes the typed reading, because the
type-blind one would move `using` answers on SQL for a non-`datetime`
column (a text column holding day-prefixed strings, and `$lte
'9999-12-31'` on text) in constructible policies.
- **H4 held on F1/F3/F7; F8 is the stop-line item above.** Face suites
are green after the change. Before: `main` at `085ca6bc1c` has `Test
Core` (6/6), `Temporal Conformance (live PG + MySQL)` and `Dogfood
Regression Gate` green.
- **H5 held**: the output introduces only `$and`, `$or`, `$lt`, `$gte`,
`$lte` and `$null`. The unit table pins that closure over
`FILTER_LOGIC_CASES`, `TEMPORAL_CASES` and every row. F1, F2, F3, F6, F7
and F8 already compile those.
- **H6 held**: the three polarity functions and the `$not` totaliser are
the SQL copies' tables cell for cell. The copies stay.
## Evidence (all on head `9ca3698b67`)
- New pins:
- `packages/spec/src/data/filter-lowering.test.ts`: 46 tests. The rule
table, the item-7 scope, idempotence over the table and both case sets,
vocabulary closure, copy-on-write, provenance, and pass-through.
- `packages/objectql/src/engine-shared-filter-lowering-seam.test.ts`: 11
tests, one per verb and position, plus `{today}` resolved-then-widened,
the typed scope, the last supported day, copy-on-write and the judge.
-
`packages/plugins/plugin-security/src/rls-shared-lowering-seam.test.ts`:
14 tests. Both clauses through `RLSCompiler`, plus
`SecurityPlugin.getReadFilter` and `computeWriteCheckFilter` fed the
declared `datetime` set.
- Existing shape pins updated to expect the lowered driver input. No
asserted row count changed. The door suites for number comparands, text
operators and filter arrays now compare against the lowering of the
door's output. `rls-compiled-comparand-faces` gets the same treatment.
`rls-empty-membership-polarity`'s `not in` shape is updated, and its
admitted-row count stays 3.
- Suites after the change:
- spec: 578 files, 17065 passed.
- objectql: 341 files, 6729 passed.
- plugin-security: 148 files, 3216 passed.
- driver-sql: 200 files passed and 11 skipped (live PG/MySQL cells).
- driver-memory: 65 files, 1470 passed.
- driver-turso: 80 files, 2195 passed.
- driver-sqlite-wasm: 36 files, 675 passed.
- driver-mongodb: 29 files passed; 5 skipped, because they need a real
mongod.
- formula: 42 files, 1240 passed.
- `typecheck` is green for spec, objectql, plugin-security, the five
drivers and formula. `check:driver-conformance` is OK (50 cells).
- Ablations, each with the seam committed and then mutated on disk
through `scripts/ablation-replace.mjs` and restored (blob equals HEAD,
`git diff HEAD` empty):
- engine `resolveThenLowerWhere` without the lowering: 8 of 11 seam pins
red, across every verb and all three `aggregate` positions.
- RLS `judgeCompiledComparands` without the lowering: 8 of 14 red.
- `SecurityPlugin` without the `datetime` hand-off: the 2 plugin-level
pins red.
- The pins that stay green are the pass-through rows, which hold with or
without the lowering.
- Gates: `node scripts/pm/dispatch-gates.mjs --commands` derived 103
families; all 103 were run with exit 0, and `--ran` reports 0
NOT-MEASURED and 0 UNRUN. Three gates (`check:dual-build-cjs-loads`,
`check:i18n`, `check:type-check-debt`) first refused with `PREREQUISITE
NOT MET`. They were re-run green after `turbo run build
--filter='./packages/*' --filter='./packages/*/*'`. `check:generated`
shows 15 of 15 up to date after regenerating `api-surface/` and
`export-origins/`.
- Lint, as a narrowed run:
- The population is the `eslint.config.mjs` block `files:
['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']`.
- `eslint --no-inline-config --format json` over the 13 changed source
files reports 13 files, 0 errors and 0 warnings.
- The config enables no type-aware linting (every `parserOptions` is
`ecmaVersion` / `sourceType` only), so this diff cannot move a verdict
on an untouched file.
- NOT MEASURED locally: live PostgreSQL / MySQL (the CI job covers
them), a real mongod, and Turso remote against a live server.
- Changed lines: 1318 (+1289 / −29, 17 files).
## Acceptance notes
- **Item 7, when declarations are absent at run time.** Both seams read
NO column as `datetime` when the object's declaration is missing (a
registry-less host, or a guard without types). They do not apply the
rule type-blind. This keeps every driver's answer where it was
(`SqlDriver` also widens nothing without a declaration). The step-4
deletion cards should re-read this: once a face's copy is gone, a
declaration-less path gets no whole-day bound.
- **Two sibling files outside the declared file surface.**
`security-plugin.ts` changes in two places: the `datetime` set is read
in the existing declaration pass, and it is handed in at the two compile
sites. The ADR anchor is one new JSON file. Neither adds a seam.
- **RLS `check` with a date token.** `os validate` refuses a
`{placeholder}` in a read-scope `using` clause
(`validate-rls-predicate-enforceability.ts` judges it through the
engine). A `check` clause is not judged there, and nothing resolves the
token at run time. A `check` of `record.signed_on <= '{today}'`
therefore compares against the literal text, and every ISO value sorts
below `{`. Public-door reach is not measured. Carrier: none.
- **The analytics read scope.** It reads `using` through
`getReadFilter`, so from this PR on it receives the RLS seam's lowered
policy. For a policy with a bare-day `$lte` on a `datetime` column (none
in-repo), it now keeps the whole day, which is #20733's direction.
#20733 itself (the scope's own bound on a caller's filter) is step 3 and
is not addressed here.
- Serial: #20766 landed before this PR and was merged in cleanly. #20745
has not landed. It edits other regions of `engine.ts` and
`filter.zod.ts` prose, and whichever lands second merges `main`.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent df67985 commit cfa9315
17 files changed
Lines changed: 1289 additions & 29 deletions
File tree
- .changeset
- packages
- objectql/src
- plugins/plugin-security/src
- spec
- api-surface
- export-origins
- src/data
- scripts/adr-anchors
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
Lines changed: 9 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
673 | 673 | | |
674 | 674 | | |
675 | 675 | | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
676 | 679 | | |
677 | | - | |
| 680 | + | |
678 | 681 | | |
679 | 682 | | |
680 | 683 | | |
| |||
686 | 689 | | |
687 | 690 | | |
688 | 691 | | |
689 | | - | |
| 692 | + | |
| 693 | + | |
690 | 694 | | |
691 | 695 | | |
692 | 696 | | |
| |||
728 | 732 | | |
729 | 733 | | |
730 | 734 | | |
| 735 | + | |
| 736 | + | |
731 | 737 | | |
732 | | - | |
| 738 | + | |
733 | 739 | | |
734 | 740 | | |
735 | 741 | | |
| |||
Lines changed: 17 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
| 55 | + | |
55 | 56 | | |
56 | 57 | | |
57 | 58 | | |
| |||
68 | 69 | | |
69 | 70 | | |
70 | 71 | | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
71 | 85 | | |
72 | 86 | | |
73 | 87 | | |
| |||
195 | 209 | | |
196 | 210 | | |
197 | 211 | | |
198 | | - | |
| 212 | + | |
199 | 213 | | |
200 | 214 | | |
201 | 215 | | |
| |||
207 | 221 | | |
208 | 222 | | |
209 | 223 | | |
210 | | - | |
| 224 | + | |
211 | 225 | | |
212 | 226 | | |
213 | 227 | | |
| |||
291 | 305 | | |
292 | 306 | | |
293 | 307 | | |
294 | | - | |
| 308 | + | |
295 | 309 | | |
296 | 310 | | |
297 | 311 | | |
| |||
Lines changed: 172 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
Lines changed: 16 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
| 71 | + | |
71 | 72 | | |
72 | 73 | | |
73 | 74 | | |
| |||
81 | 82 | | |
82 | 83 | | |
83 | 84 | | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
84 | 98 | | |
85 | 99 | | |
86 | 100 | | |
| |||
187 | 201 | | |
188 | 202 | | |
189 | 203 | | |
190 | | - | |
| 204 | + | |
191 | 205 | | |
192 | 206 | | |
193 | 207 | | |
| |||
197 | 211 | | |
198 | 212 | | |
199 | 213 | | |
200 | | - | |
| 214 | + | |
201 | 215 | | |
202 | 216 | | |
203 | 217 | | |
| |||
0 commit comments