Repository navigation
Commit 4a1df19
Fixes #20351
Clause-②: no (narrowing)
## What this adds
Lane (2) of the two-lane route #20336 took on #15661's precedent: the
engine door that consults the contract PR #20414 published in
`@objectstack/spec/data` (`filter-number-comparand-declared-type.ts`).
The contract half is untouched; `packages/spec` is not in this diff.
- **The door**,
`packages/objectql/src/number-comparand-declared-type-door.ts`, beside
the text-operator and temporal doors. For each comparand at a judged
position on a declared numeric field it asks
`numberComparandDoorVerdict` and routes the answer:
- `door-refusal`: throws `INVALID_FILTER` / 400 (the existing
`invalidFilterError` envelope) in the contract's words,
`numberComparandRefusalMessage`, before any driver is resolved;
- `narrows`: rewrites the numeric string to its number, copy-on-write
(the caller's filter is never edited, and a filter with nothing to
narrow comes back by reference);
- `passes` / `deferred`: leaves it alone.
The door reads no string itself. The grammar, the judged types
(`NUMERIC_VALUE_TYPES` by identity), the judged operators
(`NUMBER_COMPARAND_DOOR_SCALAR_OPERATORS` /
`NUMBER_COMPARAND_DOOR_LIST_OPERATORS`) and the words are all the
spec's.
- **Its calls in `engine.ts`, at the collection point only**, fifth
after the temporal door in the same order everywhere:
- `lowerWhereFilterArray`, object form (before
`normalizeFilterComparandTypes`) and array form (on the lowered
condition). So `find` / `findOne` / `count` / `aggregate` / `update` /
`delete` and the judge-only `judgeFilter` (`judgeWhereAdmission` calls
the same function) all inherit it;
- each per-aggregation `filter`, rooted at `aggregations[i].filter`,
against the object's declared fields;
- `having`, after the temporal `having` door, over the columns
`aggregatedRowColumnClasses` classes `numeric` (`count` / `sum` / `avg`,
and a groupBy or `min` / `max` of a numeric field).
The `judgeWhereAdmission` docblock's pipeline list names the new door
(comment only).
- **A changeset**, `.changeset/20351-number-comparand-door.md`:
`@objectstack/objectql` `minor`, BREAKING, `Clause-②: no (narrowing)`, a
FROM → TO line, and the ADR-0087 disposition `not-required
(no-migration-prescription)` in the form PR #20469 and PR #20370 used.
`@objectstack/objectql`'s root exports are unchanged: the door module is
not re-exported from `index.ts` or `core.ts`, like its two siblings.
## What it does to the card's three answers
Measured through `engine.find` / `engine.aggregate` and `POST
/api/v1/data/:object/query`, three rows (5, 12, 30), on InMemoryDriver,
SqlDriver on SQLite and SqlDriver on a local PostgreSQL 16.13 server:
| position | comparand on a `number` field | base `3062e5001`: memory ·
SQLite · PostgreSQL | this branch, all three |
|:--|:--|:--|:--|
| `where` | `$gt` / `$eq` / implicit / a `$in` member `"abc"` | 200 no
rows · 200 no rows · 500 `DATABASE_ERROR` | 400 `INVALID_FILTER` |
| `where` | `$ne "abc"` | every row · every row · 500 | 400 |
| `where` | `$eq ""` | no rows · no rows · 500 | 400 |
| `where`, REST | `$gt "{current_user_id}"` (resolved to the user's id)
| no rows · no rows · 500 | 400 |
| per-aggregation `filter` | `$gt "abc"` / `$ne "abc"` | count 0 / count
3, on all three | 400 |
| `having` on `sum(amount)` | `$gt "abc"` / `$ne "abc"` | no group /
every group, on all three | 400 |
| `where` | `$gt "12"` / `$eq "12"` | **no rows** · 1 row · 1 row | 1
row on all three |
| all three positions | `$gt 10` (the numeric control) | 2 rows / count
2 / both groups | the same |
The last-but-one row is the narrowing's point: InMemoryDriver compared
`"12"` as a string and matched nothing.
## Premise check, and the order's hypotheses
- **H1 holds, reproduced at `3062e5001`** (the table above). SqlDriver's
server-side log line on PostgreSQL reads `(22P02) … invalid input syntax
for type numeric: "abc"`.
- **H2: the collection point is where the order says**, and the new door
sits after the temporal door at each call. `judgeFilter` passes through
it: `judgeWhereAdmission` calls `lowerWhereFilterArray` (pinned:
`judgeFilter` answers `INVALID_FILTER` / 400 for `"abc"` and `{ ok: true
}` for `"12"`). **RLS / sharing / tenant predicates do NOT pass through
it at runtime.** The middleware chain composes them onto the AST after
this seam, and `plugin-security`'s `judgeCompiledComparands` runs only
the two field-agnostic faces (`rls-compiler.ts`, the `[#20212]` block).
A policy predicate reaches this door at authoring instead:
`validateRlsPredicateEnforceability` asks the engine's `judgeFilter`
when the host hands the rule a judge.
- **H3 holds.** The verdict is `numberComparandDoorVerdict` over
`NUMBER_COMPARAND_DOOR_JUDGED_TYPES` with the scalar and list operators,
and the words are `numberComparandRefusalMessage`. A numeric string is
**narrowed** to its number (the verdict's `narrows`, as the contract
review's judgment 7 asks). The pins assert the rewritten filter the
driver receives, not only the 400s.
- **H4: MySQL is NOT MEASURED.** No MySQL server is available in this
container. The REST suite carries a MySQL cell, a named skip without
`OS_TEST_MYSQL_URL`.
- **H5: neither consults the same verdict everywhere.**
- `service-analytics`: the ObjectQL strategy sends the caller's `where`
into `engine.aggregate` and asks `judgeFilter` about the read scope
(`assertReadScopeAdmittedByEngine`), so both inherit the door. The
**NativeSQL strategy's decline** (`NativeSQLStrategy.canHandle`)
declines a cross-field reference and an uninterpretable temporal
comparand, but does not consult the number verdict. So a raw-SQL
deployment compiles `amount > 'abc'` itself (read at source, not
measured).
- **The metadata save door:** RLS `using` is judged through
`judgeFilter`, as above. No lint rule reads `numberComparandDoorVerdict`
(`git grep` over `packages/lint/src` finds zero hits), so a stored view
or report filter comparing a number field with a non-numeric string
saves clean and is refused at query time.
Both are reported as findings below and are not edited here.
## The staged `$empty` row: pinned at the door alone
`NUMBER_COMPARAND_DOOR_CASES` carries PR #20442's `unjudged` `$empty`
row. The engine suite partitions it out of the end-to-end drive and pins
it at the door alone: `findNonNumericComparand` answers `null`, and
`narrowNumberComparands` returns the same reference. A partition guard
asserts the table is split exactly. So the row can neither turn this
suite red for a reason that is not the door's, nor vanish unnoticed.
The contract's `formula` rows are partitioned the same way the text
door's suite does it: they are pinned in the direction they answer
(`INVALID_FIELD` / 400 from the #8296 materializable door, one door
earlier). The door's own walk is pinned to judge `f_formula_number` by
its `returnType`.
## Tests (at `09da7a4cc`, the merged head, unless noted)
- **New:
`packages/objectql/src/engine-number-comparand-declared-type-door.test.ts`,
29 tests.** It drives the contract's case table through a real
`ObjectQL` and a recording driver, per the contract header:
- of the table's 137 cases, 51 refusals (the 52nd is the
`f_formula_number` row), asserting `code` + `status` + `httpStatus`,
every `mustMention` substring, and no driver read. All 8 refusal forms
and every judged position are covered, both ways;
- 23 `narrows` cases, asserting the driver receives `c.expectedFilter()`
and the caller's filter is untouched;
- 57 `passes` cases, reaching the driver unchanged;
- the formula (5) and `$empty` (1) partitions above.
Beside the table:
- every verb (read and write, no read and no write on refusal);
- `FilterArray` sugar, both refused and narrowed;
- `$and` / `$or` / `$not`;
- a placeholder refused unresolved;
- `judgeFilter`;
- the per-aggregation `filter`, refused at its path, with numeric
strings counting what their numbers count;
- `having` on `count` / `sum` / a numeric `min`, refused, narrowed, and
a placeholder on `count`;
- the four `findData` doors (`where` object, `$filter`, filter AST,
implicit query parameter), both ways;
- the registry-less, unknown-key, by-reference and
unrecognised-combinator guards.
- **New: `packages/rest/src/data-number-comparand-door.test.ts`.** It
runs `POST /api/v1/data/:object/query` and `engine.find` /
`engine.aggregate` over SqlDriver, with a cell per dialect:
- `where`: 9 refused spellings;
- the per-aggregation `filter`;
- `having` on `sum` and `max(currency)`, on the native and the rows
path;
- numeric-string controls, equal to their numbers at all three
positions.
The SQLite cell always runs. The PostgreSQL cell ran against the local
server: 3/3 passed at `09da7a4cc`. ⚠️ **No CI job provisions
`OS_TEST_POSTGRES_URL` for `@objectstack/rest`.** The `Temporal
Conformance (live PG + MySQL)` job runs `driver-sql`'s suite,
`metadata-protocol`'s `live-*` files and one `runtime` file, and a
`driver-sql`-only pin cannot reach an engine door. So the live cells are
red-capable and un-run in CI; the local run above is their measurement.
- **Re-pinned, test side only.** Four existing pins asserted the old
silent answer for a string on a numeric column:
- `engine-aggregate-having-temporal-door.test.ts`: the three "a string
on sum / count / avg keeps no group" rows move to a refusal pin in the
number door's words;
- `engine-aggregate-positions.test.ts`: the "unknown token on count" row
moves to a text column, which neither field-aware door judges, and the
count-column case is pinned in the new suite;
- `rest-aggregate-numeric-having.test.ts`: three rows move from `KEPT`
to a `REFUSED` table, SQLite and PostgreSQL both run locally;
- `data-query-having-temporal-door.test.ts`: "a string on sum" becomes a
number control plus a refusal pin.
- `pnpm --filter @objectstack/objectql exec vitest run --project local
--maxWorkers=2`: 330 files, 6118 tests passed. `--project repo`: 1 file,
5 passed.
- `pnpm --filter @objectstack/rest exec vitest run --project local
--maxWorkers=2`: 219 files, 3930 passed, 40 skipped. `--project repo`: 1
file, 8 passed.
- The live PostgreSQL run of the two PostgreSQL-capable REST files: 30
passed (15 live-postgres), 15 skipped (MySQL).
- `pnpm --filter @objectstack/objectql typecheck` and `pnpm --filter
@objectstack/rest typecheck`: exit 0. `check:test-typecheck` is OK for
both, with no debt added (objectql 40 files / 234 errors held; rest 0 /
0).
## Ablation (reverse verification)
The mutation is in the door's walk, which every position routes through:
`if (!meta || numberComparandFieldVerdict(meta) !== 'judged') continue;`
→ `if (meta || 'ABLATION_20351') continue;`. It is made with
`scripts/ablation-replace.mjs`: anchor 1 → 0, and blob `aa4a3247` →
`56a5bdf6`.
- **Mutated leg:** after `pnpm --filter @objectstack/objectql build`,
`ablation-dist-preflight` found the marker in 4 built files. The
objectql door suite went **20 failed / 8 passed**; the 8 are the guards
and partitions that do not depend on the door firing. The REST door
suite went **6 failed / 3 skipped**. The SQLite cell answered `200` with
`records: []`, the PostgreSQL cell `500 DATABASE_ERROR`, and the
per-aggregation `$in ["5","30"]` counted 0 instead of 2: the card's
defect, back.
- **Restore leg:** the blob is back to `aa4a3247` = HEAD and `git diff
HEAD` is empty. After a rebuild, `ablation-dist-preflight --absent`
found the marker absent from all 14 built files and the tree clean. Both
suites passed again (28/28 and 6 + 3 skipped at that commit,
`872d7708b`).
## Gates
`node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` at **`09da7a4cc`** derives 65 commands, the
same list as at the first merged head. All 65 ran with each exit code
recorded before any pipe:
- 63 exited 0 on the first pass;
- `check:dual-build-cjs-loads` and `check:type-check-debt` answered exit
3 (PREREQUISITE NOT MET) until the whole workspace was built (`turbo run
build --filter=!@objectstack/docs`, 72/72), then exited 0.
`dispatch-gates --ran`: 65 derived, 65 run, 0 NOT-MEASURED, 0 UNRUN. The
branch merged `origin/main` twice with true merge commits, no rebase and
no force-push; the last merge base is `45f428d8f`.
## Acceptance notes
- **`having` words.** A numeric aggregated column has no declared
`FieldType`, so the door hands the verdict `number` (the member of the
numeric class the column holds). The spec's words then read "compares a
declared number field against … at having.total.$gt". The `not-a-number`
clause ("backends answer it differently (PostgreSQL with a server
error)") is the `where` fact: `having` is evaluated by the engine on
every driver, and there it kept no group, or every group under `$ne`.
The words are the contract's, and the path names the position.
- **Out of the contract, measured, unchanged:** a boolean or a `Date`
compared against a number field is not judged (the contract judges
strings). `$gt true`: no rows on memory, every row on SQLite, 500 on
PostgreSQL. A `Date`: no rows · no rows · 500. Both hold on the base and
on this branch. Handed to the seat below.
- **Not measured:** MySQL (no server in this container);
`driver-mongodb` (the door sits in front of it); the NativeSQL analytics
path (read at source).
- **Line budget:** n/a (no `skills/**` path in the diff).
## Out of scope, handed to the seat (not filed by this dev)
1. **Class (a), reach measured at REST.** A boolean or a `Date`
comparand against a number field answers `500 DATABASE_ERROR` on
PostgreSQL. It is `POST /api/v1/data/:object/query` with `where: {
amount: { $gt: true } }` against a `number` field, on a local PostgreSQL
16 server, on the base and on this branch. The contract review of PR
#20414 said to file this only if it answered 500; it does. Dedupe words:
`boolean comparand number field postgres 500` · `Date comparand numeric
column database_error` · `non-string comparand declared number type`.
2. **Carrier: none. Noted, not filed (read at source, reach not
measured).** `NativeSQLStrategy.canHandle` does not consult the number
verdict, so a raw-SQL analytics deployment does not fall through to this
door. Dedupe words: `native sql decline number comparand` · `analytics
raw sql non-numeric string`.
3. **Carrier: none. Noted, not filed (read at source, no named
producer).** No authoring rule reads `numberComparandDoorVerdict`, so a
stored view or report filter with a non-numeric string on a number field
saves clean and is refused at query time. Dedupe words: `stored view
filter non-numeric number field lint` · `authoring number comparand
verdict`.
4. **Carrier: none. Noted, not filed.** The runtime RLS compile
(`judgeCompiledComparands`) does not consult the number verdict. The
authoring judge does, when present. Dedupe words: `rls compiled
predicate number comparand` · `policy using string against number
field`.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent 2b53993 commit 4a1df19
9 files changed
Lines changed: 1219 additions & 16 deletions
File tree
- .changeset
- packages
- objectql/src
- rest/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
Lines changed: 16 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
331 | 331 | | |
332 | 332 | | |
333 | 333 | | |
334 | | - | |
335 | | - | |
336 | | - | |
337 | 334 | | |
338 | 335 | | |
339 | 336 | | |
| |||
353 | 350 | | |
354 | 351 | | |
355 | 352 | | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
356 | 369 | | |
357 | 370 | | |
358 | 371 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
242 | 242 | | |
243 | 243 | | |
244 | 244 | | |
245 | | - | |
246 | | - | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
247 | 250 | | |
248 | 251 | | |
249 | 252 | | |
| |||
0 commit comments