Repository navigation
Commit 81e69ca
fix(service-analytics)!: the read scope, the where tree and the draft preview take the shared lowering's bound and NULL guards; their own copies are deleted (#21553)
Fixes #21417
Clause-②: no (narrowing)
#5930 step 4 (`domain:services`) covers three faces:
- **F9**, the analytics read scope;
- **F10**, the analytics `where` tree, its native compiler and its
`/analytics/sql` echo;
- **F11**, the draft preview.
On all three, the shared lowering (`lowerFilterCondition`,
`@objectstack/spec/data`) is now the one source of the whole-day bound,
the `$between` split and the NULL-polarity guards, and each face's own
copy is deleted. With F11 in, nothing remains open on the card.
Claim: the PM's `Claim:` comment 5962872882 and its revision 1,
5963878819 (Q1 = A: one PR; Q2 = A: F11 in, with one
`analytics-service.ts` call site). Branch
`claude/issue-21417-analytics-faces-delete-copies`. Base `0b8239111`,
with `origin/main` merged twice (no rebase).
## Each face and its typed column reader
| Face | Copy deleted | Reader that now carries the rule |
|:--|:--|:--|
| F10, native SQL strategy | `buildFilterClause`'s bare-day `lte` arm;
the `dateRange` window arm's own widening; the `$not`-operand rewrite
and the #5298 leaf wrap with their polarity tables
(`filter-normalizer.ts`) | `declaredDatetimeLowering(ctx, target,
'type-blind')`, the context's `declaredFieldType` hook, which the plugin
answers from the engine registry (`sourceFieldMeta`). A column the hook
names no type for is read type-blind (ADR-0053 D-D1 item 7), because
this face is the last seam before its statement runs. The window is the
`{ $gte, $lte }` pair, lowered by the same reader (item 8). |
| F10, ObjectQL strategy (engine hand-off and echo) | the echo's own
window widening | the same hook, read `'as-written'` for a column it
names no type for: the engine's `where` seam reads the object's own
field map, and applies item 7 itself to an object with none. |
| F9, read scope | the `$not`-operand rewrite and its three polarity
tables; the `IS NULL OR` wrap on `$ne` / `$nin` / `$notContains`. F9
never had a whole-day copy (measured). |
`ReadScopeCompileOptions.declaredValueShape`, filled by both consumers
from the context's `declaredValueShape` hook. |
| F11, draft preview | `lteBound` (the `$lte` and `$between`
predicates); the window's own bare-day, last-day and `'~'`-suffix
readings | `declaredPreviewLowering`, over the drafted object's declared
types. `queryDataset`'s preview branch passes
`sourceFieldMeta(dataset.object, field)?.type`; that one call site is
the only `analytics-service.ts` edit. A declared `datetime` is
rewritten, any other declared type is compared as written, and a column
with no declared type is read type-blind (item 7). The window is the `{
$gte, $lte }` pair, matched by the same `matchesWhere` as the `where`
(item 8). |
The `$between` split in `fieldLeaves` stays. After the lowering it only
meets a column declared something other than `datetime`, where it is
structural: inclusive at both ends, which is the typed drivers'
comparison.
## Measured, before and after
**F9 and F10** were measured through the plugin's own composition over a
real `ObjectQL` engine with `SqlDriver`. The databases were SQLite
(better-sqlite3) and PostgreSQL 16.14 (a private server, session
timezone America/New_York, stopped and deleted afterwards).
Rows:
- `signed_at` (datetime): `2026-07-27T10Z`, `2026-07-28T00Z`,
`2026-07-28T10Z`, `2026-07-29T10Z`, null;
- `due_on` (date): the same days;
- `note` (text): `'2026-07-27'`, `'2026-07-28'`, `'2026-07-28 late'`,
`'n'`, null.
**F11** was measured through the production path:
`AnalyticsServicePlugin`'s own composition (`sourceFieldMeta` from the
engine registry, `draftRowsResolver` from a `protocol` service returning
the same rows as a pending seed draft), then `queryDataset` with
`previewDrafts`, over SQLite. "Before" is the same harness on the
pre-F11 sources (`d2f452b88`); it reproduces the round-1 preview column
cell for cell.
The `note` cells. Every move lands on the engine's answer:
| `note` cell | native before | preview before | after: native = preview
= engine = ObjectQL |
|:--|:--|:--|:--|
| `$lte '2026-07-28'` | r1, r2, r3 | r1, r2, r3 | r1, r2 |
| `$lte '9999-12-31'` (the carrier note's cell) | r1, r2, r3, r4 | r1,
r2, r3 | r1, r2, r3 |
| `$between ['2026-07-28', '2026-07-28']` | r2, r3 | r2, r3 | r2 |
| `$between ['2026-07-28', '9999-12-31']` | r2, r3, r4 | r2, r3 | r2, r3
|
| `$not: { $lte '2026-07-28' }` | r4, r5 | r4, r5 | r3, r4, r5 |
| window `['2026-07-28', '2026-07-28']` | r2, r3 | r2, r3 | r2 |
| window `['2026-07-28', '9999-12-31']` | r2, r3, r4 | r2, r3, r4 | r2,
r3 |
Native results are identical on SQLite and PostgreSQL.
These are unchanged:
- every `datetime` and `date` cell, on all three faces;
- every cell of the ObjectQL face;
- every read-scope cell;
- every native cell of a host with no typed reader.
**A host with no typed reader** reads every column type-blind, as
ADR-0053 D-D1 item 7 prescribes for a seam that cannot read
declarations. The engine lane's seams already do this (`engine.ts`
`declaredDatetimeLowering`, the RLS seam). It does not read every column
as "not datetime": read that way, the native face would have lost the
last day of every `datetime` bound (measured: `datetime $lte a day` went
from r1, r2, r3 to r1, r2).
- On the native face, such a host keeps its old answers.
- On the preview, one pair of cells moves. A preview built without
`sourceFieldMeta`, or an object the registry does not hold yet, now
answers `note $lte '9999-12-31'` with r1, r2, r3, r4 (was r1, r2, r3)
and the `$between` to that day with r2, r3, r4 (was r2, r3).
- Why it moved: the deleted `lteBound` sniffed instants on the last day.
Every other type-blind seam does not, and this host now answers as they
do.
- These cells, where "answer as the typed drivers" cannot hold without a
reader, are pinned by name.
**The preview window's full-timestamp end** (the deleted `'~'` reading,
"that instant and its own sub-values"). On a `datetime` column over
SQLite:
- A canonical end (`…T10:00:00.000Z`) answers as the engine, before and
after.
- An end spelled shorter than the stored value is now compared as text,
exactly as the preview's `where` already compared the same `{ $gte, $lte
}` pair (that column did not move).
The cells that moved:
| end | engine | preview before | preview after |
|:--|:--|:--|:--|
| `…T10:00`, main rows | r1, r2, r3 | r1, r2, r3 | r1, r2 |
| `…T10:00:00`, main rows | r1, r2, r3 | r1, r2, r3 | r1, r2 |
| `…T10:00`, rows at 09:59:59.999 / 10:00:00.000 / 10:00:00.500 /
10:00:30 / 10:01 | s1, s2 | s1, s2, s3, s4 | s1 |
| `…T10:00:00`, same rows | s1, s2 | s1, s2, s3 | s1 |
| `…T09:59`, same rows | none | s1 | none |
Across the 10 full-timestamp `datetime` cells, 4 differed from the
engine before this change and 5 differ after. Every difference, before
and after, comes from comparing an instant written in a different
spelling as text. The seat ruled that kind of defect out of this card
("a separate coercion defect, same class as #21505 … ⛔ do not build a
coercion fix in this card"). See Acceptance notes.
**The preview window over `Date` rows** (the BSON storage form): it is
now compared as instants, like the `where`. Run against the pre-F11
evaluator, the new matrix's `dateRange` assertions on the BSON row
population fail 10 times, and the evaluator answers them by the `Date`'s
display text. They are green after.
**SQL text that moved, rows unchanged:**
- Every NULL guard prints once where it printed two or three nested
copies.
- A native window on a declared `date` prints the inclusive bound.
- The `/analytics/sql` echo prints a window on a declared `date`
inclusive, and a window on an undeclared column as the bound handed to
the engine.
- The echo prints a resolved preset that stops before its end with that
end bound. Before, it printed the inclusive operator with no value bound
(measured at base on `this_year`).
## The temporal conformance matrix, on every face
`TEMPORAL_CASES`, with its token and `dateRange` axes, answers unchanged
on every face:
- through the native strategy, over a context with no hook and with the
declared-type hook;
- through the native and the ObjectQL faces of the plugin's composition;
- through the read scope;
- through the preview evaluator, read with the declared types and with
none.
The diff against the base sources is empty: the face files at
`0b8239111` (F9/F10) and the preview at `d2f452b88` (F11) were checked
out in this tree and restored afterwards, with the blob equal to HEAD.
The one exception is the preview's `dateRange` on BSON `Date` rows,
which is new coverage. It was red on the pre-F11 evaluator (above).
## Pins
`analytics-faces-one-lowering.test.ts` (new):
- **Enumeration:** no analytics source file uses a whole-day helper
(`nextUtcCalendarDay`, `isUnboundedAbove`, `UNBOUNDED_ABOVE`,
`lteBound`), and none uses a NULL-polarity copy's name; both holder maps
are empty. A file "uses" a name when it imports, declares or calls it. A
positive control proves the scan reaches the seams that call
`lowerFilterCondition`.
- **One source:** the native compiler and the echo emit, for each
declared type, exactly the bound the lowering hands them. Every null
predicate in F10's tree and F9's SQL is one the lowering wrote.
- **Parity:** the 18-cell table, checked against `engine.find`:
- on the native and ObjectQL faces, on SQLite and PostgreSQL
(`OS_TEST_POSTGRES_URL`; a named skip without it);
- on the read scope, on SQLite;
- on the draft preview, through `queryDataset` with `previewDrafts` and
the live path wired to throw. This covers `note lte 2026-07-28`,
`between`, `$not` and the window cells, plus the undeclared-host row.
- **No reader:** the type-blind answers of the native face and the
preview, and the ObjectQL face handing the engine the bound as written.
- **Matrix:** as above.
`preview-temporal-conformance.test.ts` now runs the matrix through
`evaluateAnalyticsQueryOverRows` under both readers, on both row
populations. `preview-evaluator.test.ts` pins that `matchesWhere`
compares the bound it is handed, and that the evaluator gives the whole
day on a declared `datetime`, compares as written on a declared `date`
or text column, and reads type-blind with no reader.
Against the base sources, the new file is red on 21 tests for F9/F10
(enumeration 2, one-source 5, the text cells 7 on each database).
Against the pre-F11 preview sources it is red on 7 more: the whole-day
enumeration, the preview text cells `$lte`, `$between` and window one
day, `$not`, window to the last day, and the undeclared-host row. Both
runs are green on the matrix, the read scope and the reader-less native
rows.
Fifteen existing pin files moved from the stacked guards to the single
guard: these are the step-3 rows marked "until the copy's deletion
card". No id-set assertion changed.
## Ablations
Each direction was predicted before the run. Each mutation went through
`scripts/ablation-replace.mjs`, and every restore was proven by the blob
equalling HEAD and `git diff HEAD` being empty. The pins import `src/`
by relative path.
| Ablation | Predicted | Observed |
|:--|:--|:--|
| A1 restore the native `lte` copy (import plus arm) | enumeration
whole-day pin red; one-source native rows red; the seven text cells red
on both databases; matrix green | 18 red, exactly those |
| A2 unwire the native reader | native datetime cells red; native matrix
red under both readers; reader-less datetime cells red; text and date
cells green | 62 red, exactly those |
| A3 unwire the read scope's reader | F9 parity and F9 matrix datetime
rows red | 16 red: those plus the step-3 read-scope seam pins |
| A4 unwire the echo's reader | one-source echo rows and the echo window
pins red | 7 red: those plus the step-3 echo pin |
| A5 restore the read scope's `IS NULL OR` wrap | enumeration
NULL-polarity pin and the F9 one-source count red | 2 red, exactly those
|
| A6 restore `lteBound` (import, function, `$lte`, `$between` max) |
whole-day enumeration red; preview text `$lte` a day, `$between` one
day, `$not`, window one day red; the last-day text cells green (instant
sniff); the matcher pin red | 6 red, exactly those |
| A7a drop the declared type at the `analytics-service.ts` call site |
the seven preview text cells red; everything else green | 7 red, exactly
those |
| A7b the preview reader answers "not datetime" | preview datetime cells
(`$lte` a day, `$between` one day, `$not`, window one day) red, last-day
ones green; the undeclared-host row red; both matcher pins red; the
matrix's datetime bare-day cases red under both readers | 69 red: those
(5 + 2 + 60) plus two more datetime bare-day preview pins in files not
named, `date-range-array-arm-arity` (#17124 control) and the step-3
`where-door` F11 pin |
## Gates
The pushed head `e08db24240` has tree `a8afd8be53`. The runs below were
made on that same tree as local commit `cf643cce9e`. Before the first
push, the three unpublished commits' co-author trailer was reworded to
the model-free pair. Every rewritten commit's tree equals its
original's, so no measurement below moved.
- **Derived gates** (at `cf643cce9e`): `dispatch-gates --commands`
derives 64 families. All 64 were run with exit codes, all 0, and `--ran`
reconciles 64 derived / 64 run / 0 NOT MEASURED.
- **Full `@objectstack/service-analytics` `test`** (at `c466dc77c8`,
rewritten as `bbabcc6978`; the later commit only edits the changeset):
174 files, 4142 passed, 247 skipped.
- **`typecheck`:** green. It covers the test files (171 listed).
- **Downstream, on the rebuilt `dist`, all green:**
- rest `analytics-*`: 22 files, 291 tests;
- runtime `analytics-*`: 6 files, 41 tests;
- driver-memory date-range conformance;
- client;
- dogfood `analytics-*`: 8 files, 70 tests.
- **ESLint:** the run was narrowed to the 27 changed code files. The
config's population is `**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}` minus its
ignores, with no `parserOptions.project`, so type-aware linting is off
and this diff cannot move any untouched file's result. ESLint's JSON
reports 27 files, 0 errors and 0 warnings.
Size: 28 files, +1418 / -1107.
## Acceptance notes
- `assertDefinedComparands` in F9 and F10 is a refusal door, not the
lowering's meaning, and is kept.
- The package README's `sourceFieldMeta` paragraph names `$empty` as the
reason to wire it. It is now also the reader that tells a `datetime`
column from a text one, on the native face and the preview. No sentence
is false, so it is not edited here.
- F9 binds a bare-day comparand as written, with no storage-form
coercion; this is filed as #21505.
- The preview compares a temporal comparand as written, which is the
same class of defect, and it is left as it is:
- `{ signed_at: { $ne: '2026-07-28' } }` answers r1 to r5 on the preview
and r1, r3, r4, r5 on the engine, before and after (the reader does not
move it);
- the full-timestamp window ends above also belong to this class.
- The seat answered the report's question on the `'~'` deletion with A:
keep it. The window is the `{ $gte, $lte }` pair (item 8), and the
remaining off-engine cells belong to the draft-preview part of #21505.
- `docs/design/predicate-compilation-convergence.md` §4.1's face table
still lists F9, F10 and F11 under "Deleted later". The F7 row was marked
retired by its own deletion card (#21242). That file is outside this
claim's surface, so it is named here for the seat rather than edited.
- The PostgreSQL parity cells were not re-run at this head. The F11
commits touch only the preview evaluator and its call site, which those
cells do not execute, and the `main` merged since touches no analytics,
ObjectQL or SQL-driver source (three test files in `driver-sql` and
`objectql`, nothing else there).
---
_Generated by [Claude
Code](https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent aa0d4b9 commit 81e69ca
28 files changed
Lines changed: 1418 additions & 1107 deletions
File tree
- .changeset
- packages/services/service-analytics/src
- __tests__
- strategies
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
0 commit comments