fix(service-analytics): native SQL judges a comparand against a declared number column by the spec's verdict, as the comparand walk's second arm - #21446
Conversation
…erdict as the walk's second arm The native strategy compiles its own SQL past the engine's field-aware walk, so a comparand against a declared number column reached the driver as written: "abc" counted no row on SQLite and was a 500 on PostgreSQL, true bound 1, and a bare-day $lte met the window rule and counted every row, where the engine door answers INVALID_FILTER / 400. The boolean walk becomes one walk with two arms (number first, the classes are disjoint), each reading its own spec verdict and operator lists, at the same three positions: where (runtimeFilter merged), each measure filter, the dataset scope. A numeric string narrows to its number, copy-on-write. Claude-Session: https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ Co-authored-by: Claude <noreply@anthropic.com>
…he engine door's answers Each cell is asked at the cube read and the dataset door on both faces (native statement, engine aggregate), on SQLite and on live PostgreSQL where OS_TEST_POSTGRES_URL is set: the card's four cells refuse INVALID_FILTER / 400 before any statement runs, a number is the control, and a numeric string binds the number the engine handed its driver. A registered dataset's own scope and measure filters are judged too, frozen so narrowing must be copy-on-write. Claude-Session: https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ Co-authored-by: Claude <noreply@anthropic.com>
…rand arm Claude-Session: https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift Check16 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run. What this run could not see
Coarse fallback — 10 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 711b79109677defb3dd2ccdd6ca4967e6d3b2051 && git checkout 711b79109677defb3dd2ccdd6ca4967e6d3b2051
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin b79301000c85fd5986c0656bde27bb7a70eadf60 7a626eb094e77281f8e7bf4c4869f042ea9de843 && git checkout -B drift-repro b79301000c85fd5986c0656bde27bb7a70eadf60 && git merge --no-ff 7a626eb094e77281f8e7bf4c4869f042ea9de843
node scripts/docs-audit/affected-docs.mjs --json b79301000c85fd5986c0656bde27bb7a70eadf60 |
…t-set narrowing The native face now refuses INVALID_FILTER / 400 where it answered 200 (a 500 on PostgreSQL), the same class of change the boolean arm declared: minor, Clause-② no (narrowing), a BREAKING banner and an ADR-0087 not-required disposition stating this change's facts. Claude-Session: https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ Co-authored-by: Claude <noreply@anthropic.com>
Fixes #21426
Clause-②: no (narrowing)
What this changes
NativeSQLStrategycompiles its own SQL past the engine's field-aware filter walk, so it skipped the spec's number-comparand verdict (numberComparandDoorVerdict,@objectstack/spec/data). It now runs that verdict as the second arm of the one walk that the boolean arm added (#21376, PR #21424). The arm runs at the same three positions:where, with the dataset door'sruntimeFiltermerged into it;filter;Details:
One walk, two arms.
judgedBooleanComparands/narrowBooleanComparandsare renamedjudgedComparands/narrowComparands. A walk named for booleans would lie once it also judges numbers. Each arm is aComparandArmrow with three parts, and nothing in it is copied from the spec (no table, regex or refusal words):numberComparandFieldVerdict/booleanComparandFieldVerdict,judgedalone);NUMBER_COMPARAND_DOOR_*_OPERATORS/BOOLEAN_COMPARAND_DOOR_*_OPERATORS);The two classes are disjoint, so at most one arm judges a member. The number arm is asked first, which is the engine walk's order.
Refusal envelope. The arm throws
invalidFilterError(from the strategy'sfilter-normalizer.ts):INVALID_FILTER/ 400, the same constructor the boolean arm throws through. The message isnumberComparandRefusalMessage's sentence behind the[analytics]prefix. Every native position is bound by the driver (a measure filter compiles into its conditional aggregate's bind), so the sentence uses the spec's default, driver-bound reading.Narrowing. A numeric string narrows to the number the verdict names, so the native statement binds what the engine hands its driver:
12, never"12". It is copy-on-write: a subtree that nothing narrowed is returned by reference. The pins deep-freeze every filter handed in and every registered dataset.Member reader. Unchanged. The declared type comes from the host's
declaredFieldTypehook, for the (object, column) pair thatresolveStorageTargetreturns. Socurrencyandpercentare judged, like everyNUMERIC_VALUE_TYPESmember. A relationship-path member is judged at the related object's declared column. Aformulareaches the walk with noreturnType(the plugin relays none), so both verdicts answerdeferredfor it.{ amount: true }. The boolean arm never touched it: its field verdict isnot-judgedfor a number column. The number arm refuses it as thebooleanform.No
packages/specedit and noobjectql-strategy.tsedit.windowClauseSqlis untouched.Measured
Setup:
amount5, 12 and 30.AnalyticsServicePluginover a realObjectQLengine andSqlDriver.AnalyticsService.query(whatPOST /api/v1/analytics/queryrelays) andAnalyticsService.queryDatasetwith aruntimeFilter(whatPOST /api/v1/analytics/dataset/queryrelays).where3a6d92f78{ amount: "abc" }DATABASE_ERRORINVALID_FILTERINVALID_FILTER{ amount: { $lte: "9999-12-31" } }{ amount: { $ne: "abc" } }{ amount: true }{ amount: 12 }(the control){ amount: "12" }"12""12"1212runtimeFilteranswered identically on every cell.{ amount: "abc" }and a measure filter{ amount: { $ne: "abc" } }answered 200 on SQLite and 500 on PG on the native face, and 400 on the engine face. Both answer 400 now.$lteand$neat count 2 over different rows. The shape is the same: 200 where the engine refuses.Pins:
native-sql-number-comparand-door.test.tsThe file mirrors
native-sql-boolean-comparand-door.test.ts. Each parity cell asks both faces, at the cube read and at the dataset door. It asserts:where.What it covers:
number,currencyandpercent;$or, as a list member, a blank string and"+5";[12, 30]for$in ["12", 30].The PostgreSQL cell runs where
OS_TEST_POSTGRES_URLis set, and is a named skip otherwise, as in the boolean twin. It ran here against the live server: SQLite plus PG is 114 tests.Two cells are pinned on the native face alone, because the engine face does not reach this verdict there:
account_credit, the cube dimension overaccount.credit). The engine face refuses every cross-object filter (INVALID_FIELD/ 400). The native face joins and judges the related object'snumbercolumn:"abc"is refused, and{ $gt: "100" }binds100.{ amount: { $gt: [10] } }. The shared analytics lowering hands the engine only the list's first member, so the engine face answers 200, 2. The spec's verdict refuses a list where one number belongs, and the native face now does too. See the acceptance notes.Ablations (one-shot and restored; nothing left in the tree)
How each leg ran:
native-sql-strategy.tswas mutated throughscripts/ablation-replace.mjs. Its anchor must hit exactly once.git diff HEADis empty. Each leg also ran inside a script with an EXIT/INT/TERM restore trap.../plugin.jsby relative path, so it reads the source, notdist/.The predicted direction was named before each run. The observation matched the prediction on every leg, on SQLite and PG:
comparandArmFornever returnsNUMBER_ARM)narrowsverdict read aspasses)whereposition's call removedThe narrowing leg's first attempt did not run.
ablation-replacerefused it because the replacement text contained the anchor (count 1 → 1), restored the file, and ran no test. The row above is the re-run with a non-overlapping replacement.Gates
Patch round (declaration only), at HEAD
7a626eb09. The only change is the changeset; no code moved.dispatch-gates --commandsderived the same 63 commands as round 1. All 63 ran in a freshly recreated worktree (fullturbo run build, 72 of 72 tasks from cache, first) and exited 0.--ranreconciled 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN.check-adr-0087-registration --base origin/main: reads the changeset as[BREAKING+clause-②-narrowing]and accepts the dispositionnot-required (no-migration-prescription).check-changeset-no-major: nomajorbump. Its level axis needs apull_requestpayload, so it was also driven offline with--eventcarrying this body (see the report on the card).check:changeset-gate-self-tests: exit 0.Round 1, at HEAD
a17f21b80:node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commandsderived 63 commands. All 63 ran and exited 0.--ranreconciled 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN.pnpm check:dual-build-cjs-loads: the first attempt answeredPREREQUISITE NOT MET(exit 3, nodist/for unbuilt packages). It was re-run after a fullturbo run build(72 of 72 tasks, 71 cached) and exited 0.For
@objectstack/service-analytics:typecheck: exit 0, andtsc --listFilesincludes both edited files.test: 170 files and 4065 tests passed, withOS_TEST_POSTGRES_URLpointing at the live server. This ran atd77d545a1; the later commit adds only the changeset.Docs
I grepped
content/docs/**(outsidereleases/) andskills/**for the analytics filter's comparand handling. No sentence describes the native face's number comparands, so none became false.Acceptance notes
@objectstack/service-analyticsminor,Clause-②: no (narrowing), a BREAKING banner and an ADR-0087not-required (no-migration-prescription)disposition. This matches the boolean twin's declaration for the same class of change; the seat's review on the card corrected the claim's line tono (narrowing).where(for examplewhere.amount.$ne), as the boolean arm already did. The engine roots it ataggregations[i].filter. This is wording only, and no one is set to carry it.{ note: { $gt: ["a", "z"] } }binds"a". On the engine face,{ amount: { $gt: [10, 99] } }answers 200, 2 (bound10). Filed by the seat as analytics: a list comparand at a scalar operator ({ amount: { $gt: [10, 99] } }) answers 200 bound to its first member on the engine-aggregate face (and on both faces for a text column), where the spec's verdict refuses the list form #21448.$lterule ignores the column type.{ note: { $lte: "9999-12-31" } }on atextcolumn binds nothing and counts every row (3), where the engine face counts 0. The seat's carrier is #5930 step 4 (domain:services): the analytics faces delete their hand-copied filter meaning — the read scope (F9), thewheretree and its compilers (F10), the draft preview (F11) — each naming its typed column reader #21417, which deletes that copy.Generated by Claude Code