fix(service-analytics)!: refuse an analytics order key that names no member the query selects - #21314
Conversation
…member the query selects An `order` key must be a column the answer carries: a `dimensions` entry, a `measures` entry, or a `timeDimensions` entry with a `granularity`, spelled exactly as selected. Anything else is refused INVALID_FIELD / 400 at the analytics door (query and the generateSql dry run), after ensureCube and before strategy selection, so the native-SQL and the ObjectQL face answer alike. The ObjectQL strategy's projection rule moves into the door module so both read one definition. Claude-Session: https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ Co-authored-by: Claude <noreply@anthropic.com>
…admission verdicts callCtx is the one seam query() and generateSql() share, so the door has one call site. Running it after the object, stored-metadata-body, field read and field query gates keeps the 403 a field the caller may not read gets in every position, the order key included. Claude-Session: https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ Co-authored-by: Claude <noreply@anthropic.com>
…dd its changeset Claude-Session: https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ Co-authored-by: Claude <noreply@anthropic.com>
…alytics-order-key-selected
📓 Docs Drift CheckThis PR changes 1 package(s): 15 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 4 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 10 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 64cf7e8457094712e72a3e3b3d485db06bf5409a && git checkout 64cf7e8457094712e72a3e3b3d485db06bf5409a
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 1371dc980cdf0d3128bee4a5441f2c6bec18f008 34d48294a17830298d61b016c0282ae183e98ce4 && git checkout -B drift-repro 1371dc980cdf0d3128bee4a5441f2c6bec18f008 && git merge --no-ff 34d48294a17830298d61b016c0282ae183e98ce4
node scripts/docs-audit/affected-docs.mjs --json 1371dc980cdf0d3128bee4a5441f2c6bec18f008
|
Fixes #21267
Clause-②: no (narrowing)
What changed
The analytics door now refuses an
orderkey that names no member the query selects, with400 INVALID_FIELD, on the native-SQL and the ObjectQL face alike, before either strategy runs. This is triage's ruling (a) on the card (comment5943018488).The rule. An
orderkey must be a column the answer carries: one of the query's owndimensionsentries, one of itsmeasuresentries, or atimeDimensionsentry that carries agranularity. It must be spelled exactly as it is selected. AtimeDimensionsentry with only adateRangebounds the rows and is not a column, so it is not orderable. The spelling is exact because the answer keys each column by the spelling the caller sent (a cube-qualified measure keeps its qualifier), and both strategies write the key intoORDER BYas that column's name. The member-bearing keys were read offAnalyticsQuerySchemainpackages/spec/src/data/analytics.zod.ts:measures,dimensionsandtimeDimensions[].dimension.wherenames filter members, not columns. Spec is not edited.order-key-door.ts(new, internal to the package, not exported fromindex.ts).assertOrderKeysSelected(query)throws the door's existing member-gate envelope:code: 'INVALID_FIELD',status: 400,param: 'order',field(the first offending key). The message names every offending key and every member the query does select, and says the query was not run. ⛔ No new error code.AnalyticsService.callCtx. That is the one seamquery()(/analytics/query, and every queryDatasetExecutorruns) andgenerateSql()(/analytics/sql) share, so both doors and both faces answer alike. It runs afterensureCube(an unknown cube still answers 404 first) and after the admission verdicts (object, stored metadata body, field read, field query). It runs before the read scopes are resolved and before any strategy is selected. ⛔ No per-strategy copy.ObjectQLStrategy.projectedDimensions(which groups, maps rows and describesfields[]) now delegates to the door module'sprojectedDimensions. The door and the strategy read the same rule for which time dimensions are columns.Measured:
POST /api/v1/analytics/queryand/sqlon the real dispatcher routeSetup:
AnalyticsServicePluginover a realObjectQLengine andSqlDriver, a signed-in caller, the realdispatcher-pluginmount, SQLite in memory and a private PostgreSQL 16.14. The cube overdealdeclares no join.owneris a lookup whosereferenceis a person object that also declaresnoteandamount. The ObjectQL face is the same plugin narrowed to the engine-aggregate path. Before:origin/mainat3196ef1a1. After: this branch'sservice-analyticsbuild at4362776a3. The scratch probe was deleted.dimensions: ['owner.email'],order: { note }DATABASE_ERROR→ 400INVALID_FIELDnoteambiguous) → 400dimensions: ['note'],order: { amount }dimensions: ['note'],order: { 'owner.email' }order: { note: 'desc' }(selected dimension)order: { amount_sum: 'desc' }(selected measure)/sql(the dry run) answered 200 on every refused row before, with a statement whoseORDER BYnames a column the statement does not select. It now answers the same 400. The control cells are byte-identical before and after: 16 cells (2 drivers × 2 faces × 2 routes × 2 controls), comparing status, rows and the echoed statement.What the ObjectQL face answered today (triage's first measurement). It answered
200for all three rows on both drivers. That is because its execution never appliesorderat all, not because it orders by anything. The selected-dimension control asked fordescand came backx, y, zon SQLite andz, x, yon PostgreSQL. Its echoed statement shows the sameORDER BYthe native face could not run. See Acceptance notes.Census (before the code change)
examples/**andpackages/apps/**: zero hits. No shipped dashboard widget carriessortBy/sortOrder. The onlysortBytext is a comment inexamples/app-todo/src/dashboards/task.dashboard.ts:21. No report carriesorder. No dataset or cube carries one, and no page builds an analytics query with one.packages/apps/**issues no analytics query at all. Theorder:hits that do exist (active-projects.page.ts:28,review-queue.page.ts:34, the twotask.view.tsfiles,embed-objectql/src/index.ts:58) are record-querysortentries on the data API, not analytics.examples/app-showcase/src/ui/pages/command-center.page.ts(held by finding(examples): the app-showcase command-center KPI tiles writeobject-metricfilteras a record, which the spec's ownComponentPropsMap['object-metric'].filterrefuses ("takes the ViewFilterRule ARRAY form") #21251): no hit, so nothing to report. Its KPIs areobject-metricscalars (aggregatewith noorder, lines 147–152). Its charts are dataset-bound with nosortBy(lines 158–167). Its work queue is anobject-grid(line 172). Not edited..objectui-sha31971ff1e28f: no hit. Its one/analytics/querysender isObjectStackAdapter.aggregate()inpackages/data-objectstack/src/index.ts. Its payload (lines 6278–6306) carriescube,measures,dimensionsandwhere, neverorder.git grepforanalytics.query(and for acube:payload finds no other sender. DashboardsortByreaches the dataset door (selection.order), not this route.DatasetExecutorcalls the analytics query door with anorder. The executor pushes anorderdown only when the selection is one query and every key is a dimension or unfiltered measure that query selects (canPushDownWindow,dataset-executor.ts:1136–1143). The dataset door refuses an unselectedselection.orderkey itself first (resolveOrdering,400 DATASET_INVALID). So this door never refuses a dataset selection the dataset door accepted.A census pin was not added. The population of shipped analytics queries with an
orderis zero, and a pin over an empty population asserts nothing. A pin that walks future dashboards would be a new gate, which the dispatch's axes default to "no". The dataset door already refuses an unselected widgetsortByat runtime.Pins
New file:
packages/services/service-analytics/src/__tests__/order-key-selected.test.ts. It uses the plugin's own composition over a real engine, a SQLite cell and a PostgreSQL cell (a named skip withoutOS_TEST_POSTGRES_URL), and both faces. Each refusal pin runs both doors (queryandgenerateSql) on both faces. It assertscode,status,paramandfield, that the message names the key and every selected member, and that nothing ran (zero raw statements and zero engine aggregates). There are 16 tests, 8 per cell.1–3. The card's three rows are refused 400
INVALID_FIELDon both faces and both doors.4. A
timeDimensionsentry that only sets adateRangeis not a column, so ordering by it is refused.5. Exact spelling: a cube-qualified spelling of a measure selected bare is refused.
6. CONTROL: ordering by a selected dimension is served. The native face returns the requested order (
z, y, x) and the ObjectQL face the same groups./sqlshowsORDER BY "note" DESC.7. CONTROL: ordering by a selected measure is served. The native face returns
x 15, y 7, z 1.8. CONTROL: a bucketed time dimension is a column, so ordering by it is served (three month buckets).
Existing pins that now also hold this door's slot. A key naming a field the caller may not read still gets the field gate's 403, because the door runs after admission. That is pinned, unchanged and green, by
field-read-admission-gate.test.ts("an order key", "an expression member as an order key"),field-query-admission-gate.test.ts("a masked field as an order key") and the route-levelpackages/rest/src/analytics-field-permission-gate.test.ts:300. A first draft placed the door ahead of admission. Those six service-level cases went red (a 400 replacing the 403), and the rest route pin would have too. That placement was dropped rather than the fixtures rewritten.Ablations
The pin file imports the subject by relative path (
../plugin.js), so each run readssrcand there is nodistleg. Each mutation went throughscripts/ablation-replace.mjsin WRAP mode, with an outertraprestore onEXIT INT TERMagainst the absolute path. Both ran from committed4362776a3with the PostgreSQL cell live. Predictions were written before each run.order-key-door.ts)if (unselected.length === 0) return;becomes a test that is always true (greater than or equal to 0)unselected = keysEach mutation landed: anchor 1 → 0, and the blob changed from
21103a6b4884to A1488d0e6274bband A25c62c68e0315. Each was restored and proven: the blob equals the HEAD blob21103a6b4884, andgit diff HEADis empty.Docs
git grepovercontent/docs/**(excludingreleases/) andskills/**for analyticsorder,sortByandorderByfound no sentence or example this change makes false. Dashboards (ui/dashboards.mdx:161,skills/objectstack-ui/rules/dashboards.md:349) and reports (data-modeling/analytics.mdx:140) already say the sort key must be selected. The README example orders by a selected measure. Two sentences document the rule where/analytics/queryhad none:content/docs/api/data-api.mdx,POST /analytics/query: a new callout after thewherecallout. It states the rule and the 400, says/sqlrefuses the same keys, and says "To order by a member, select it."packages/services/service-analytics/README.md:100: theorderrow's empty description becomes the rule.Changeset
.changeset/21267-analytics-order-key-selected.md:@objectstack/service-analyticsminor, withClause-②: no (narrowing), a BREAKING note, and the ADR-0087 dispositionnot-required (no-migration-prescription)with its reasons. See Deviations forminor.Verification at
34d48294a34d48294ais this branch's head. It carriesorigin/mainat4e530568amerged in (no overlap with this diff's files). Install and the full build (turbo run build, 72/72) were refreshed after the merge.pnpm --filter @objectstack/service-analytics test, with the PostgreSQL cells live: 166 files, 3843 passed, 0 skipped, 0 failed.typecheck(tsc --noEmit): exit 0, and its program lists all 4 touched.tsfiles (--listFiles).OS_TEST_POSTGRES_URLset. These are the test files that drive the analytics service or its routes inpackages/rest(21 files),packages/runtime(19),packages/drivers/driver-memory(26),packages/drivers/driver-sql(2) andpackages/plugins/plugin-security(1). Results: 292/292, 551/551, 848/848, 60 passed / 1 skipped (a pre-existing skip insql-driver-13714-aggregate-alias-single-identifier.test.ts), and 262/262. No failures. NOT MEASURED: thepackages/qa/dogfoodanalytics files. They boot the example apps, which carry no analyticsorder(census above), and are left to the requiredDogfood Regression Gate.node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commandsderives 91 commands at34d48294a, and all 91 exit 0.--ranreconciliation: "91 derived, 91 run, 0 NOT-MEASURED, 0 UNRUN", with every exit code recorded. No run left a managed block inAGENTS.md.eslint --no-inline-config --format jsonover the 4 touched.tsfiles at34d48294areports 4 files, 0 errors and 0 warnings. ① All 4 are inside the populationeslint.config.mjslints (**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}, minusNEVER_LINTED). The.md/.mdxfiles match nofilesglob ("File ignored because no matching configuration was supplied"). ② The count of 4 is read from the JSON output. ③ The config enables no type-aware linting (--print-configgivesparserOptions{"ecmaVersion":"latest","sourceType":"module"}, with noproject), so this diff cannot move a verdict on an untouched file. The repo-widepnpm lintis left to CI.Acceptance notes
order,limitoroffseton/analytics/query, while its echoed statement says it does. This is a separate defect, reported to the seat and not handled here.ObjectQLStrategy.execute()handsengine.aggregateno ordering or window (objectql-strategy.ts:304–321), but itsgenerateSqlrendersORDER BY/LIMIT/OFFSET(:634–639). On the default composition every bucketed time-dimension query lands on this face, because the native face declines granularity. Measured at34d48294a, SQLite and PostgreSQL 16.14:timeDimensions: [{ dimension: 'closed_on', granularity: 'month' }],order: { closed_on: 'desc' },limit: 1answers all three month rows, ascending on SQLite and03, 05, 04on PostgreSQL. The dataset door is unaffected:DatasetExecutorre-applies ordering and the window over the assembled grid.orderposition (namedQueryFields) stays load-bearing. It still decides the 403 for an unselected key over a field the caller may not read, because the door runs after it./sqldry run's answer changes only for a refused query: 200 with an unrunnable statement becomes 400.Deviations
minor, notpatch. The dispatch namedpatch.check-changeset-no-major.mjsgrades aClause-②: no (narrowing)declaration whose moved package is gradedpatchasenforce(exit 1). Its reading says a narrowing "is a BREAKING change; during the launch window it shipsminor". The landed precedent for an analytics-door narrowing, the JSON-door changeset for analytics: a JSON-stored dimension or count_distinct over a relationship path whose lookup has no declared cube join answers 500 on PostgreSQL; the structured-JSON door resolves hops through cube.joins only #21232, isminor.callCtx, the one seam both doors share, after admission and before strategy selection, for the reason under Pins.packages/services/service-analytics/src/and its tests, the package README, one docs page and the changeset. No census fix was needed.Generated by Claude Code