Merge origin/main 810d42b into claude/issue-19278-shard-hash-carries-closure

Claude-Session: https://claude.ai/code/session_01TdiauJaVCHuj45EzZGUxHh
Co-authored-by: Claude <noreply@anthropic.com>

Author: claude

.changeset/20513-driver-sql-runtime-strings-state-the-decision.md

@@ -0,0 +1,26 @@
+---
+'@objectstack/driver-sql': patch
+---
+
+driver-sql refusals, drift reports and log lines no longer cite tracker numbers; each states the reason in words
+
+Clause-②: no
+
+Many messages the SQL driver shows to authors and operators ended with an issue-tracker number where
+the reason belonged. The number goes, and where the sentence did not already say what was decided, it
+now does:
+
+- Filter refusals (`INVALID_FILTER`): the withheld-detail wording ("withheld from the message; the full
+  diagnostic is in the server log"), the JSON-column, zero-operator, `$null` / `$exists`, undefined
+  comparand and unknown-combinator refusals, and the filter-array refusal.
+- Schema and index messages: the `reference_to` DDL refusal (the FOREIGN KEY DDL that key used to gate
+  is retired, because it could never fire for a spec-conformant lookup), the MySQL TEXT-key and
+  row-size explanations, the hash-shadow UNIQUE messages, and the `os migrate plan` drift entries.
+- The NULL-safe UNIQUE messages now say why rows without an organization were never constrained: SQL
+  UNIQUE is NULL-distinct.
+- Boot log lines for the SQLite datetime, time and json canonicalisation and the MySQL `TIMESTAMP` /
+  `TIME` widening now say what the conversion is for.
+
+Text only: no error code, field name, status or behaviour changes. Three aggregate refusals keep their
+citation for now, because a test in `@objectstack/driver-turso` compares them byte for byte with the
+Turso remote transport's copies; they change together with those copies.

.changeset/20578-sdui-parser-base-props-one-list.md

@@ -0,0 +1,10 @@
+---
+'@objectstack/sdui-parser': minor
+---
+
+`@objectstack/sdui-parser` now reads one base-prop list, ported from objectui's `SDUI_BASE_PROPS` at the console pin `db11afd4967c` (objectui#11008, #11044). Both `validateTree` and the generated JSX types (`generateDts`'s `SduiBaseProps`) are driven by it.
+
+- On every node, whatever the component declares: `bind`, `hidden`, `visibleWhen`, `hiddenOn`, `testId` are newly accepted. They no longer draw `unknown-prop`, and the generated types accept them as attributes.
+- Only on a type whose registration declares no input of that name: `name`, `label`, `description`, `placeholder`, `data`, `ariaLabel` are newly accepted. A type that declares one keeps its declared type check and its declared attribute type; its generated interface `Omit`s that key from `SduiBaseProps`.
+
+Effect for consumers: `os validate` stops warning `unknown-prop` on those keys, and a `.tsx` page that authors them now type-checks against `generateDts` output where it was a TypeScript error before. Measured on the tracked `sdui.manifest.json` (107 components), no component declares any of the five every-node keys, and every declared where-undeclared key is checked as before, so no diagnostic of error severity is removed for that manifest. The wider type surface is why this is a minor, not a patch.

.changeset/20696-migrate-meta-strict-factories.md

@@ -0,0 +1,54 @@
+---
+'@objectstack/cli': patch
+---
+
+fix(cli): `os migrate meta` converts an object built with `ObjectSchema.create(…)` instead of stopping at load when the object carries a retired key
+
+Clause-②: no
+
+`os migrate meta` reads a config the current schema refuses, so it can rewrite the
+retired keys in it. It did that for artifacts built with a `define*` helper and for
+plain object literals. It did not do it for artifacts built with a factory such as
+`ObjectSchema.create(…)`, which validates when it is called. An object like this:
+
+```ts
+ObjectSchema.create({
+  name: 'ticket',
+  fields: { title: { type: 'text' } },
+  tenancy: { enabled: true, organizationField: 'organization_id' },
+})
+```
+
+stopped `os migrate meta --from 17` at load with exit 1 and a raw JSON array of
+validation issues. The message in that array told the author to run
+`os migrate meta --from 17`.
+
+The command now loads it, applies the conversion (here
+`object-tenancy-organization-field-removed`), and reports `schemaValid` for the
+migrated stack, exactly as it does for the same object written as a plain literal.
+This covers the five factories in `@objectstack/spec` that validate when called:
+`ObjectSchema.create` (`@objectstack/spec/data`) and `App.create`,
+`Dashboard.create`, `Report.create` and `Action.create` (`@objectstack/spec/ui`).
+The other `create` factories spec exports return their argument unchanged and
+never refused anything, so nothing changes for them.
+
+A schema problem the migration cannot fix is still reported: it is listed among
+the refusals under the verdict, and `schemaValid` is `false`. A check that only
+the factory makes when it is called, such as `ObjectSchema.create` refusing a
+`managedBy: 'system-data'` object that grants no create, edit or delete, is not
+part of the stack schema. It is reported on the stderr line described below and
+does not change `schemaValid`, the same as `defineStack`'s own call-time checks.
+`os validate` still refuses it.
+
+While the config loads, `os migrate meta` prints one stderr line for each
+artifact the current schema refused. A raw validation error on that line is now
+printed as a block, for example `ObjectSchema.create validation failed (1 issue):`
+followed by one `✗ path: message` line per issue, instead of a raw JSON array.
+This also applies to `define*` helpers that throw a raw validation error, such
+as `defineAgent`.
+
+Nothing else changes. `os validate`, `os build` and every other command still
+refuse the retired key at load, with the same message. `ObjectSchema.create` and
+the other factories stay strict everywhere outside `os migrate meta`. The keys
+of the `--json` payload are unchanged, and a run whose migrated stack does not
+parse still exits 0.

.changeset/20768-first-boot-migration-gate-read.md

@@ -0,0 +1,22 @@
+---
+'@objectstack/driver-sql': patch
+---
+
+fix(driver-sql): the first boot of a new database no longer prints a `DATABASE_ERROR` for `sys_migration` (#20768)
+
+On the first boot of a new database, the SQL driver printed this line once, on its warn channel (stderr by default):
+
+```text
+[sql-driver] DATABASE_ERROR — the backend refused a read on 'sys_migration' (SQLITE_ERROR) ... no such table: sys_migration
+```
+
+Nothing was wrong. At the start of its first schema sync, before it creates any table, the driver asks whether this deployment's file columns have moved (the ADR-0104 media-arm resolver). The resolver the engine supplies answers by reading `sys_migration`. On a new database that table does not exist yet, so the read is refused and the answer is "not moved", which is correct for an empty store.
+
+The driver now asks that question inside an async scope. Inside it, a read refused because its own target table does not exist goes to the logger's `debug` channel instead of `warn`. The default logger has no `debug`, so the line is not printed. The logger shape gains an optional `debug`. The refusal is still thrown to the resolver, and the resolver's answer is the same as before.
+
+What still warns:
+
+- every other refusal inside that scope, such as a malformed statement on a table that exists, or a missing table named by another relation (a view over a dropped table);
+- a missing table read anywhere else, as before.
+
+The missing-table check is the shared `isMissingTableError` from `@objectstack/types`, which `@objectstack/metadata/errors` re-exports. There is nothing to migrate.

.changeset/20783-groupby-structured-json-refused.md

@@ -0,0 +1,19 @@
+---
+"@objectstack/objectql": minor
+---
+
+fix(objectql)!: a `groupBy` on a structured-JSON field is refused with `INVALID_FIELD` / 400 at the engine's `aggregate`, on every driver
+
+Clause-②: no (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) a refusal of a grouping TARGET at the engine's aggregate door: a groupBy entry naming a declared json, composite, repeater, record, location, address or vector field. No authorable key, spelling, export or stored shape moves (the door module is internal; `@objectstack/objectql` exports nothing new and nothing less, and `EngineAggregateOptions` / `QuerySchema.groupBy` keep parsing the entry), and no stored row is read or rewritten. The grouping had no shared meaning to preserve (one merged group on the in-memory driver, one group per serialized document on SQLite, a 500 on PostgreSQL), and which scalar part of the document a caller meant to group on is not something a ledger entry can rewrite. The other categories are closed on facts: the package publishes (not `unpublished`); no ADR-0087 id covers a grouping target (not `registered` / `already-registered`); and the change is runtime behaviour, not a declaration (not `runtime-interface-only` / `type-surface-only`). -->
+
+**BREAKING**: this narrows what `aggregate` accepts as a grouping target. A `groupBy` entry that names a declared field of the structured-JSON class (`json`, `composite`, `repeater`, `record`, `location`, `address`, `vector`) is refused by the engine before any driver is asked. Both entry spellings are judged, the field name and the `{ field }` object, a `dateGranularity` bucket included. It ships as `minor` under the launch-window convention for accept-set narrowings. No export or published type changes.
+
+**What an author sees now.** `400 INVALID_FIELD`, naming the position (`groupBy[0]`, or `groupBy[0].field` for the object form), the field and its declared type, saying the query was not run, and naming the route inside the first 500 characters the REST door keeps: group by a field that stores one scalar value, storing the part of the document you group on in a field of its own. The thrown error carries `field`, `fields`, `object` and `param: 'groupBy'`.
+
+**Why a refusal.** The drivers share no meaning for a JSON document as a group key. Measured through `POST /api/v1/data/:object/query` over three rows with different documents under the grouped field: the in-memory driver answered 200 with one group holding every row, SQLite answered 200 with one group per serialized document, and PostgreSQL answered 500 `DATABASE_ERROR`. A `vector` field split the same three ways, and a date bucket over a `json` field answered one `null` bucket on memory and SQLite and 500 on PostgreSQL. No producer that groups by a structured-JSON field was found (no dataset, cube, view grouping or `groupBy` in the example apps names one), so no meaning is defined for it here.
+
+**Who is affected.** A caller of `engine.aggregate` or of the REST query door that grouped by such a field on the in-memory driver or on SQLite and read the merged or per-serialization groups as real ones. On PostgreSQL the same query was already a 500. The analytics service's aggregate path (a cube query the native-SQL strategy declines, such as a time dimension with a granularity, or any cube query on the in-memory driver) reaches the engine and answers this refusal too.
+
+**Unchanged.** A `groupBy` on any other type (`text`, `number`, a `multiple: true` select, a file field), a structured-JSON field as an AGGREGATED column (`count`, `count_distinct`, `min`, `max`), and an undeclared name, which the REST door answers `INVALID_FIELD` as unknown before the engine is reached.

.changeset/5930-shared-filter-lowering.md

@@ -0,0 +1,26 @@
+---
+'@objectstack/spec': minor
+'@objectstack/objectql': patch
+'@objectstack/plugin-security': minor
+---
+
+feat(spec, objectql, plugin-security): one shared filter lowering, run once at the engine and RLS seams (ADR-0053 D-D1, amended)
+
+Clause-②: yes
+
+`@objectstack/spec/data` exports `lowerFilterCondition(filter, options?)` and its `FilterLoweringOptions` type. It is not exported from the package root entry. It is a pure `FilterCondition → FilterCondition` rewrite that applies three rules once:
+
+- `$between` becomes `$gte` its minimum and `$lte` its maximum.
+- A `$lte` whose comparand is a bare `YYYY-MM-DD` day becomes `$lt` the next day, in the calendar-string domain. On the last supported day (`9999-12-31`) a lone `$lte` becomes `{ $null: false }`, and a `$between` keeps only its minimum.
+- The NULL-polarity guards the drivers already compile. A `$ne` of a value, a `$nin` or a `$notContains` holds for a row with no value. Every leaf of a `$not` operand is made total.
+
+The rewrite is copy-on-write, idempotent and never refuses. A node it rewrites keeps its filter-subtree provenance mark. With `options.isDatetimeColumn` (a typed seam), the first two rules change only a declared `datetime` column. Without it they apply to every column.
+
+As ADR-0053 D-D1 (amended 2026-09-30) requires, the seams now run it once, after the comparand doors and after filter-token resolution:
+
+- **`@objectstack/objectql`** runs it on every filter position, typed by the object's declared fields. That covers `where` on `find`, `findOne`, `count`, `update` and `delete`, and `aggregate`'s `where`, `aggregations[i].filter` and `having`. `having` is typed by the aggregated row's columns, so `max` of a `datetime` field counts as a `datetime`. Drivers receive the lowered filter. A date macro such as `{today}` is resolved before the lowering reads it.
+- **`@objectstack/plugin-security`** runs it on every compiled RLS policy filter (`using` and `check`), right after the two comparand faces. `SecurityPlugin` now hands the compile seam the object's declared `datetime` columns (`RlsFieldGuard.datetime`). A guard without that set treats no column as `datetime`.
+
+Row answers stay the same on every driver. Each driver keeps its own copy of these rules, and every copy gives the same answer on lowered input. One result changes. The engine evaluates `aggregate`'s `aggregations[i].filter` and `having` itself, and that evaluator now treats a row or group with no value the way every driver's `where` already does. It no longer counts such a row in a `$between` on a `datetime` column. It now keeps such a row under a `$not` over an ordering such as `$lt`.
+
+Nothing is removed or renamed, and there is nothing to migrate.