Merge remote-tracking branch 'origin/main' into claude/issue-20821-migrate-plan-deferred-reads

Author: claude

.changeset/20621-retired-key-tsc-names-retirement.md

@@ -0,0 +1,32 @@
+---
+'@objectstack/spec': minor
+---
+
+feat(spec): writing a retired key now fails `tsc` with an error that says the key was retired and where its migration is printed (#20621)
+
+Clause-②: yes
+
+<!-- adr-0087: not-required (no-migration-prescription) this changes only the declared TypeScript type of every `retiredKey()` tombstone, from `undefined` to an uninhabited mark carrying the retirement sentence, on both the input and the parsed side. No authorable key, spelling, export or stored shape moves: the parse, its prescription, the JSON schemas and the authorable surface are unchanged. Every tombstoned key's own retirement is already ledgered under its own ADR-0087 id, and this diff retires nothing new, so `objectstack migrate meta` has nothing to rewrite. The body carries no migration prescription. The only party affected is a TypeScript consumer that reads a tombstoned key into a typed slot, and the compiler shows that consumer the dead read at its own call site. The other categories are closed on facts: the package publishes (not `unpublished`); no ADR-0087 id covers this type mark and this diff adds none (not `registered` / `already-registered`); `retiredKey` is referenced in code at 287 sites (not `runtime-interface-only`); and the diff touches `packages/spec` (not `type-surface-only`). -->
+
+**What an author sees.** Every key a `retiredKey()` tombstone declares used to be typed `undefined`, so writing one failed `tsc` with an error that named no retirement:
+
+```
+error TS2322: Type 'string[]' is not assignable to type 'undefined'.
+```
+
+Upgrading authors read those errors as typing bugs. The same line now fails like this, for arrays, objects and primitive values alike:
+
+```
+error TS2741: Property ''[REMOVED] Key retired: run `os validate` for its migration.'' is missing in type 'string[]' but required in type '{ '[REMOVED] Key retired: run `os validate` for its migration.': never; }'.
+error TS2322: Type 'string' is not assignable to type '{ '[REMOVED] Key retired: run `os validate` for its migration.': never; }'.
+```
+
+A hover on the key shows the same text. `os validate` and the parse print the key's own prescription, which says what replaced the key and gives the one-line fix.
+
+**What changed.** The declared type of each tombstoned key, on both the input side (`z.input`, the bare `X` aliases) and the parsed side (`z.infer`, the `XParsed` aliases), is now `` { '[REMOVED] Key retired: run `os validate` for its migration.': never } | undefined `` instead of `undefined`. No value can have that object type, because its one property is typed `never`. So `tsc` still accepts only absence, as before. Both sides carry the same type, which keeps the ADR-0122 isomorphism pins true.
+
+**What did not change.** Runtime behaviour is the same. Each tombstone is still `z.never().optional()`. The parse error and its prescription, the text `os validate` prints, the ADR-0087 conversions, the JSON schemas and the authorable-surface artifacts are all unchanged. No export was added or removed.
+
+**BREAKING**: a read of a tombstoned key into a slot typed `undefined`, or typed with the key's old type, no longer compiles, because the key's declared type is now its tombstone mark. The key never holds a value, so delete the dead read. It ships as `minor` under the launch-window convention.
+
+**Who might notice.** Code that assigns a tombstoned key's value to a slot typed exactly `undefined` (for example `const x: undefined = page.assignedProfiles`) no longer compiles. The key never holds a value, so delete the read. The published declaration files are about 7.5% larger, because the declaration emitter writes the type out at every site of the 287 `retiredKey()` calls.

.changeset/20686-pure-plugin-skips-strings.md

@@ -0,0 +1,11 @@
+---
+'@objectstack/spec': patch
+---
+
+fix(spec): the published bundles' text equals the source again — the build marks pure calls only, never a string (#20686)
+
+The build annotates each call of `lazySchema`, `strictObject` and `defineForm` as pure so a consumer's bundler can drop the schemas it never reaches. The rule that placed those annotations also rewrote a marked name quoted inside a string, so one D3 migration entry of protocol 18 (`dashboard-widget-stage-order-non-funnel-refused`) shipped a build-time comment marker inside its `acceptanceCriteria` text in `@objectstack/spec/migrations`, where its source reads `` `z.strictObject(DashboardWidgetSchema.shape)` ``. `os migrate meta` prints that text as the entry's `verify:` line when protocol 18 is the migration target. The published value now equals the source, character for character.
+
+The annotations now come from the TypeScript parse of each file, so a marked name inside a string, template text, a comment or a declaration is never touched. Every call that carried an annotation still carries one, and the published code is otherwise unchanged. One call, `z.strictObject(…)` in the Turso driver config schema, now has its annotation in front of the call rather than after the `z.`, where esbuild had been dropping it. Measured with esbuild, a consumer that imports one schema from `@objectstack/spec/data` or `defineStack` from the root gets a bundle of the same size as before.
+
+Clause-②: no

.changeset/20822-driver-sql-turso-copies.md

@@ -0,0 +1,28 @@
+---
+'@objectstack/driver-sql': minor
+'@objectstack/driver-sqlite-wasm': patch
+'@objectstack/driver-turso': patch
+'@objectstack/plugin-security': patch
+'@objectstack/spec': patch
+---
+
+fix(driver-sql, driver-turso, plugin-security, spec)!: `SqlDriver` and `TursoDriver` compile the filter they are handed — their copies of the whole-day bound and of the NULL-safe `$not` rewrite are deleted, and the RLS compile seam lowers type-blind when it cannot read the declared types (ADR-0053 D-D1 items 5, 7 and 9, #20822)
+
+Clause-②: no (narrowing)
+
+<!-- adr-0087: registered driver-sql-calendar-day-methods-removed -->
+
+**BREAKING**: `SqlDriver` in `@objectstack/driver-sql` loses three `protected` methods: `calendarDayExclusiveUpperBound`, `calendarDayUpperBoundRewrite` and `calendarDayBetweenRewrite`. They were the driver's copy of the whole-day bound, which the shared lowering now applies once, at the seams, before a driver sees the filter. A subclass of `SqlDriver` that calls one of them, or overrides one with the `override` modifier, no longer compiles (TS2339, TS4113). That includes a subclass of `SqliteWasmDriver` or `TursoDriver`, which extend `SqlDriver`. A subclass that re-declares one without `override` still compiles, but the driver never calls it, so the rule it carried stops applying. It ships as `minor` under the launch-window convention. The class's public methods are unchanged.
+
+FROM → TO: a `SqlDriver` subclass that called `this.calendarDayUpperBoundRewrite(table, field, op, value)`, `this.calendarDayBetweenRewrite(table, field, value)` or `this.calendarDayExclusiveUpperBound(table, field, value)`, or overrode one of them, lowers the filter with `lowerFilterCondition` from `@objectstack/spec/data` instead, before the driver compiles it: `lowerFilterCondition(where, { isDatetimeColumn })`, where `isDatetimeColumn` answers which columns get the whole-day bound.
+
+**Supersedes two sentences of this release's shared-lowering entry** (`lowerFilterCondition`, #5930), which this change makes false:
+
+- "A guard without that set treats no column as `datetime`." Now: when the RLS compile seam has no field guard, or one without a `datetime` set, it cannot read which columns are `datetime`, so it applies the whole-day rule to every column (the `@objectstack/plugin-security` entry below).
+- "Each driver keeps its own copy of these rules, and every copy gives the same answer on lowered input." Now: `SqlDriver`, `SqliteWasmDriver` and `TursoDriver` keep no copy of the whole-day bound or of the NULL-safe `$not` rewrite. A read through the engine or the RLS compile seam gets the lowered answer, and a call on the driver itself gets the comparison it wrote (the `@objectstack/driver-sql` and `@objectstack/driver-turso` entries below).
+
+- **`@objectstack/plugin-security` — an RLS policy compiled with no field guard is lowered type-blind.** The RLS compile seam runs the shared `lowerFilterCondition` on every compiled `using` and `check` filter. When the security plugin could not resolve the object's declared fields (no field guard), or a caller of `RLSCompiler.compileFilter` passes a guard without a `datetime` set, the seam cannot read which columns are `datetime`, and it now applies the whole-day rule to every column (a bare-day upper bound becomes `$lt` the next day), as ADR-0053 D-D1 item 7 rules for a seam that cannot read the type. It used to read no column as `datetime`, which left the bound to each driver's own copy of the rule. Visible on a `using` policy such as `record.signed_on <= '2026-01-05'` on such an object: every row of that day is kept on every driver, including `InMemoryDriver`, which had compared it as written since its own copy was deleted. A guard with a `datetime` set is unchanged, and so are the NULL-polarity guards.
+- **`@objectstack/driver-sql` — `SqlDriver` keeps no copy of the rules the seams apply.** Deleted: the whole-day rewrite of a bare-day `$lte` and of a `$between` maximum on a `datetime` column, on the plain and the legacy-normalised column paths, including the last supported day (the protected methods `calendarDayExclusiveUpperBound`, `calendarDayUpperBoundRewrite` and `calendarDayBetweenRewrite` are removed from the class); and the NULL-safe rewrite of a `$not` operand (`nullSafeNegationOperand` and its polarity tables, module-private). A read through the engine or the RLS compile seam is unchanged: the seam hands the driver a filter the shared lowering has already rewritten, and the deleted copies gave the same answer on that input. A caller that passes no seam — `find`, `findOne`, `count`, `aggregate`, `distinct`, `updateMany`, `deleteMany` or `findWithWindowFunctions` called on the driver itself — now gets the comparison it wrote: a bare-day `$lte` compares against that day's midnight, a `$between` is inclusive at both ends, `$lte '9999-12-31'` compares against that midnight, and a `$not` is SQL's three-valued negation, so a row whose compared column is NULL is not returned by it. `$ne`, `$nin` and `$notContains` keep their NULL-safe form, which this emitter spells for the operator itself. The refusal of an `undefined` comparand (`INVALID_FILTER` / 400) is kept: without it some positions would answer instead of refusing. To keep the seam's reading on a direct call, lower the filter first: `driver.find(object, { where: lowerFilterCondition(where, { isDatetimeColumn }) })`, with `lowerFilterCondition` from `@objectstack/spec/data`. A subclass that called or overrode one of the three removed methods: see **BREAKING** above.
+- **`@objectstack/driver-sqlite-wasm` — `SqliteWasmDriver` inherits the `SqlDriver` change above**, with the same answers on a seamed read and on a direct call.
+- **`@objectstack/driver-turso` — both faces of `TursoDriver` compile the filter they are handed.** Local and replica mode inherit the `SqlDriver` change. Remote mode: `toRemoteFilter` no longer widens a bare-day `$lte` or a `$between` maximum (it still splits a two-bound `$between` into the `$gte` / `$lte` pair the remote transport compiles, both ends inclusive, and still converts each comparand to storage form), and `RemoteTransport` no longer rewrites a `$not` operand (its copy of the polarity tables is deleted). The two faces still answer every filter alike, on a seamed read and on a direct call. The remote transport keeps its refusal of an `undefined` comparand, worded as `driver-sql`'s, so both faces refuse it in one sentence. The same one line keeps the seam's reading on a direct call.
+- **`@objectstack/spec` — the ADR-0087 ledger records the removal.** The protocol-18 step of `MIGRATIONS_BY_MAJOR` gains the semantic entry `driver-sql-calendar-day-methods-removed`, which names the three removed methods with their replacement and acceptance criteria. Every upgrade channel that projects protocol 18 carries it. `spec-changes.json` and the generated upgrade guide stop at the current protocol, 17, so neither changes in this release. A subclass that re-declares one of the methods without `override` still compiles and is never called, so the ledger, not the compiler, is the notice that reaches it.

.changeset/20825-cli-picklist-extend.md

@@ -0,0 +1,32 @@
+---
+'@objectstack/cli': minor
+---
+
+feat(cli)!: `objectstack validate` and `objectstack build` refuse a `picklistExtensions` entry whose `extend` names no picklist the stack declares (#20825)
+
+Clause-②: no (narrowing — `objectstack validate` / `objectstack build` newly refuse a `picklistExtensions[].extend` that names no picklist the stack declares; nothing is accepted that was refused before)
+
+<!-- adr-0087: not-required (no-migration-prescription) Nothing authorable changes spelling or type: `packages/spec` is untouched, and `picklistExtensions[].extend` stays the snake_case name it was. What changes is that two authoring commands now refuse one authored shape, an extension whose `extend` names no picklist in the stack. `objectstack migrate meta` could not rewrite that shape even in principle, because which list the author meant is not in the metadata. Nothing here judges a stored row. -->
+
+**BREAKING** — an accept-set narrowing on two authoring commands, shipped as
+`minor` under the launch-window convention. A stack with a `picklistExtensions`
+entry whose `extend` names no picklist the stack declares — `extend: 'industy'`
+beside a `picklists: [{ name: 'industry', … }]` — used to pass `objectstack
+validate` and `objectstack build` (which wrote the artifact). Both now exit 1 and
+name the extension and the list it names (`picklist-reference-unknown`, the rule a
+field's dangling `picklist` already gets).
+**One-line fix:** correct `extend` to the picklist the entry adds options to, declare
+the list it names (`picklists: [{ name, label, options }]`, or a `*.picklist.ts` file
+the stack imports), or remove the entry.
+
+**Which extensions are judged.** The ones the load path registers: the top-level
+`picklistExtensions` of a one-package stack, or each `packages[]` entry's own. An
+`extend` resolves against every picklist the stack declares, including one a sibling
+package in the same artifact owns.
+
+**A list from a package outside the stack is reported, not refused.** When the
+package declaring the extension lists a `manifest.dependencies` entry the stack does
+not carry, the list may live there, and these commands cannot read it. That
+extension is an `info` notice (`picklist-reference-unverified`) in `warnings` and on
+the console, naming the extension, the list and the dependencies — never a failure,
+not even under `--strict`.

.changeset/20889-analytics-native-measure-number.md

@@ -0,0 +1,45 @@
+---
+'@objectstack/core': minor
+'@objectstack/driver-sql': patch
+'@objectstack/service-analytics': patch
+---
+
+fix: the analytics native-SQL path answers a measure its response declares `number` as a number on every dialect, presented by the one rule `driver-sql`'s `aggregate()` applies, which `@objectstack/core` now exports as `AGGREGATE_ANSWER_KIND` and `presentAsNumber` (#20889)
+
+Clause-②: yes (widening)
+
+**New exports.** `@objectstack/core` exports two names, moved here unchanged
+from `@objectstack/driver-sql`, which now imports them instead of keeping them
+private:
+
+- `AGGREGATE_ANSWER_KIND`: what each declared aggregate function answers.
+  `count`, `count_distinct`, `sum` and `avg` answer `'number'`; `min` and `max`
+  answer `'column'`, a value of the aggregated column.
+- `presentAsNumber(value)`: the `'number'` presentation. A string `Number()`
+  reads as a number becomes that number. Any other value is returned as given:
+  a number, `null`, a boolean, empty or blank text, or text that reads as NaN.
+
+**What changed.** On PostgreSQL, `POST /api/v1/analytics/query` and
+`POST /api/v1/analytics/dataset/query` answered through `NativeSQLStrategy`
+returned count, count_distinct, sum, avg, and min / max over a numeric column
+as strings, such as `count: "2"` and
+`sum: "500.000000000000000000000000000000"`, while `fields[]` declared
+`number`. A dataset's `row_count` did the same, and a measure-scoped count
+mixed `"1"` with the number `0` in one column. SQLite answered numbers. The
+strategy now presents each measure column by its declared aggregate function,
+through the same table and presenter as `SqlDriver.aggregate()`. `min` / `max`
+are presented only when their column is declared numeric, so `max` over a text
+column, every dimension, and expression measures keep the value the database
+returned.
+
+**Precision.** The answer is one JS number, the policy `driver-sql`'s
+`aggregate()` already applies. A total that needs more digits than a double
+holds, such as `9007199254740993`, answers the nearest double
+(`9007199254740992`), which is also what SQLite and the engine path answer.
+
+**What did not move.** `@objectstack/driver-sql`'s behaviour is unchanged: its
+`aggregate()` reads the same table, and its read presenter calls the same
+function. The answers on SQLite are byte-identical. The arithmetic of the
+analytics native statement did not change either. On PostgreSQL its `sum` and
+`avg` still add exact decimals, so `0.1 + 0.2` answers `0.3` where the engine
+path answers `0.30000000000000004`.

.changeset/20910-regime-c-action-permission-paths.md

@@ -0,0 +1,23 @@
+---
+'@objectstack/metadata-protocol': patch
+---
+
+fix(metadata-protocol): every refusal of an in-place edit of a packaged flow, action or permission set names that type's own sanctioned path, at every door, and a packaged action's removal names one too, not a redeploy or `OS_METADATA_WRITABLE` (#20910)
+
+Clause-②: no
+
+ADR-0126 puts `flow`, `action` and `permission` in Regime C. The packaged base is locked, and the refusal names the sanctioned path. Until now only a flow's package-less refusal did, and only at one of the three places that refuse such a write. The other places prescribed "Edit the source artifact and redeploy, or set OS_METADATA_WRITABLE …" or "Set OS_METADATA_WRITABLE to enable additional types at runtime". A packaged action's removal named no path at all.
+
+There is now one regime table, and each type's row names only the primitives that type has:
+
+- a packaged flow: clone it under a new name with `POST /api/v1/automation/:name/clone` and `{ name, label }`, or switch it off with `POST /api/v1/automation/:name/toggle` and `{ enabled: false }`;
+- a packaged action: switch it off with `POST /api/v1/actions/_activation/:object/:action` and `{ enabled: false }` (`:object` is `global` for an object-less action). No clone is named, because cloning an action is not a sanctioned path;
+- a packaged permission set: clone it under a new name, with the "Clone" action on the permission set or `POST /api/v1/data/sys_permission_set` with a new name. This is the same wording the permission-set lock in `@objectstack/plugin-security` already uses.
+
+The switches are operator-only where one install serves several organizations. Every refusal cites ADR-0126. All three places that refuse such a write read the same table:
+
+- **`403 NOT_OVERRIDABLE` on an environment-scoped kernel.** This covers `PUT /api/v1/meta/:type/:name` without `?package=`, and for a flow or an action `DELETE` too.
+- **`403 NOT_OVERRIDABLE` where the `/meta` protocol is not environment-scoped**, for example the default local `pnpm dev` boot. The metadata repository refuses that write one layer down, and now with the same sentence.
+- **`403 ITEM_LOCKED` for a write that names the read-only package with `?package=`, while `OS_METADATA_WRITABLE` is not set for the type.** The sentence now opens "Cannot overlay 'TYPE' in package 'ID': that package is read-only, and its packaged base is locked against in-place edits." and then names the path.
+
+A packaged flow's package-less sentence is byte-for-byte unchanged. Statuses, codes, `lockSource`, `packageId`, `docs` and which writes are refused are unchanged too. `OS_METADATA_WRITABLE` still opens the lock for a write that names no package. The `ITEM_LOCKED` refusal given while the variable IS set reads exactly as before. Removing a permission set's overlay row is still allowed, as repair. Every type with no declared regime reads exactly as before, at every door.