Merge remote-tracking branch 'origin/main' into claude/issue-17306-screen-field-keys-live

Author: claude

.changeset/21277-agent-structured-output-json-only.md

@@ -0,0 +1,78 @@
+---
+'@objectstack/spec': minor
+---
+
+feat(spec)!: an agent's `structuredOutput` is JSON-only — the `regex` / `grammar` / `xml` formats and the `coerce_types` step are retired, and the block is `live`, enforced by the cloud AI runtime (#21277)
+
+**BREAKING** — four members leave the agent's structured-output vocabulary:
+`regex`, `grammar` and `xml` from `StructuredOutputFormat` (so from
+`agent.structuredOutput.format` and `agent.structuredOutput.fallbackFormat`), and
+`coerce_types` from `TransformPipelineStep` (so from
+`agent.structuredOutput.transformPipeline`). ADR-0049 enforce-or-remove, ruled
+retire. The cloud AI runtime, the one runtime that executes agents, enforces
+`structuredOutput` on every final answer and refused an agent declaring any of the
+four before its first turn: the spec never had a key to carry the pattern or
+grammar a `regex` / `grammar` answer would be checked against, an answer is checked
+only as JSON, and no coercion engine exists. So no authored value of the four ever
+did what it named, and authoring now refuses them by name instead of the first
+live turn refusing the agent. `json_object`, `json_schema`, `trim`, `parse_json`
+and `validate` are unchanged.
+
+### FROM → TO
+
+| removed | what to write instead |
+| --- | --- |
+| `structuredOutput.format: 'regex'`, `'grammar'` or `'xml'` | `format: 'json_schema'` with a JSON Schema in `schema` when the answer must have a shape, or `format: 'json_object'`; or delete the `structuredOutput` block if the agent needs no output contract. |
+| `structuredOutput.fallbackFormat: 'regex'`, `'grammar'` or `'xml'` | `'json_object'` or `'json_schema'`, or delete the key. |
+| `'coerce_types'` in `structuredOutput.transformPipeline` | delete the step, and declare the exact types in `schema` so the answer is validated as the model wrote it. |
+
+**The one-line fix: use `json_schema` with a JSON Schema; drop `coerce_types`.**
+`os migrate meta --from 17` lists the mechanical edits for existing sources.
+
+Each retired member is refused at parse with a prescription naming the JSON
+formats, and in `tsc` (the members are gone from the `StructuredOutputFormat` /
+`TransformPipelineStep` types). Any other unknown value keeps zod's own message.
+
+### The retirement kit
+
+- **Value-level retirement.** Both enums are declared through
+  `enumWithRetiredValues` (`shared/retired-key.ts`), the house mechanism for a
+  narrowed vocabulary, with the prescriptions module-private. No authorable KEY and
+  no def changed, so nothing lands in `RETIRED_KEYS_BY_MAJOR` and the four surface
+  ratchets (`api-surface`, `authorable-surface`, `json-schema.manifest`,
+  `api-surface-signatures`) are byte-identical.
+- **D2 conversion `agent-structured-output-refused-members-removed`** (step 18,
+  retired from the load path): it deletes a `structuredOutput` block whose `format`
+  was retired (the format is required, and no rewrite can say which JSON contract
+  was meant), deletes a retired `fallbackFormat`, and drops `coerce_types` from the
+  pipeline, keeping the other steps in order. Stored `sys_metadata` agent rows replay
+  it at rehydration; one notice per edit.
+- **D3 entry `agent-structured-output-refused-members-retired`** carries the
+  judgement the conversion cannot make: whether an agent whose block was deleted
+  should now carry a `json_schema` contract.
+- **No deprecation window**, per the project's startup-stage posture.
+
+### Describes and the liveness ledger
+
+- `agent.structuredOutput` drops `[EXPERIMENTAL — not enforced]`: it states that the
+  cloud AI runtime enforces it on every final answer and that the open framework
+  edition does not run agents. Its ledger row moves `experimental` → `live`, citing
+  the cloud readers (`agent-runtime.ts#compileStructuredOutput`,
+  `ai-service.ts#AIService.settleFinalAnswer`) as attested by the cloud seat's
+  reading at cloud `cb62c3ea`, `verifiedAt` 2026-10-02. `os lint` / `os validate` no
+  longer warn `liveness-experimental-property` on an agent that sets it.
+- `fallbackFormat`'s describe states what the runtime does with it: once the primary
+  format's retries are spent, the last answer is checked against the fallback.
+- `guardrails.blockedTopics`'s describe states the enforced match: an exact,
+  case-sensitive match on the tool name, on `action_` plus the action type, or on
+  the tool category.
+- The generated agent reference page follows.
+
+⚠️ **The out-of-repo consumer population is NOT MEASURED.** `@objectstack/spec` is
+published, and tenant-authored agents were not measured. This repo authors no
+`structuredOutput` outside `packages/spec`, and the cloud seat's reading found no
+producer in cloud.
+
+Clause-②: no (narrowing)
+
+<!-- adr-0087: registered agent-structured-output-refused-members-removed, agent-structured-output-refused-members-retired -->

.changeset/21316-objectql-face-order-limit.md

@@ -0,0 +1,20 @@
+---
+"@objectstack/service-analytics": patch
+---
+
+fix(service-analytics): the ObjectQL strategy applies a query's `order`, then its `offset` and `limit`, to the aggregated answer, as its echoed `sql` says
+
+Clause-②: no
+
+**Before**, the ObjectQL strategy passed none of the three keys to `engine.aggregate`, which has no ordering or window grammar, and applied none of them itself. Every date-bucketed query lands on that strategy, because the native-SQL strategy declines `granularity`. Measured through `POST /api/v1/analytics/query` on SQLite and PostgreSQL 16.14:
+
+- `timeDimensions: [{ dimension: 'closed_on', granularity: 'month' }]`, `order: { closed_on: 'desc' }`, `limit: 1` answered every month, unordered (ascending on SQLite, `04, 03, 05` on PostgreSQL).
+- A selected dimension with `order: { note: 'desc' }`, and a selected measure with `limit: 2, offset: 1`, answered every group in the engine's order.
+
+The echoed `sql` and `POST /api/v1/analytics/sql` rendered `ORDER BY … LIMIT … OFFSET …` for all three.
+
+**Now** the strategy orders the answer by `order`, in the key order given, and then applies `offset` and `limit`. This happens on the direct path and on the cross-object (FK-expand) path, after the re-bucket. A bare `limit` with no `order` slices the engine's order, as `LIMIT` without `ORDER BY` does. Where the native-SQL strategy answers the same query, the two answer the same rows for numbers and for text of single-case ASCII letters. The comparison is the dataset door's own `applyOrdering`, which sorts NULL and `''` last in both directions, while SQL places NULL by driver (lowest on SQLite, highest on PostgreSQL), so the two faces can still order NULL, `''`, numeric text and mixed-case text differently.
+
+**Dataset door.** `POST /api/v1/analytics/dataset/query` pushes a single query's `order`, `limit` and `offset` down to the strategy, and then windowed the answer a second time, so `offset` was applied twice. `limit: 2, offset: 1` over five groups answered one row, the third, on the native-SQL strategy. It now windows only a grid it could not push down. The ObjectQL strategy answered that page correctly before, because it dropped the window; it still does.
+
+**Unchanged.** A query with no `order`, `limit` or `offset` answers exactly the engine's aggregate rows. Which `order` keys are accepted is unchanged: the analytics door still refuses a key the query does not select. The dataset door's own ordering is unchanged too: label sort keys, derived measures, the implicit dimension order for a bare `limit`, and the chronological default.

.changeset/21323-verify-author-time-rules.md

@@ -0,0 +1,22 @@
+---
+'@objectstack/cli': minor
+---
+
+fix(cli)!: `os verify` runs the author-time rules first, and a stack they refuse fails `verify` with the findings `os validate` reports (#21323)
+
+Clause-②: yes (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) a CLI command's verdict, not a declaration: os verify now refuses, before it boots anything, a stack the author-time rule registry refuses or that does not parse against the protocol schema. No authorable key, spelling, export or stored shape moves: every stack parses and loads exactly as before, os validate and os build answer exactly as before, and nothing on the metadata load path is read or rewritten. What an author does about a refusal is fix the finding the message names, which os validate and os build already report for the same stack, so there is no rewrite a ledger entry could carry. The other categories are closed on facts: the package publishes (not unpublished); no ADR-0087 id covers a command's verdict (not registered / already-registered); and the change is CLI behaviour, not a TypeScript declaration (not runtime-interface-only / type-surface-only). -->
+
+**BREAKING** — `os verify` narrows what it passes. It ships as `minor` under the launch-window convention for accept-set narrowings.
+
+**What was accepted before.** `os verify` booted the app and exercised CRUD round-trip fidelity and, with `--rls`, the RLS invariant — and nothing else. A stack carrying a lookup to an object that does not exist, an action `visible` expression naming a field without `record.`, or a list column naming no field booted, round-tripped its records and printed `✓ verify passed` at exit 0, while `os validate`, `os build` and `os lint` all refused it. The documented done-bar ("`objectstack verify` is green") was green on a stack the build refuses to ship.
+
+**What is refused now.** `os verify` runs two stages. The first is the author-time rule registry `os validate` runs, over the stack prepared the way `os validate` prepares it: normalized, inline handlers lowered, parsed against the protocol schema, the SDUI manifest read beside the config, judged whole and then once per package of a multi-package artifact. A gating finding, or a stack that does not parse, exits 1 with those findings and the runtime stage never starts:
+
+- text face: `✗ Author-time rules failed (N issues) — the runtime stage did not run`, then each finding with its rule and location (the per-package and schema refusals have their own sentence);
+- `--json`: the command's failure envelope, `error` (the sentence), plus a new key, `errors`, carrying the findings in the shape `os validate --json` carries them under `errors` — rule findings (with `package` on a per-package one), or the schema issues.
+
+Advisories never fail the stage; the text face counts them and points at `os validate`. On a passing stack the text face prints one step line and `✓ Author-time rules passed (N rules)` before the runtime stage, and the `--json` report of a run that reaches the runtime stage is unchanged.
+
+**Who is affected.** Only a stack `os build` already refuses: the first stage runs the same gating rules over the same prepared stack, so every stack it refuses, `os build` refuses too. The remedy is the one `os validate` prints for each finding. Measured with this branch's CLI over the examples at `222ecc27f9` (unchanged on this branch): `os validate` exits 0 on `examples/app-todo`, `examples/app-crm`, `examples/app-showcase` and `examples/app-multi-package`, so none of the four is refused by the new stage.

content/docs/deployment/cli.mdx

@@ -1614,6 +1614,7 @@ emits a scoped, publishable `@objectstack/plugin-<name>`, and it lands under
 | Command | Description |
 |---------|-------------|
 | `os lint [config]` | Every author-time gate `validate`/`build` run, plus style and convention checks |
+| `os verify` | The author-time rules `validate` runs, then boot the app in-process and verify it through the real HTTP stack |
 | `os test [files]` | Run Quality Protocol test scenarios against a running server |
 | `os doctor` | Check development environment health |
 
@@ -1679,6 +1680,62 @@ the flag was in effect), `failing` (the count the exit code was read from —
 `errors`, or `errors + warnings` under `--strict`) and `passed` (`failing` is
 `0` — the same statement the exit code makes).
 
+#### `os verify`
+
+The done-bar: `os verify` green means the author-time rules pass **and** the
+app behaves at runtime. It runs two stages, in order, and the second starts
+only when the first passes.
+
+```bash
+os verify                                      # Author-time rules, then CRUD round-trip fidelity
+os verify --app ./objectstack.config.ts --rls  # Also prove the cross-owner RLS invariant
+os verify --rls --multi-tenant --json          # Org-scoped boot, structured report
+```
+
+1. **Author-time rules.** The rule registry `os validate` runs, over the stack
+   prepared the way `os validate` prepares it: normalized, inline handlers
+   lowered, parsed against the protocol schema, and judged whole and then once
+   per package of a multi-package artifact. A stack that does not parse, or
+   that any gating rule refuses, fails the run here with those findings — the
+   same ones `os validate` reports — and the app is never booted. Advisories
+   (`warning` and `info` findings) never fail `os verify`; the text face counts
+   them, and `os validate` prints them. This stage is the rule registry, not all
+   of `os validate`: the package-docs lint, the capability-provider check, the
+   picklist-reference and view-container-name checks and `--strict` stay
+   `os validate`'s, so run it too.
+2. **Runtime.** Boot the app in-process and exercise it through the real HTTP
+   stack: for each object, a record derived from its fields is created, read
+   back and compared (CRUD round-trip fidelity), and a create, read or
+   fidelity failure fails the run. With `--rls`, a second fresh boot proves the
+   cross-owner invariant — a member must not write what it cannot read — for a
+   probe persona and for one persona per declared position, and a hole fails
+   the run. `--multi-tenant` boots org-scoped so tenant-isolation RLS policies
+   apply; a walled tenancy posture (`OS_TENANCY_POSTURE` set to `isolated` or
+   `group`) asks for the same boot.
+
+The config is the one `--app` names, or the auto-detected one — see
+[Config File Auto-Detection](#config-file-auto-detection).
+
+**Exit status.**
+
+| Exit | Meaning |
+|---|---|
+| `0` | Both stages passed: no gating author-time finding, and no runtime failure |
+| `1` | The author-time stage refused the stack (the runtime stage did not run), the runtime stage found failures, or the command could not run at all (no config found, a config that does not load, a boot failure) |
+
+**`--json`.** What the document carries depends on where the run ended:
+
+- **Refused by the author-time stage:** `{ "error": "<sentence>", "errors": [...] }`,
+  where `errors` carries the findings in the shape `os validate --json` carries
+  them under `errors` — the gating findings (`severity`, `rule`, `where`,
+  `path`, `message`, `hint`, plus `package` for a finding raised inside one
+  package of the artifact), or the schema issues when the stack does not parse.
+- **Reached the runtime stage:** the report — `app`, `config`, `multiTenant`,
+  `crud`, `rls` (with `--rls`) and `hardFailures`, the count the exit status is
+  read from.
+- **Could not run:** `{ "error": "<sentence>" }`, plus `code` and `httpStatus`
+  when the failure carries them.
+
 #### `os test`
 
 Runs Quality Protocol test scenarios (JSON-based BDD) against a running ObjectStack server.