Merge remote-tracking branch 'origin/main' into claude/issue-21388-withheld-only-activity-row

Author: claude

.changeset/20790-flow-credential-channel.md

@@ -0,0 +1,27 @@
+---
+'@objectstack/spec': minor
+'@objectstack/service-automation': minor
+'@objectstack/metadata-protocol': minor
+'@objectstack/trigger-api': minor
+'@objectstack/runtime': minor
+---
+
+feat(automation): a flow's credentials live in a write-only channel, not in its stored definition (#20790)
+
+Clause-②: yes (widening)
+
+A flow's two credentials, an inbound hook's `secret` on its start node and an `http` node's `signingSecret`, are no longer stored in the flow definition. The metadata save door moves each explicit value into a new platform object, `sys_flow_credential`, owned by `@objectstack/service-automation`. Its one field is `type: 'secret'`, so the engine encrypts it through the host crypto provider, masks it on every read, and dereferences it only through `resolveSecretField`. This is the same seam the webhook signing secret uses. The stored row, every new version-history row and the row's content hash carry no credential. The engine reads the value only when it verifies an inbound post or signs an outbound request. Authoring does not change: you still write the literal, a save that leaves the key out (the form every read serves) keeps the stored secret, `''` clears it, and only an explicit new value rotates it.
+
+**⚠️ Rotate every inbound and outbound flow secret that existed before this release.** On the first boot with a crypto provider, or when a provider registers after a boot without one, each stored flow that still carries a credential is moved into the channel once, and the log prints one notice per flow: `[Automation] flow '<name>' (<state>): … was stored in cleartext … ROTATE: …`. The move guarantees no new copy, but the version-history rows and audit snapshots written before it stay as they were (both are append-only), so an administrator could have read those values. To rotate, save the flow with a new `config.secret` / `config.signingSecret`, then give the new value to whoever signs posts to the hook or verifies its deliveries. The run is recorded in `sys_migration` as `flow-credential-channel` (flow names only, never values). Packaged flows are not moved: a packaged flow's literal stays its source of truth, and where the channel holds a row for it, the row wins at verification.
+
+What else changes:
+
+- **`@objectstack/spec`**: `PLATFORM_OBJECTS_BY_PACKAGE['service-automation']` lists `sys_flow_credential`.
+- **`@objectstack/metadata-protocol`**: `registerCredentialChannel(type, channel)` registers a type's write-only credential channel (exported type `MetadataCredentialChannel`). `saveMetaItem` stores the body the channel returns, after the carry-forward and before the put. The runtime authoring gate reads the channel's held positions as present, on an active save and when a draft is published. `SysMetadataRepository.restoreVersion` takes `deriveRestoredBody`, shaped like `promoteDraft`'s `deriveActiveBody`. Rollback and revert pass the channel's strip, so restoring a version written before the move never puts its credential back at rest, and the channel keeps its current credential.
+- **`@objectstack/service-automation`**: exports `SysFlowCredential`, `FlowCredentialChannel` and `migrateFlowCredentialsIntoChannel`. `AutomationEngine` gains `setFlowCredentialSource`, `holdsFlowCredential`, `resolveFlowCredential` and `flowCredentialHoldings`. An `api` binding carries `resolveSecret()`, which reads the secret at verification time, so a rotation applies to the next post. A draft save never rotates the live secret; publishing the draft promotes it. Deleting a flow's stored row drops its credentials.
+- **`@objectstack/trigger-api`**: `FlowTriggerBinding.resolveSecret` arms a hook without a literal. A post whose secret cannot be read is answered `503 SERVICE_UNAVAILABLE` and is never verified against nothing.
+- **Refused now, loudly**:
+  - With no crypto provider, a save that carries a flow credential is refused with `503 SERVICE_UNAVAILABLE` before anything is written. Register a provider (`setCryptoProvider`) and save again.
+  - The clone door (`POST /api/v1/automation/:name/clone`) refuses a source that holds a credential, as a literal or in the channel, with `409 RESOURCE_CONFLICT`, because a copy would share it. ⚠️ Accepted cost: a packaged inbound flow can no longer be cloned in one step. Author the copy as a new flow under a new name, with its own secret.
+
+<!-- adr-0087: not-required (no-migration-prescription) the one-time move rewrites stored flow rows through the metadata save door itself, at boot; no authorable key, spelling, export or stored shape is retired, so an author or an upgrading agent has nothing to rewrite. The operator's action is the rotation stated above, which is not a FROM to TO mapping. The gate reads this changeset as non-breaking; the disposition is stated for the migration the ruling named. -->

.changeset/20822-retired-matcher-pointers.md

@@ -0,0 +1,23 @@
+---
+'@objectstack/spec': patch
+'@objectstack/service-analytics': patch
+'@objectstack/formula': patch
+'@objectstack/objectql': patch
+---
+
+Published comments that named `driver-memory`'s retired reference matcher as a live filter backend now name what replaced it
+
+Clause-②: no
+
+`driver-memory`'s reference matcher (`memory-matcher.ts`) was retired in commit `8fec76a2b`. Four published packages still described it as a live surface in text that ships:
+
+- `@objectstack/spec`:
+  - The backend table in the filter-logic conformance docblock, which ships in `data/index.d.ts` and `data/index.d.mts`, now lists the in-memory backend as `driver-memory`'s query path (`normalizeFilterCondition`, then mingo) where it listed `memory-matcher`, and says the matcher held that row until commit `8fec76a2b` retired it.
+  - `src/data/filter.zod.ts` ships as source. In it, the `$icontains` implementation table lists `driver-memory`'s query path and analytics face, both on `asciiCaseInsensitiveRegexSource`. The `$like` / `$ilike` and `$empty` tables keep the matcher only in a note that commit `8fec76a2b` retired it. The `foldAsciiCase` docblock counts five JS evaluation faces where it counted six. The `asciiCaseInsensitiveContains` docblock names objectql's `having` and `formula` as its callers. The string-ordering note says `driver-memory`'s query path hands the comparison to mingo. Of these, the `foldAsciiCase`, `asciiCaseInsensitiveContains` and `FILTER_OPERATORS` docblocks also ship in the filter declaration chunk (`filter.zod-*.d.ts` / `.d.mts`).
+  - `src/ui/view.zod.ts` ships as source. It now says that `driver-memory`'s query path runs `assertFilterConditionShape` through `convertToMongoQuery`, where it said `match()` did.
+  - A comment inside `FILTER_TEXT_CASES` ships in `data/index.js` / `.mjs` and `browser/data/index.js` / `.mjs`. It now says the reference matcher measured case-exact until commit `8fec76a2b` retired it.
+- `@objectstack/service-analytics`: two comments in `ObjectQLStrategy`, which ship in the JavaScript output (the first also in `index.d.ts` / `index.d.cts`), changed. The first names `driver-memory`'s query path, not its matcher, as a face that pins `{$not: {}}` as the zero-row filter. The second says in the past tense that `memory-matcher.ts` read `$regex` as a real regex, until `$regex` was retired and commit `8fec76a2b` retired the matcher too.
+- `@objectstack/formula`: the comment over the `$icontains` arm in `matches-filter.ts` ships in `index.js` / `index.mjs`. It now names objectql's `having` as the other caller of `asciiCaseInsensitiveContains`. It says `driver-memory`'s reference matcher called it until commit `8fec76a2b` retired it, and that `driver-memory`'s query path folds through `asciiCaseInsensitiveRegexSource`.
+- `@objectstack/objectql`: the comment over the `having` walker's `$notContains` arm in `having-filter.ts` ships in `index.js` / `index.mjs` and `core.js` / `core.mjs`. It now says the record-at-a-time faces (`formula` and this walker) answer the predicate on a stored value that is not a string, as `driver-memory`'s reference matcher did until commit `8fec76a2b` retired it.
+
+Comment only: no export, type, error code, status, message text or runtime behaviour changes.

.changeset/21299-having-no-class-reference.md

@@ -0,0 +1,28 @@
+---
+"@objectstack/objectql": minor
+---
+
+fix(objectql)!: `having` and the per-aggregation `filter` refuse a `{ $field }` comparison against a column with no comparison class (a file field, a list, a formula) with `INVALID_FILTER` / 400, as `where` refuses it; `applyInMemoryAggregation` takes the same reference rules
+
+Clause-②: no (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) a refusal of filter STRUCTURE at the engine's query door and at its published in-memory aggregation function: a { $field } comparison against a column the spec's crossFieldComparisonVerdict answers no-class for, at having and at a per-aggregation filter, and the reference rules engine.aggregate already applied, now applied by applyInMemoryAggregation when its caller passes a field map. No authorable key, spelling, export or stored shape moves: FieldReferenceSchema, every query shape and every object definition parse as before, the package entries export nothing new and nothing less, applyInMemoryAggregation keeps its signature, and no stored row is read or rewritten. What is refused is a comparison the same query's where already refuses on driver-sql, and which comparable column the caller meant is not something a ledger entry can decide. The other categories are closed on facts: the package publishes (not unpublished); no ADR-0087 id covers a filter's comparison class (not registered / already-registered); and the change is runtime behaviour, not a declaration (not runtime-interface-only / type-surface-only). -->
+
+**BREAKING**: this narrows what a `{ $field }` reference may pair at two positions of `engine.aggregate`, and what `applyInMemoryAggregation` accepts when it is handed a field map. It ships as `minor` under the launch-window convention for accept-set narrowings. No export or published type changes.
+
+**What was accepted before.** The spec's comparison-class verdict (`crossFieldComparisonVerdict`) answers `no-class` for a pair in which either column has no comparison class: a list or an object (a structured-JSON type, a multi-option type, a multi-capable type flagged `multiple: true`), a file field (`FILE_REFERENCE_TYPES`), or a formula. `having` and a per-aggregation `filter` (`aggregations[i].filter`) did not judge that answer. Measured on `SqlDriver` over better-sqlite3 through `engine.aggregate`, beside a `where` twin that `driver-sql` refused `INVALID_FILTER` / 400 each time:
+
+- a per-aggregation `{ customer_id: { $ne: { $field: 'photo' } } }` (text against an image) counted 6 of 6 rows;
+- a per-aggregation `{ closed_at: { $lte: { $field: 'due_f' } } }` (datetime against a formula) counted 0 of 6;
+- a per-aggregation `{ amount: { $ne: { $field: 'tags' } } }` (number against a multiselect) counted 6 of 6 when the column held no value, 0 on an empty table, and was refused by the per-row array check, in other words, when the column held a list;
+- `having: { photo: { $ne: { $field: 'n' } } }` over a groupBy on an image field kept all 6 groups.
+
+A `{ $field, addDays }` pair against a formula was answered at both positions. `applyInMemoryAggregation`, called with a field map, applied none of `engine.aggregate`'s reference rules: a reference to a field the map does not declare, a pair across two classes and a pair against a column with no class were all counted.
+
+**What is refused now.** At `having` and at a per-aggregation `filter`, a scalar comparison (`$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`) whose `{ $field }` comparand, or whose own column, has no comparison class, with or without `addDays`. The refusal is `INVALID_FILTER` / 400, raised before any driver is asked for a row, on an empty set as on a populated one, in the reason `driver-sql`'s `where` logs for the same pair: the column it names (the referenced one first, as `where` asks it first) "has no scalar stored column a comparison can read". The per-aggregation `filter` withholds the fields, the operator and the reason from the message and writes them to the server log, as `where` does; `having` names the two columns of the query's own projection. A `{ $field, addDays }` pair against a file or list column was already refused, in the `addDays` pair rule's words (that rule reads those types as text, so it answered with a cross-class or an offset sentence); it is now refused in this one.
+
+`applyInMemoryAggregation(rows, ast, timezone, fields, reportWithheld)`, when `fields` is passed, judges each per-aggregation `filter` by the reference rules `engine.aggregate` applies at that position, through the same function, before any row is judged: the referenced column (and an `addDays` offset column) is declared, a pair across two classes or against a column with no class is refused, and an `addDays` pair follows its class rule. The refusal is `INVALID_FILTER` / 400; the withheld diagnostic goes to `reportWithheld`, and it names no object (this function is not told one).
+
+**The remedy.** Compare two columns that each have a comparison class, and the same one: a file field, a list and a formula have no stored scalar a comparison can read. Compare the scalar column the value is derived from, or filter the column with a literal.
+
+**Unchanged.** A reference between two columns of one class answers as before, and the cross-class refusal keeps its words. A side with no declaration is not judged at any of the three positions: a host with no registered object, a column the field map does not carry (`id`, and an audit-opt-out object's row-carried `created_at` / `updated_at`), an aggregation over an undeclared field. A declared type outside `FieldType` is not judged either. `applyInMemoryAggregation` called without `fields` judges nothing it did not judge before.

.changeset/21328-share-links-self-scoped-list.md

@@ -0,0 +1,17 @@
+---
+"@objectstack/plugin-sharing": patch
+"@objectstack/spec": patch
+---
+
+A plain member's share-link list now answers: `GET /api/v1/share-links` is self-scoped for every signed-in caller, as ADR-0111 rules it
+
+Clause-②: no
+
+`ShareLinkService.listLinks` read `sys_share_link` under the caller's context. Both share-link doors force the list's `createdBy` to the caller, but the read still needed an object-level grant on `sys_share_link`, and the platform's member baseline does not grant one. So every plain member's list was refused, with or without an object filter, and the Share dialog, which loads this list when it opens, showed an error for them on every record. An admin's list answered.
+
+- The caller's own list is now read under the system context. This happens only when the caller has a non-empty user identity and the creator filter equals it. The read is constrained server-side to that identity, and each row it returns must pass the creator rule before it leaves.
+- One creator rule now serves both `listLinks` and `revokeLink`. It never matches a caller with no user identity. Neither HTTP door reaches that case, because both answer 401 first, so for an internal caller with no user identity, `revokeLink` now refuses a link whose `created_by` is absent or empty instead of treating it as theirs.
+- Every other list shape keeps the caller's context, as before: no creator filter, another user as creator, no user identity, or an admin listing someone else's links. A system caller keeps its bypass.
+- The rows carry the same columns as before. The token comes back so the console can build the link URL, and the password hash never does.
+- `@objectstack/spec`: the `IShareLinkService.listLinks` doc comment now describes the self-scoped own list. It previously said every listing is read under `context`. This is a doc comment only, with no type or export change.
+- ⛔ No permission set changes, and no new grant on `sys_share_link`.

.changeset/21370-starter-field-groups.md

@@ -0,0 +1,12 @@
+---
+'create-objectstack': patch
+'@objectstack/cli': patch
+---
+
+fix: a fresh project no longer warns about its own starter fields after the first `objectstack generate`
+
+Clause-②: no
+
+The blank starter's `note` object (`npm create objectstack`) and the item object of the `app` template (`objectstack init -t app`) now declare one field group, `fieldGroups: [{ key: 'details', label: 'Details' }]`, and place every field in it with `group: 'details'`. Before this, the first view, flow, dashboard or other metadata that can read a field made `objectstack validate` and `objectstack lint` report `field-no-consumers` on a field the author never wrote: the note's `body`, or the item's `description` and `status`. That held whether the author generated it or wrote it by hand. Both commands still exited 0. A field placed in a declared group is drawn by the object's form and detail page, and the rule counts that as displayed, so a fresh project now reports nothing. The `plugin` and `empty` templates are unchanged: the plugin's one field is the record's title, which the rule never reports, and the empty template declares no object.
+
+**What changes for an author.** In a new project, the object's form and detail page show the starter fields in one section labelled Details instead of a flat list. A field you add joins a section the same way, by naming its `key` in `group`. A project scaffolded by an earlier release keeps its files. To clear the warning there, add the same `fieldGroups` entry to the object and `group: 'details'` to each field the warning names, or give each field another consumer, such as a view column.

.changeset/21379-position-address-readers.md

@@ -0,0 +1,18 @@
+---
+'@objectstack/plugin-approvals': patch
+---
+
+A holder of a position whose approval slot reads `position:<name>` now decides it from the stock console, sees `can_act` on it, and keeps sight of it after deciding; a reviewer named by a `user` approver authored as an email does too
+
+Clause-②: no
+
+A request whose approver position nobody held when it opened keeps the literal `position:<name>` slot. After the position is staffed, its holder found the request in "My Pending", but `viewer.can_act` was `false`, an approve with no `actorId` (what the console's approve action sends) or with `role:<name>` (the deprecated pre-rename spelling the console uses) answered 403, only naming `position:<name>` decided it, and `GET /api/v1/approvals/requests/:id` then answered 404 to the holder who had just decided it. A `user` approver authored as an email had the same shape: its reviewer saw neither the request nor `can_act`, and only naming the email decided it.
+
+Every place the approvals service compares a slot with the caller now reads the caller's acting addresses, the set its decision routes already admitted: the user id, the email the caller's own account carries, and both spellings of each position on the caller's server-resolved context.
+
+- **Decisions** (approve, reject, send back, reassign, request info, comment): with no `actorId`, the caller takes the first pending slot keyed by one of those addresses, their user id first. A named `role:<name>` or `position:<name>` takes that position's slot under either spelling. Nobody new may decide: a user who holds another position is still refused with 403.
+- **What is recorded:** `sys_approval_action.actor_id` holds the slot the action took, in that slot's stored spelling. That is what naming the slot always recorded, and the multi-approver tally counts approvals by matching it against the slate.
+- **`viewer.can_act`** is computed by the same slot test the decision routes run with no `actorId`, so it is `true` exactly when such an approve would be admitted as a slot holder.
+- **Visibility:** the participant gate counts a current approver by the email half too. "Already acted" is counted by the same addresses, so a request decided under `position:<name>` stays visible to whoever holds that position.
+
+An admin who holds the routed position now decides it as a slot holder (`via_override: false`, one vote in a multi-approver tally), exactly as when they named the slot; an admin who holds no slot is unchanged.