Merge commit '31ed067639' into claude/issue-20594-client-dead-citations

Author: hotlong

.changeset/20596-service-datasource-provenance-anchors.md

@@ -0,0 +1,10 @@
+---
+'@objectstack/service-datasource': patch
+---
+
+Provenance comments in `service-datasource` were re-anchored
+
+Comment and docblock lines under `src/` that cited tracker numbers which no
+longer resolve on GitHub now cite the commit in this repository's history that
+decided the matter, and say in their own words what was decided. Comments
+only: no type, schema, export, log or refusal text, or runtime behaviour changes.

.changeset/20611-gate-reads-redaction-context.md

@@ -0,0 +1,36 @@
+---
+'@objectstack/lint': minor
+'@objectstack/metadata-protocol': minor
+---
+
+The runtime metadata publish gate refuses an `api` flow with no per-flow secret, and reads a secret the flow read path withheld as present (#20611).
+
+Clause-②: yes (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) Nothing authorable changes spelling or type: `packages/spec` is untouched, and the start node `config` stays the open record it was. What changes is that the runtime metadata write door now refuses one authored shape at publish: an `api`-bound flow whose start node carries no usable `config.secret`. `objectstack migrate meta` could not rewrite that shape even in principle, because the missing value is a shared secret only the author and the sending system can supply. Rows at rest are not judged or rewritten; the automation engine has refused to register such a flow since 17.5.0, and that load path's disposition is recorded in its own published changelog entry. -->
+
+**BREAKING** — an accept-set narrowing on the runtime metadata write door,
+shipped as `minor` under the launch-window convention (`check-changeset-no-major`
+refuses `major` until GA; breaking-ness is carried by this banner and the ADR-0087
+disposition above, not by the level). An `active` save through `/meta` of an
+`api`-bound flow whose start node carries no usable `config.secret` (a
+`PUT /api/v1/meta/flow/:name`, or the publish of such a draft) used to be stored;
+the automation engine then refused to register it (`400` on the `/automation`
+doors, a skip with a warning at boot). It is now refused at the save with
+`422 INVALID_METADATA`, the issue naming `flow-api-trigger-secret-missing` at the
+start node's `config.secret`, and nothing is stored. A draft save is still
+accepted; its publish is refused the same way.
+**One-line fix:** set a non-blank `config.secret` on the flow's start node — or,
+for a flow that is only ever started explicitly, declare `type: 'autolaunched'`
+with no `triggerType: 'api'`.
+
+**What does not change: a signed flow's round trip.** Every served flow definition withholds the start node's `config.secret`, so a body saved back after a read arrives without it, and the save restores the stored secret only after every gate has run, so that no gate handles a restored credential. The gate is now told WHERE the save will restore a credential from the stored row: those positions only, never the values. `flow-api-trigger-secret-missing` reads a secret that was withheld and is stored as present, and one that is absent and not stored as missing. So a GET → edit → PUT of a signed flow, and the first save of a code-authored flow whose secret is in the app's source, keep passing and keep their secret. An explicit empty `config.secret` is the author's own value and is refused as blank.
+
+`@objectstack/lint`:
+
+- `validateFlowApiTriggerSecret` now runs on the runtime publish gate too (`surfaces` `['cli', 'runtime-publish']`, `runtimeTypes: ['flow']`). Its `surfaceReason` is gone.
+- `AuthoringRuleContext` gains an optional `restoredCredentialPaths`: a `ReadonlySet<string>` of stack-relative positions in the rules' own finding-path spelling (`flows[0].nodes[1].config.secret`). Only the runtime publish gate sets it; `runAuthoringRules` never forwards it, so `os validate`, `os build` and `os lint` judge the author's own values as before.
+- `runRuntimeAuthoringRules` accepts an optional `restoredCredentialPaths`: item-relative dotted positions in the `@objectstack/spec/kernel` redactor registry's `redactedKeys` spelling (`nodes.1.config.secret`). The gate re-spells them against the written item's place in its snapshot.
+- `validateFlowApiTriggerSecret(stack, options?)` accepts an optional `{ restoredCredentialPaths }`, and treats a listed start-node secret position as present.
+
+`@objectstack/metadata-protocol`: `saveMetaItem` hands the runtime authoring gate the positions its own credential carry-forward will fill, computed from the same stored body. This costs one indexed `sys_metadata` read on an `active` save of a type with a registered redactor (`datasource`, and `flow` where the automation plugin registers one), and nothing for any other type or for a draft save. The carry-forward itself is unchanged and still runs after every gate. The draft→active promotion judges the stored draft row, which already holds what that draft's save carried forward, so it needs no such positions.

packages/lint/src/authoring-rules.ts

@@ -337,6 +337,28 @@ export interface AuthoringRuleContext {
    * this input or does not judge.
    */
   judgeFilter?: IObjectQLEngine['judgeFilter'];
+  /**
+   * [#20611] The credential positions of the WRITTEN item that the write path
+   * restores from the stored row before it persists the item — stack-relative,
+   * in the rules' own finding-path spelling (`flows[0].nodes[1].config.secret`).
+   *
+   * Set only by the runtime publish gate (`runtime-gate.ts`), from what its host
+   * states; ABSENT on the three CLI commands, whose stacks carry the author's
+   * own values. It exists because the read path withholds a credential from
+   * every served definition (a flow's start-node `config.secret`), so a body
+   * saved back after a read arrives WITHOUT it, and the host's carry-forward
+   * puts the stored value back only after every gate has run — on purpose, so
+   * that no gate handles a restored credential. A rule therefore cannot tell a
+   * withheld credential from a missing one by reading the body, and this set is
+   * how it is told: a path listed here is WITHHELD AND STORED, and a rule
+   * judging whether a credential is present reads it as present. A path not
+   * listed is judged on the body as sent, so a credential that is absent and
+   * not stored is missing.
+   *
+   * ⛔ Positions only, never values: the gate never sees a restored credential.
+   * Read by `validateFlowApiTriggerSecret` only.
+   */
+  restoredCredentialPaths?: ReadonlySet<string>;
 }
 
 export interface AuthoringRule {
@@ -1113,7 +1135,8 @@ export const AUTHORING_RULES: readonly AuthoringRule[] = [
   // routes nowhere, silently demoting it to a manual flow. #20553 made it five: an
   // `api`-bound flow with no usable `config.secret`, which the engine's own
   // `registerFlow` refuses (ADR-0041) — on its OWN entry below
-  // (`validateFlowApiTriggerSecret`), because it is CLI-only for now. None of those verdicts
+  // (`validateFlowApiTriggerSecret`), because it reads a context input this entry
+  // has no use for (#20611). None of those verdicts
   // can be changed by installing a package, so there is no reading under which
   // the flow fires. `flow-trigger-unknown-object` deliberately stayed `warning`
   // (the object may come from another installed package — a hedge this rule
@@ -1144,33 +1167,32 @@ export const AUTHORING_RULES: readonly AuthoringRule[] = [
     run: (stack) => validateFlowTriggerReadiness(stack),
   },
   // #20553 — `flow-api-trigger-secret-missing`, split out of the entry above as
-  // its own exported rule (the `validateSecurityRoleWord` precedent: one rule id
-  // sits on ONE side of the runtime wall). Same family, same `error`, all three
-  // commands — but NOT the runtime publish gate yet, and #20611 is the card that
-  // moves it across. The flow read path withholds `config.secret` from every
-  // served definition (#20552) and `saveMetaItem` restores the stored secret only
-  // just before the put, AFTER this table has judged the body the caller sent —
-  // so on the gate, a signed flow's ordinary GET → edit → PUT reads as
-  // secretless. Measured on `825c33ff9f`: with this id on the gate, the two
-  // round-trip pins in `protocol.metadata-redaction.test.ts` fail; off it, 26/26.
-  // The `/meta` door therefore keeps its pre-rule behaviour (it stores a
-  // secretless flow, and the engine refuses it at registration) until the gate
-  // judges the carried-forward body — then this entry becomes `CLI_AND_RUNTIME`
-  // with `runtimeTypes: ['flow']`, like the one above.
+  // its own exported rule. Same family, same `error`, all three commands.
+  //
+  // #20611 — and the runtime publish gate too. The flow read path withholds
+  // `config.secret` from every served definition (#20552), and `saveMetaItem`
+  // restores the stored secret only just before the put, AFTER this table has
+  // judged the body the caller sent — deliberately, so no gate handles a
+  // restored credential. So the gate is handed the POSITIONS that restore will
+  // fill (`AuthoringRuleContext.restoredCredentialPaths`), and the rule reads a
+  // withheld-and-stored secret as present: a signed flow's ordinary
+  // GET → edit → PUT passes, and a secretless `api` flow is refused at `/meta`
+  // with this id instead of being stored for the engine to refuse at
+  // registration.
   {
     name: 'validateFlowApiTriggerSecret',
     tier: 'gating',
     input: 'normalized',
     commands: ALL,
     source: 'packages/lint/src/validate-flow-trigger-readiness.ts',
-    surfaces: CLI_ONLY,
-    surfaceReason:
-      'Not yet runtime-safe: the publish gate judges a /meta save BEFORE saveMetaItem restores the ' +
-      'inbound-hook secret the flow read path withholds, so a signed api flow\'s ordinary GET, edit, PUT ' +
-      'round trip reaches this rule secretless and would be refused. It crosses when the gate judges the ' +
-      'carried-forward body (the seam follow-up named in the comment above); until then the engine\'s ' +
-      'registerFlow refusal is what a secretless flow saved through /meta meets.',
-    run: (stack) => validateFlowApiTriggerSecret(stack),
+    // Runtime publish gate (#20611): judged on the per-write snapshot like the
+    // entry above, with the host's restored-credential positions read as
+    // present. The CLI never sets that input, so there the author's own
+    // `config.secret` is the whole answer.
+    surfaces: CLI_AND_RUNTIME,
+    runtimeTypes: ['flow'],
+    run: (stack, ctx) =>
+      validateFlowApiTriggerSecret(stack, { restoredCredentialPaths: ctx.restoredCredentialPaths }),
   },
   // ADR-0090 D3 fallout — an approval `{ type: 'role' }` resolves against the
   // better-auth org-membership tier, not positions, so a position name authored

packages/lint/src/runtime-gate.ts

@@ -832,6 +832,49 @@ export function nameKeyFindingPath(path: string, candidate: AnyRec): string {
   return `${stackKey}.${name}${rest}`;
 }
 
+/**
+ * [#20611] The host's restored-credential positions, re-spelled from the
+ * redactor registry's item-relative dotted form (`nodes.1.config.secret`) into
+ * the finding-path form the rules emit and compare against, anchored at the
+ * written item's place in `candidate` (`flows[0].nodes[1].config.secret`).
+ *
+ * The written item is the LAST member of its collection in the candidate —
+ * {@link buildRuntimeWriteSnapshots} appends it — so that index is the anchor.
+ * Each segment is spelled `[n]` exactly where the item holds an array at that
+ * point of the walk and `.key` everywhere else: read off the item, never
+ * guessed from the segment's digits, so a record key that happens to be
+ * numeric keeps its key spelling. A path that walks off the item keeps the key
+ * spelling from there on; it names no position any rule reports, so it
+ * excuses nothing.
+ */
+function restoredCredentialStackPaths(
+  candidate: AnyRec,
+  type: string,
+  dottedPaths: readonly string[],
+): ReadonlySet<string> {
+  const stackKey = stackKeyForType(type);
+  const collection = stackKey ? candidate[stackKey] : undefined;
+  if (!stackKey || !Array.isArray(collection) || collection.length === 0) return new Set();
+  const index = collection.length - 1;
+  const item: unknown = collection[index];
+  const out = new Set<string>();
+  for (const dotted of dottedPaths) {
+    let path = `${stackKey}[${index}]`;
+    let node: unknown = item;
+    for (const segment of dotted.split('.')) {
+      if (Array.isArray(node) && /^(0|[1-9][0-9]*)$/.test(segment)) {
+        path += `[${segment}]`;
+        node = node[Number(segment)];
+      } else {
+        path += `.${segment}`;
+        node = node !== null && typeof node === 'object' ? (node as AnyRec)[segment] : undefined;
+      }
+    }
+    out.add(path);
+  }
+  return out;
+}
+
 function runRules(
   rules: readonly AuthoringRule[],
   stack: AnyRec,
@@ -894,6 +937,20 @@ export function runRuntimeAuthoringRules(args: {
    * it skip the engine's judgement and answer as they did without it.
    */
   judgeFilter?: IObjectQLEngine['judgeFilter'];
+  /**
+   * [#20611] The positions in `item` at which the host's write path restores a
+   * credential from the stored row before it persists the item — dotted and
+   * item-relative, the `@objectstack/spec/kernel` redactor registry's
+   * `redactedKeys` spelling (`nodes.1.config.secret`). The read path withholds
+   * those credentials, so a body saved back after a read arrives without them,
+   * and the host restores them only after this gate has run.
+   *
+   * Handed to the rules as `AuthoringRuleContext.restoredCredentialPaths`,
+   * translated into the candidate snapshot's finding-path spelling; see
+   * {@link restoredCredentialStackPaths}. Omitted, every position is judged on
+   * the body as sent. ⛔ Positions only: no credential value reaches this gate.
+   */
+  restoredCredentialPaths?: readonly string[];
 }): RuntimeGateResult {
   const rules = runtimeAuthoringRulesFor(args.type);
   const empty: RuntimeGateResult = { errors: [], advisories: [], rulesRun: [] };
@@ -921,6 +978,19 @@ export function runRuntimeAuthoringRules(args: {
     sduiManifest: args.sduiManifest,
     runtimeWriteType: args.type,
     judgeFilter: args.judgeFilter,
+    // [#20611] Spelled against the CANDIDATE, the one snapshot that holds the
+    // written item. The baseline pass shares the set and cannot match it: the
+    // item is not in the baseline, and every other entry sits at an index the
+    // item does not.
+    ...(args.restoredCredentialPaths !== undefined && args.restoredCredentialPaths.length > 0
+      ? {
+          restoredCredentialPaths: restoredCredentialStackPaths(
+            snapshots.candidate,
+            args.type,
+            args.restoredCredentialPaths,
+          ),
+        }
+      : {}),
   };
   const before = new Set(runRules(rules, snapshots.baseline, ctx).map(fingerprint));
   const added = runRules(rules, snapshots.candidate, ctx)