fix(metadata-protocol): one head row per /meta write address — a second unpinned package-less draft save is accepted (#22128)

.changeset/22128-draft-head-resolution.md

@@ -0,0 +1,17 @@
+---
+"@objectstack/metadata-protocol": minor
+"@objectstack/rest": patch
+"@objectstack/spec": patch
+---
+
+fix(meta): a second unpinned draft save of a package-owned item is no longer refused 409, the item read serves that draft's `version`, and a package-less draft save keeps a package-owned datasource's stored credential
+
+Clause-②: yes (widening)
+
+- **The second draft save is accepted.** Save an item into a package (`PUT /api/v1/meta/:type/:name?package=<id>`), then save a draft of it without `?package=` twice. The repository stores that draft in the item's package (the draft inheritance that keeps a pending change in the package it edits). The save door took its expected parent from the package-unbound draft row instead. It found none, so the second save, which sent no `If-Match`, was refused `409 METADATA_CONFLICT` ("Expected parent null but current is hmac-sha256:…"). ADR-0008 makes an unpinned save last-writer-wins, and it is accepted again.
+- **One head row per write address.** `SysMetadataRepository.headAt(ref, { state, packageId })` answers `{ head, packageId }`: the row a `put` at that address upserts (`null` for a create), and the package binding that put writes under. Both come from the same resolution `put`'s optimistic lock runs. That resolution includes the draft inheritance and the adoption of a draft stored without a package. The save door's expected parent and the item read's `version` both come from it, so the token `GET /meta/:type/:name?state=draft` serves is the token a draft save at the same address accepts. Before this, that read served `version: null` while the inherited draft existed.
+- **`?package=all` on the item read.** `GET /meta/:type/:name?package=all` now reads `all` as the save and publish doors read it: as naming no package. Before, the read forwarded the literal `all`. Its `version` was resolved at a package no save writes, so it served `null` beside a row the save then judged. Its `?state=draft` read answered `404 NO_DRAFT` for a draft stored in the item's package. The read now serves the item as a read with no `?package=` serves it, and `?package=all` still takes the uncached arm, so it keeps serving `version`.
+- **The credential carry-forward reads the same row.** A save of a type whose read withholds a credential (`datasource`; `flow` with the automation plugin) carries the stored credential back into a body that returns as it was served. It compared that body against the row at the key the caller named. A package-less draft save of a package-owned item therefore found neither the inherited draft nor the package's active row, and compared against the code layer. The stored credential was persisted away on the first such save, measured on the real route with a datasource whose `config.url` holds one. Both reads now go through `headAt`: the draft the save overwrites, or with no draft yet the active row in the package the draft is stamped into. An item with no package binding is unchanged.
+- **Descriptions only, in `@objectstack/spec`.** `SaveMetaItemRequestSchema.packageId` now states the draft inheritance, where it said "absent = env-local overlay". The `POST /meta/:type/:name/publish` route description spells the conflict code `METADATA_CONFLICT`, as the wire sends it.
+- A stale `If-Match` is still refused `409`, and `If-None-Match: *` is still refused while a row exists. An item with no package binding is unchanged. No spec key, request field or response field is added or removed.
+- The only change to the public surface is one added method on the exported `SysMetadataRepository`: `headAt`. No existing signature, schema or accept set changes.

content/docs/references/api/protocol.mdx

@@ -2601,7 +2601,7 @@ Installed package with runtime lifecycle state
 | **actor** | `string` | optional | Identity recorded on the write's history event (`recorded_by`, a lookup into `sys_user`) and audit row. On the REST door this is the request's authenticated identity (one producer) — never a caller-supplied header. Absent, the event is recorded actor-less (null), deliberately not attributed to "system". |
 | **force** | `boolean` | optional | Destructive-change acknowledgement (`?force=true` on the REST door): skips the safety diff that refuses an `object` save whose body drops fields or narrows types the stored item still carries (409 with the findings otherwise). Only `object` saves reach that diff, so the flag is inert for every other type. Absent = the guard runs. |
 | **mode** | `Enum<'draft' \| 'publish'>` | optional | Per-item lifecycle (ADR-0005 drafts): `draft` stages the body as a pending draft overlay (`?mode=draft` on the REST door; the publish door promotes it later); `publish` or ABSENT writes straight to the live `active` row — the legacy default, kept so callers that predate the draft/publish split keep working. Any value other than `draft` is read as `publish`. |
-| **packageId** | `string \| null` | optional | ADR-0048 — the software package to bind the saved row to (`sys_metadata.package_id`; `?package=<id>` on the REST door, sent only when it names a real package). Set when authoring inside a Studio package workspace; a named read-only base package is refused. On create the row is stamped with this id; on update an existing binding is preserved, never silently re-bound. Absent = env-local overlay (no package stamp); it also scopes which row the unpinned parent-version resolution reads. |
+| **packageId** | `string \| null` | optional | ADR-0048 — the software package to bind the saved row to (`sys_metadata.package_id`; `?package=<id>` on the REST door, where `all` and an empty value name no package). Set when authoring inside a Studio package workspace; a named read-only base package is refused. On create the row is stamped with this id; on update an existing binding is preserved, never silently re-bound. Absent or `null` names no package: an `active` save writes the env-local overlay (no package stamp), and a `draft` save writes the draft of the package the item's active row is bound to when it is package-bound (a pending change stays in the package it edits), else the env-local draft. That row is also the one the parent-version resolution reads: an unpinned save takes its version as the parent (last-write-wins), a pinned one is judged against it, and the item read at the same scope serves it as `version`. |
 | **writeFace** | `Enum<'package-duplicate' \| 'meta-envelope' \| 'meta-dispatch' \| 'external-import'>` | optional | Which write door a refusal is being rendered FOR — stated by the SERVER, never by a remote caller: every door builds this request field by field and never spreads a request body into it, so there is no path for a client to smuggle a face in, and a face arriving in a wire body is simply never read. Two refusals branch on it, for different questions: the 409 destructive-change remedy names the acknowledgement mechanism that actually exists on the refusing door (`?force=true` on the REST doors; the dispatcher, the duplicate-package door and the external-table import have none), and the 422 invalid-metadata message adapts to whether a structured `issues[]` channel reaches the consumer beside it. Absent = the conservative default wording. |
 
 

packages/metadata-protocol/src/protocol.ts

@@ -8586,12 +8586,20 @@ export class ObjectStackProtocolImplementation implements
      * back. Here the stakes are higher than a phantom customization — the
      * subtracted material is a credential nobody can retype from the wire.
      *
-     * Reads the row at rest through the overlay repository, whose `get()` is
-     * documented VERBATIM (no ADR-0087 conversion): the comparison must be
+     * Reads the row at rest through the overlay repository, whose rows are
+     * served VERBATIM (no ADR-0087 conversion): the comparison must be
      * against the bytes that were written, because those are the bytes the read
      * exit redacted. {@link carryForwardRedactedValues} then decides per
      * redacted path — see its docblock for the three outcomes.
      *
+     * [#22128] The row is the one THIS save overwrites, from the repository's
+     * write-address resolution ({@link SysMetadataRepository.headAt}) — ⛔ never
+     * a read at the key the caller named. A draft save that names no package is
+     * stored in the package of the item's active row (#11087); read at the
+     * package-unbound key, neither that draft nor that active row was found,
+     * the comparison fell to the code layer, and the first package-less draft
+     * save of a package-owned datasource persisted its stored credential away.
+     *
      * ⚠️ THE DRAFT FALLBACK IS LOAD-BEARING, not defensive. A `?mode=draft`
      * save of an item with no draft row yet has nothing at its own state to
      * compare against, while the body the author edited came from the ACTIVE
@@ -8621,8 +8629,9 @@ export class ObjectStackProtocolImplementation implements
     }
 
     /**
-     * The body a carry-forward compares against: the overlay row at the write's
-     * own state, the ACTIVE row for a draft save with no draft row yet (see
+     * The body a carry-forward compares against: the row the write overwrites
+     * at its own state ({@link SysMetadataRepository.headAt}), the ACTIVE row
+     * the draft overlays for a draft save with no draft row yet (see
      * {@link carryForwardRedactedCredentials} for why that fallback is
      * load-bearing), else the CODE layer. RAW in every case — the bytes the
      * read exits redacted — and read with no `try`/`catch`, for the reason the
@@ -8635,15 +8644,19 @@ export class ObjectStackProtocolImplementation implements
         state: 'draft' | 'active';
         packageId: string | null;
     }): Promise<unknown> {
-        let stored = await args.repo.get(args.ref, {
+        // [#22128] The row this save overwrites, and with no draft row yet the
+        // active row the draft overlays: the one in the package the draft is
+        // stamped into (`packageId`, the inherited binding included).
+        const target = await args.repo.headAt(args.ref, {
             state: args.state,
             packageId: args.packageId,
         });
+        let stored = target.head;
         if (!stored && args.state === 'draft') {
-            stored = await args.repo.get(args.ref, {
+            stored = (await args.repo.headAt(args.ref, {
                 state: 'active',
-                packageId: args.packageId,
-            });
+                packageId: target.packageId,
+            })).head;
         }
         // [#20552] NO overlay row at either state: the body the read served is
         // the CODE layer, so that is what the incoming body is compared with.
@@ -18039,24 +18052,33 @@ export class ObjectStackProtocolImplementation implements
     /**
      * [#22114] The stored head a `/meta` save at one address compares a
      * caller's version token against: the `state` row at `ref`'s organization
-     * partition, bound to `packageId` (`null` = the package-unbound row), as
-     * the repository reads and serves it ({@link SysMetadataRepository.get}).
-     * `null` when no such row exists.
+     * partition that a save naming `packageId` (`null` = no package) upserts,
+     * as the repository serves it. `null` when that save would create.
      *
      * The ONE head read both halves of the ADR-0008 chain make. The save door
-     * judges an inbound token against it ({@link storedParentForToken}), and
-     * the item read serves its keyed form as `version`
-     * ({@link readVersionToken}). So the token a read hands out is the token a
-     * save at the same address accepts, and the one a receipt serves for that
-     * row — ⛔ never a second derivation of either.
+     * takes its expected parent from it — the inbound token judged against it
+     * ({@link storedParentForToken}), or, unpinned, the head itself — and the
+     * item read serves its keyed form as `version` ({@link readVersionToken}).
+     * So the token a read hands out is the token a save at the same address
+     * accepts, and the one a receipt serves for that row — ⛔ never a second
+     * derivation of either.
+     *
+     * [#22128] Resolved by the repository's own write-address resolution
+     * ({@link SysMetadataRepository.headAt}), the one its lock judges with —
+     * ⛔ never {@link SysMetadataRepository.get} at the named key. A draft
+     * save that names no package is stored in the package of the item's
+     * active row (#11087), so the row at the package-unbound key is not the
+     * row that save overwrites: read there, the parent of an existing draft
+     * was `null`, the repository's lock refused the unpinned save 409, and the
+     * `?state=draft` read served `version: null` while the draft existed.
      */
     private async storedHeadAt(
         repo: SysMetadataRepository,
-        ref: Parameters<SysMetadataRepository['get']>[0],
+        ref: Parameters<SysMetadataRepository['headAt']>[0],
         state: 'active' | 'draft',
         packageId: string | null,
     ): Promise<string | null> {
-        return (await repo.get(ref, { state, packageId }))?.hash ?? null;
+        return (await repo.headAt(ref, { state, packageId })).head?.hash ?? null;
     }
 
     /**
@@ -18075,7 +18097,11 @@ export class ObjectStackProtocolImplementation implements
      * and from a package's own row to the package-less one (ADR-0048); a save
      * does not, it writes its own partition. A token of a row the save would
      * not overwrite would be refused by that save every time, so such a read
-     * serves `null` rather than the served row's token.
+     * serves `null` rather than the served row's token. [#22128] And the
+     * other way round: a `?state=draft` read that names no package serves the
+     * token of the draft a package-less draft save overwrites, which is the
+     * draft bound to the package of the item's active row when there is one
+     * (#11087) — the address the save resolves, not the key it was handed.
      */
     private async readVersionToken(
         request: { type: string; name: string; packageId?: string },
@@ -18088,7 +18114,7 @@ export class ObjectStackProtocolImplementation implements
             type: PLURAL_TO_SINGULAR[request.type] ?? request.type,
             name: request.name,
             org: organizationId ?? 'env',
-        } as Parameters<typeof repo.get>[0];
+        } as Parameters<typeof repo.headAt>[0];
         const stored = await this.storedHeadAt(repo, ref, state, request.packageId ?? null);
         return stored === null ? null : this.receiptVersion(stored);
     }
@@ -20932,7 +20958,9 @@ export class ObjectStackProtocolImplementation implements
             // package the upsert targets so a collision's other-package
             // row is never read as this item's parent.
             // [#22114] The head the item read's `version` is the keyed form of
-            // — one read for both halves ({@link storedHeadAt}).
+            // — one read for both halves ({@link storedHeadAt}). [#22128] The
+            // row the repository's `put` resolves for this address and locks
+            // against, the inherited draft binding included.
             const currentStored = await this.storedHeadAt(
                 repo, ref, mode === 'draft' ? 'draft' : 'active', request.packageId ?? null,
             );