diff --git a/.changeset/21934-publish-lock-resolved-package.md b/.changeset/21934-publish-lock-resolved-package.md new file mode 100644 index 00000000000..07d2a66b56b --- /dev/null +++ b/.changeset/21934-publish-lock-resolved-package.md @@ -0,0 +1,8 @@ +--- +'@objectstack/metadata-protocol': patch +--- + +A metadata publish consults the item lock at the package key it resolved + +- A publish that states no package promotes the draft row it resolves, under that row's own package. Its ADR-0010 lock lookup now uses that same resolved key (the stated package, else the draft row's own), the key the gate reads the draft under and the promotion writes under, instead of only the package the request stated. Where several packages' rows declare the strictest lock, the refusal now carries the lock of the package whose draft is being promoted. +- The authoring gate's package narrowing is unchanged: it still uses only the package the caller stated. diff --git a/.changeset/21934-publish-promotes-judged-draft.md b/.changeset/21934-publish-promotes-judged-draft.md new file mode 100644 index 00000000000..f7e656bc267 --- /dev/null +++ b/.changeset/21934-publish-promotes-judged-draft.md @@ -0,0 +1,10 @@ +--- +'@objectstack/metadata-protocol': minor +--- + +A metadata publish promotes only the draft its gate judged + +Clause-②: yes (widening) + +- A publish (`publishMetaItem`, and each promotion of `publishPackageDrafts`) reads the draft to judge it and then promotes the draft row. The promotion is now handed the judged draft's hash. A draft saved after the judgement, or a draft that appears where the judgement found none, is refused with `409 METADATA_CONFLICT`, and nothing is published. Publishing again judges and promotes the current draft. +- `SysMetadataRepository.promoteDraft` takes a new optional `expectedDraftHash` (`string | null`). When it is stated, the draft row the promotion reads must carry that hash (with `null`, no draft row may exist); otherwise the promotion throws a `ConflictError` before anything is written. When it is omitted, the promotion behaves as before. diff --git a/.changeset/21934-save-check-anchor-per-package.md b/.changeset/21934-save-check-anchor-per-package.md new file mode 100644 index 00000000000..3f8a55bf117 --- /dev/null +++ b/.changeset/21934-save-check-anchor-per-package.md @@ -0,0 +1,8 @@ +--- +'@objectstack/metadata-protocol': patch +--- + +The organization-scoped save check judges a view overlay against every package's environment-wide definition of its row + +- An organization-scoped `view` save or publish in the organization the anonymous form endpoints read is refused when it would leave open a form the environment-wide definition withdraws. Its row anchor is now resolved per package, the way the list read resolves each package's item: each package's own environment-wide row, else the package-less environment-wide row (which stands in for every package), else that package's artifact. Before, with no environment-wide row stored, it judged only the first package's artifact in registry order, and a stored row of any one package hid every package's artifact of the name. +- The known limit stated with the public-form withdrawal ("it may over-close, never under-close") is narrowed. A withdrawal of a view name still closes that name in every package, so it may over-close. The organization-scoped save check judges every package's environment-wide definition of the name. The anonymous endpoints do too, with one exception: where a package's environment-wide copy of a view container is saved, the endpoints read that copy's expansion alone for each form it expands, and can miss another package's withdrawal of that form, whether saved or shipped, until the form is withdrawn in every saved environment-wide copy of that container as well. Reading each package's expansion separately is tracked in #21967. diff --git a/.changeset/21934-view-list-one-item-per-package.md b/.changeset/21934-view-list-one-item-per-package.md new file mode 100644 index 00000000000..1d41810b0b0 --- /dev/null +++ b/.changeset/21934-view-list-one-item-per-package.md @@ -0,0 +1,8 @@ +--- +'@objectstack/metadata-protocol': patch +--- + +The view list serves one item per package for a view name that several installed packages ship, whether or not a view row is stored + +- Where two installed packages ship a view of the same name, the list read (`getMetaItems` for `view`) now serves each package's item of that name, as it already did while no view row was stored. Only a name that a stored view container's expansion writes is upserted by name. +- The environment-wide view list is the layer the anonymous form endpoints judge a withdrawal against. A package-less organization copy of the view, stored before one package withdrew the form, is now judged against every package's body of the name there, so it stays closed whichever package withdraws. diff --git a/content/docs/ui/public-data-collection.mdx b/content/docs/ui/public-data-collection.mdx index 206108708ef..46b3c624937 100644 --- a/content/docs/ui/public-data-collection.mdx +++ b/content/docs/ui/public-data-collection.mdx @@ -62,13 +62,13 @@ A withdrawal is a kill switch across metadata layers. If the environment-wide de **Which form a withdrawal closes.** Two checks apply the rule, and they match forms differently: -- **Saving and publishing in an organization** judges the organization's copy against the stored environment-wide definition it overrides (the row its copy is keyed by). Inside that definition, a withdrawn form is the same form as the organization's when they share a place (`form`, the same `formViews` entry, or the view's own `config`) or a public link. A match on either is enough, so a renamed `formViews` key, a `form.name`, a move to another place, a renamed expanded item, and a new or re-cased slug all still count as the same form. A form that differs from every withdrawn form in both place and link is a different form, such as a sibling in the same view. +- **Saving and publishing in an organization** judges the organization's copy against the stored environment-wide definition it overrides (the row its copy is keyed by, in each package that ships it). Inside that definition, a withdrawn form is the same form as the organization's when they share a place (`form`, the same `formViews` entry, or the view's own `config`) or a public link. A match on either is enough, so a renamed `formViews` key, a `form.name`, a move to another place, a renamed expanded item, and a new or re-cased slug all still count as the same form. A form that differs from every withdrawn form in both place and link is a different form, such as a sibling in the same view. - **The anonymous endpoints** judge each form by the name of the view item they serve. Beneath the organization's read they read the environment-wide view list, and a form is closed when the environment-wide item of the same name explicitly withdraws a form in the same place or under the same link. - **A different view is a different form.** A different view that uses the same public link (for example, another app's "contact us" form) neither closes this one nor is closed by it. **Forms a package ships.** A package's form is part of the environment-wide definition, not a separate layer beneath it. A definition parsed by the stack schema (strict `defineStack`, the default) gets the schema's default `enabled: false`, so a shipped form that keeps its link without setting `enabled: true` counts as withdrawn and an organization's copy cannot open it. A definition loaded without that parse (`defineStack(..., { strict: false })` or a hand-built manifest) is judged as written: a switch it leaves out is absent, which is not a withdrawal, so set `enabled: false` explicitly to ship a form closed. The environment-wide definition is the administrator's switch: an environment-wide save may open a form that the package ships closed. -**Known limit: packages and names.** A withdrawal of a view name closes that name in every package. When two packages each ship a view of the same name, one package's withdrawal also closes the other package's form of that name. This may close more than was meant, but it never leaves a withdrawn form open. Per-package precision is tracked in #21934. +**Known limit: packages and names.** A withdrawal of a view name closes that name in every package. When two packages each ship a view of the same name, one package's withdrawal also closes the other package's form of that name, so this may close more than was meant. The organization-scoped save check judges every package's environment-wide definition of the name. The endpoints do too, with one exception: where a package's environment-wide copy of a view container is saved, the endpoints read that copy's expansion alone for each form it expands, and can miss another package's withdrawal of that form, whether saved or shipped. To close such a form at the endpoints, withdraw it in every saved environment-wide copy of that container as well. Reading each package's expansion separately is tracked in #21967. **Known limit.** The save check runs only when an organization's copy is saved or published. A copy that was already stored before the environment-wide withdrawal, or that a rollback or revert restores, is judged only by the endpoints, which match by served item name. If that copy keeps the form open under a different key or place than the environment-wide definition, the endpoints can still serve it. To close it, withdraw the form in that organization's copy too; the next organization-scoped save of a copy that keeps it open is refused. diff --git a/packages/metadata-core/src/anonymous-form-intake.ts b/packages/metadata-core/src/anonymous-form-intake.ts index 4252000529a..120014958ba 100644 --- a/packages/metadata-core/src/anonymous-form-intake.ts +++ b/packages/metadata-core/src/anonymous-form-intake.ts @@ -322,9 +322,16 @@ function anonymousFormExplicitWithdrawals(view: unknown): Array<{ slot: string; * and closes nothing. The package a body is bound to is NOT compared: a * withdrawal of a name closes that name's form in every package (a known * limit that fails closed: it may over-close another package's form of the - * same name, never under-close). A layer with no body of the row, or whose body has no - * explicit withdrawal, withdraws nothing, so a form published only in an - * organization stays open there. + * same name). It judges only the bodies the layer holds, so a caller closes a + * name in every package only when its layer holds every package's body of the + * name. The organization-scoped write door anchors one body per package. The + * env-wide view list the anonymous doors read holds one item per package of a + * name, with one exception: where a package's env-wide copy of a view + * container is saved, the list holds that copy's expansion alone for each form + * it expands, so the doors can miss another package's withdrawal of that + * form, whether saved or shipped (per-package expansion is #21967). A layer + * with no body of the row, or whose body has no explicit withdrawal, withdraws + * nothing, so a form published only in an organization stays open there. */ export function anonymousFormIntakeWithdrawnIn( layer: ReadonlyArray, diff --git a/packages/metadata-protocol/src/protocol.org-scoped-write-refused.test.ts b/packages/metadata-protocol/src/protocol.org-scoped-write-refused.test.ts index c8758d8a511..3823b386eca 100644 --- a/packages/metadata-protocol/src/protocol.org-scoped-write-refused.test.ts +++ b/packages/metadata-protocol/src/protocol.org-scoped-write-refused.test.ts @@ -74,6 +74,9 @@ import { afterEach, describe, expect, it } from 'vitest'; // `@objectstack/objectql`: objectql DEPENDS ON this package, so that import // would close a dependency cycle turbo rejects outright. import { assertEngineDeleteDispatch, assertEngineUpdateDispatch, assertEngineFindOnePredicate } from '@objectstack/metadata-core'; +// The anonymous form doors' own verdict (`registerFormEndpoints` in +// `@objectstack/rest`), applied here to the protocol's real list reads. +import { anonymousFormIntakeCandidates, anonymousFormIntakeWithdrawnIn } from '@objectstack/metadata-core'; import { DEFAULT_METADATA_TYPE_REGISTRY } from '@objectstack/spec/kernel'; import { SharingConfigSchema } from '@objectstack/spec/ui'; import { ObjectStackProtocolImplementation } from './protocol.js'; @@ -999,3 +1002,295 @@ describe('org-scoped anonymous form intake changes the anonymous doors cannot se }); }); }); + +// The package identity of a served organization overlay. A package-less +// organization overlay is served once per package that ships its name, and +// the list merge stamps each copy with that package. The anonymous doors and +// the organization-scoped save compare no package: a withdrawal of a view name +// closes that name in every package. So an overlay stored before one +// package's env-wide withdrawal stays closed whichever package's copy is +// served, and whichever package withdraws, first or second in registry order. +// The env-wide list the doors judge against holds every package's body of the +// name, the withdrawing package's included. +describe('a package-less organization overlay, two packages shipping its view name', () => { + const NAME = 'task.intake_form'; + const LINK = '/forms/shared-intake'; + const formView = (allowAnonymous: boolean, label = 'Intake') => ({ + name: NAME, + label, + object: 'task', + viewKind: 'form', + config: { sharing: { enabled: true, allowAnonymous, publicLink: LINK } }, + }); + const shippedA = { ...formView(true), _packageId: 'pkg_a' }; + const shippedB = { ...formView(true), _packageId: 'pkg_b' }; + + /** Package A first in registry order: a lookup naming no package answers A's artifact. */ + function makeTwoPackageProtocol() { + const { engine, rows } = makeStubEngine(); + engine.registry.listItems = (type: string) => (type === 'view' ? [shippedA, shippedB] : []); + engine.registry.getArtifactItem = (type: string, name: string, pkg?: string) => { + if (type !== 'view' || name !== NAME) return undefined; + return pkg === 'pkg_b' ? shippedB : shippedA; + }; + const services = new Map([['tenancy', { defaultOrgId: async () => 'org_a' }]]); + const protocol = new ObjectStackProtocolImplementation(engine, () => services, 'env_prod') as any; + return { protocol, rows }; + } + + /** The organization read's copies of the view, where the doors find the form. */ + async function servedCopies(protocol: any): Promise>> { + const org: any = await protocol.getMetaItems({ type: 'view', organizationId: 'org_a' }); + return (org.items as any[]).filter((v) => v?.name === NAME); + } + + /** + * The copies the anonymous doors would serve for the slug: open, and not + * withdrawn by the env-wide layer they read beneath the organization's. + */ + async function doorsServe(protocol: any): Promise>> { + const envWide: any = await protocol.getMetaItems({ type: 'view' }); + const layer: unknown[] = envWide.items; + return (await servedCopies(protocol)).filter((view) => anonymousFormIntakeCandidates(view) + .some((c) => c.slug === 'shared-intake' && !anonymousFormIntakeWithdrawnIn(layer, view, c))); + } + + /** The organization overlay, stored package-less while the form is open everywhere. */ + async function storeOpenOverlay(protocol: any, rows: Map) { + expect((await protocol.saveMetaItem({ + type: 'view', name: NAME, item: formView(true, 'Intake (org)'), organizationId: 'org_a', + })).success).toBe(true); + expect(Array.from(rows.values()).filter((r) => r.organization_id === 'org_a').map((r) => r.package_id)) + .toEqual([null]); + } + + it('the organization read serves the overlay once per package, each copy stamped with that package', async () => { + const { protocol, rows } = makeTwoPackageProtocol(); + await storeOpenOverlay(protocol, rows); + const copies = await servedCopies(protocol); + expect(copies.map((v) => v.label)).toEqual(['Intake (org)', 'Intake (org)']); + expect(copies.map((v) => v._packageId).sort()).toEqual(['pkg_a', 'pkg_b']); + // Control: before any withdrawal the doors serve the overlay. + expect(await doorsServe(protocol)).toHaveLength(2); + }); + + for (const [order, withdrawing] of [['first', 'pkg_a'], ['second', 'pkg_b']] as const) { + describe(`the package ${order} in registry order withdraws the name env-wide`, () => { + async function withdrawAfterOverlay() { + const { protocol, rows } = makeTwoPackageProtocol(); + await storeOpenOverlay(protocol, rows); + expect((await protocol.saveMetaItem({ + type: 'view', name: NAME, item: formView(false), packageId: withdrawing, + })).success).toBe(true); + return { protocol, rows }; + } + + it('the env-wide list the doors read holds the withdrawal beside the other package\'s body', async () => { + const { protocol } = await withdrawAfterOverlay(); + const envWide: any = await protocol.getMetaItems({ type: 'view' }); + const named = (envWide.items as any[]).filter((v) => v?.name === NAME) + .map((v) => [v._packageId, v.config.sharing.allowAnonymous]) + .sort(); + expect(named).toEqual([['pkg_a', withdrawing !== 'pkg_a'], ['pkg_b', withdrawing !== 'pkg_b']]); + }); + + it('the doors serve no copy of the overlay stored before the withdrawal', async () => { + const { protocol } = await withdrawAfterOverlay(); + // Both stamped copies are still the open overlay, and both are closed. + expect((await servedCopies(protocol)).map((v) => v._packageId).sort()).toEqual(['pkg_a', 'pkg_b']); + expect(await doorsServe(protocol)).toEqual([]); + }); + + it('a re-save of the overlay is refused, and nothing changes', async () => { + const { protocol, rows } = await withdrawAfterOverlay(); + const before = Array.from(rows.values()).filter((r) => r.organization_id === 'org_a'); + await expect(protocol.saveMetaItem({ + type: 'view', name: NAME, item: formView(true, 'Intake (org, again)'), organizationId: 'org_a', + })).rejects.toMatchObject({ code: 'NOT_OVERRIDABLE', status: 403, organizationId: 'org_a' }); + expect(Array.from(rows.values()).filter((r) => r.organization_id === 'org_a')).toEqual(before); + }); + }); + } +}); + +// A publish promotes only the draft its gate judged. The gate reads the draft +// to judge it, and the promotion reads the draft row again to write it; the +// promotion is handed the judged draft's hash, so a draft saved between the +// two reads is refused as a conflict, never promoted unjudged. +describe('a publish promotes only the draft its gate judged', () => { + const judged = { ...VIEW, label: 'Org grid (judged)' }; + const later = { ...VIEW, label: 'Org grid (saved after the gate read)' }; + + /** A draft save that lands after the gate's read and before the promotion's read. */ + function saveBeforePromotion(protocol: any, body: unknown) { + const repo = protocol.getOverlayRepo(null); + const promote = repo.promoteDraft.bind(repo); + repo.promoteDraft = async (...args: unknown[]) => { + expect((await protocol.saveMetaItem({ type: 'view', name: VIEW.name, item: body, mode: 'draft' })).success) + .toBe(true); + return promote(...args); + }; + } + + const labelsIn = (rows: Map, state: string) => Array.from(rows.values()) + .filter((r) => r.type === 'view' && r.name === VIEW.name && r.state === state) + .map((r) => (typeof r.metadata === 'string' ? JSON.parse(r.metadata) : r.metadata).label); + + it('a draft saved after the gate read is not promoted, and the conflict answers', async () => { + const { protocol, rows } = makeProtocol(); + expect((await protocol.saveMetaItem({ type: 'view', name: VIEW.name, item: judged, mode: 'draft' })).success) + .toBe(true); + saveBeforePromotion(protocol, later); + + await expect(protocol.publishMetaItem({ type: 'view', name: VIEW.name })) + .rejects.toMatchObject({ code: 'METADATA_CONFLICT', status: 409 }); + expect(labelsIn(rows, 'active')).toEqual([]); + expect(labelsIn(rows, 'draft')).toEqual([later.label]); + }); + + it('a draft saved where the gate judged none is not promoted, and the conflict answers', async () => { + const { protocol, rows } = makeProtocol(); + saveBeforePromotion(protocol, later); + + await expect(protocol.publishMetaItem({ type: 'view', name: VIEW.name })) + .rejects.toMatchObject({ code: 'METADATA_CONFLICT', status: 409 }); + expect(labelsIn(rows, 'active')).toEqual([]); + expect(labelsIn(rows, 'draft')).toEqual([later.label]); + }); + + it('control: with no save in between, the judged draft is promoted and its draft row drained', async () => { + const { protocol, rows } = makeProtocol(); + expect((await protocol.saveMetaItem({ type: 'view', name: VIEW.name, item: judged, mode: 'draft' })).success) + .toBe(true); + + expect((await protocol.publishMetaItem({ type: 'view', name: VIEW.name })).success).toBe(true); + expect(labelsIn(rows, 'active')).toEqual([judged.label]); + expect(labelsIn(rows, 'draft')).toEqual([]); + }); +}); + +// The lock a publish consults is the one at the address it promotes: the +// package key the publish resolved (the caller's stated binding, else the +// draft row's own), not only the package the request stated. With two +// packages' env-wide rows of one view both locked, the refusal carries the +// lock of the package whose draft is being promoted. +describe('a publish consults the lock of the package key it resolved', () => { + const locked = (pkg: string) => ({ ...VIEW, _lock: 'full', _lockReason: `${pkg} keeps this view as shipped` }); + + async function seedLockedRowsAndDraft(draftPackage: string) { + const { protocol, rows } = makeProtocol(); + await protocol.ensureOverlayIndex(); + const repo = protocol.getOverlayRepo(null); + const ref = { type: 'view', name: VIEW.name, org: 'env' }; + const opts = { parentVersion: null, actor: null, source: 'test.seed', intent: 'runtime-only' as const }; + for (const pkg of ['pkg_a', 'pkg_b']) { + await repo.put(ref, locked(pkg), { ...opts, state: 'active', packageId: pkg }); + } + await repo.put(ref, { ...VIEW, label: 'Org grid (draft)' }, { ...opts, state: 'draft', packageId: draftPackage }); + const draftsOf = () => Array.from(rows.values()).filter((r) => r.state === 'draft').map((r) => r.package_id); + return { protocol, draftsOf }; + } + + it('a publish that states no package consults the lock of the draft row\'s own package', async () => { + const { protocol, draftsOf } = await seedLockedRowsAndDraft('pkg_b'); + await expect(protocol.publishMetaItem({ type: 'view', name: VIEW.name })) + .rejects.toMatchObject({ code: 'ITEM_LOCKED', status: 403, lock: 'full', lockReason: 'pkg_b keeps this view as shipped' }); + expect(draftsOf()).toEqual(['pkg_b']); + }); + + it('control: a publish that states its package consults that package\'s lock', async () => { + const { protocol, draftsOf } = await seedLockedRowsAndDraft('pkg_a'); + await expect(protocol.publishMetaItem({ type: 'view', name: VIEW.name, packageId: 'pkg_a' })) + .rejects.toMatchObject({ code: 'ITEM_LOCKED', status: 403, lock: 'full', lockReason: 'pkg_a keeps this view as shipped' }); + expect(draftsOf()).toEqual(['pkg_a']); + }); +}); + +// The organization-scoped save check anchors the overlay on the env-wide +// definition of its row, one per package that holds the name: each package's +// own env-wide row, else the package-less env-wide row (which stands in for +// every package), else that package's artifact. So a package that withdraws +// the form is judged whatever its place in registry order, and another +// package's stored row anchors that package only. +describe('the save check anchors each package\'s row on that package\'s env-wide definition', () => { + const LINK = '/forms/walled-intake'; + const open = { enabled: true, allowAnonymous: true, publicLink: LINK }; + const container = (sharing: Record, key = 'intake_form') => ({ + name: 'task', object: 'task', formViews: { [key]: { sharing } }, + }); + // Registry order: package A (open) first, so a lookup that names no + // package answers A's artifact. Package B, which withdraws, is second. + const shippedA = { ...container(open), _packageId: 'pkg_a' }; + const shippedB = { ...container({ ...open, allowAnonymous: false }), _packageId: 'pkg_b' }; + // The overlay moves the form to another key, so only the row anchor matches it. + const renamedOpen = container(open, 'intake_v2'); + + function makeTwoPackageProtocol() { + const { engine, rows } = makeStubEngine(); + engine.registry.listItems = (type: string) => (type === 'view' ? [shippedA, shippedB] : []); + engine.registry.getArtifactItem = (type: string, name: string, pkg?: string) => { + if (type !== 'view' || name !== 'task') return undefined; + return pkg === 'pkg_b' ? shippedB : shippedA; + }; + const services = new Map([['tenancy', { defaultOrgId: async () => 'org_a' }]]); + const protocol = new ObjectStackProtocolImplementation(engine, () => services, 'env_prod') as any; + return { protocol, rows }; + } + + for (const [label, packageId] of [['a package-less', undefined], ['a package A-bound', 'pkg_a']] as const) { + it(`the withdrawing package is not first in registry order: ${label} org save that renames the form is refused`, async () => { + const { protocol, rows } = makeTwoPackageProtocol(); + await expect(protocol.saveMetaItem({ + type: 'view', name: 'task', item: renamedOpen, organizationId: 'org_a', + ...(packageId ? { packageId } : {}), + })).rejects.toMatchObject({ code: 'NOT_OVERRIDABLE', status: 403, organizationId: 'org_a' }); + expect(orgRows(rows).filter((r) => r.org === 'org_a')).toEqual([]); + }); + } + + it('another package\'s env-wide row anchors that package only: the org save is still judged against the withdrawing package', async () => { + const { protocol, rows } = makeTwoPackageProtocol(); + expect((await protocol.saveMetaItem({ + type: 'view', name: 'task', item: { ...container(open), label: 'Task (A, env-wide)' }, packageId: 'pkg_a', + })).success).toBe(true); + await expect(protocol.saveMetaItem({ + type: 'view', name: 'task', item: renamedOpen, organizationId: 'org_a', + })).rejects.toMatchObject({ code: 'NOT_OVERRIDABLE', status: 403, organizationId: 'org_a' }); + expect(orgRows(rows).filter((r) => r.org === 'org_a')).toEqual([]); + }); + + it('control: the same org save that keeps the form withdrawn saves', async () => { + const { protocol } = makeTwoPackageProtocol(); + expect((await protocol.saveMetaItem({ + type: 'view', name: 'task', item: container({ ...open, allowAnonymous: false }, 'intake_v2'), organizationId: 'org_a', + })).success).toBe(true); + }); + + it('control: a package-less env-wide row stands in for every package, so the org save it leaves open saves', async () => { + const { protocol } = makeTwoPackageProtocol(); + expect((await protocol.saveMetaItem({ type: 'view', name: 'task', item: container(open) })).success).toBe(true); + expect((await protocol.saveMetaItem({ + type: 'view', name: 'task', item: renamedOpen, organizationId: 'org_a', + })).success).toBe(true); + }); +}); + +// The draft-key read runs ahead of the lock check, so an unreadable store is +// answered there as the lock read answers it: a 503, never a driver error. +describe('a publish that states no package, over a store that cannot be read', () => { + it('answers 503 SERVICE_UNAVAILABLE, and promotes nothing', async () => { + const { protocol, rows } = makeProtocol(); + await protocol.ensureOverlayIndex(); + await protocol.getOverlayRepo(null).put( + { type: 'view', name: VIEW.name, org: 'env' }, + VIEW, + { parentVersion: null, actor: null, source: 'test.seed', intent: 'runtime-only', state: 'draft', packageId: null }, + ); + const unreadable = () => { throw new Error('connection reset by peer'); }; + protocol.engine.findOne = unreadable; + protocol.engine.find = unreadable; + await expect(protocol.publishMetaItem({ type: 'view', name: VIEW.name })) + .rejects.toMatchObject({ code: 'SERVICE_UNAVAILABLE', status: 503 }); + expect(Array.from(rows.values()).filter((r) => r.state === 'active')).toEqual([]); + }); +}); diff --git a/packages/metadata-protocol/src/protocol.ts b/packages/metadata-protocol/src/protocol.ts index 3bc11cba4cf..961652c249e 100644 --- a/packages/metadata-protocol/src/protocol.ts +++ b/packages/metadata-protocol/src/protocol.ts @@ -43,7 +43,7 @@ import type { RuntimeAuthoringIssue } from './runtime-authoring-gate.js'; // ADR-0120 D4 reporting that replaced this file's empty `catch` blocks. import { ensureMetadataOverlayIndexes } from './migrations/overlay-index.js'; import { driverCanRunSql, resolveDriverExec } from './migrations/driver-exec.js'; -import { SysMetadataRepository, type SysMetadataEngine } from './sys-metadata-repository.js'; +import { DraftConflictError, SysMetadataRepository, type SysMetadataEngine } from './sys-metadata-repository.js'; import { isOriginGatedType, packagedBaseRegimeSentence } from './packaged-base-regime.js'; import { resolveArtifactLockLayer, @@ -9060,15 +9060,26 @@ export class ObjectStackProtocolImplementation implements // read naming the package serves that row before it asks any // expansion. The expansion still seats the slot, so a stand-in // held back by the merge is served there, as the package's. + // + // [#21934] Only a name an expansion writes is upserted by name. + // Every other name keeps what the package-aware merge seated for + // it: one item per package that ships the name (ADR-0048), as the + // list serves it when no row is stored. The env-wide list is the + // layer the anonymous form doors judge a withdrawal against, so + // it holds every package's body of a name. if (isView && records.length > 0) { + const ownRowNames = this.namesWithOwnStoredRow(records); + const standInNames = this.namesWithOwnStoredRow(standInRows); + const expansions = this.expandStoredViewContainers(request.type, overlays) + .filter(({ item: vi }) => !ownRowNames.has(vi.name as string)); + const written = new Set(expansions.map(({ item: vi }) => vi.name as string)); const byName = new Map(); for (const it of items as any[]) { - if (it && typeof it === 'object' && typeof it.name === 'string') byName.set(it.name, it); + if (it && typeof it === 'object' && typeof it.name === 'string' && written.has(it.name)) { + byName.set(it.name, it); + } } - const ownRowNames = this.namesWithOwnStoredRow(records); - const standInNames = this.namesWithOwnStoredRow(standInRows); - for (const { item: vi } of this.expandStoredViewContainers(request.type, overlays)) { - if (ownRowNames.has(vi.name as string)) continue; + for (const { item: vi } of expansions) { const held = byName.get(vi.name as string) as Record | undefined; if (held !== undefined && standInNames.has(vi.name as string)) { // Seated: a copy the `unseated` record does not hold, @@ -9080,7 +9091,17 @@ export class ObjectStackProtocolImplementation implements } byName.set(vi.name as string, vi); } - items = Array.from(byName.values()); + const merged: unknown[] = []; + for (const it of items as any[]) { + if (!it || typeof it !== 'object' || typeof it.name !== 'string') continue; + if (!written.has(it.name)) { + merged.push(it); + } else if (byName.has(it.name)) { + merged.push(byName.get(it.name)); + byName.delete(it.name); + } + } + items = [...merged, ...byName.values()]; } // Only hydrate the global registry for unscoped (control-plane) @@ -16049,7 +16070,9 @@ export class ObjectStackProtocolImplementation implements // by: the env-wide body of row `name`, as stored (a container is not // expanded, so a form moved to another key or slot, renamed through // `form.name`, or renamed by an expansion collision is still matched - // against the form it was, by slot or by slug). + // against the form it was, by slot or by slug). [#21934] One body per + // package that holds the name ({@link envWideRawViewRows}), so every + // package's withdrawal of it is judged, whatever the registry order. const envRows = (await this.envWideRawViewRows(args.type, args.name)).map((r) => ({ ...r, name: args.name })); if (envRows.length > 0) { const own = { ...raw, name: args.name }; @@ -16080,10 +16103,15 @@ export class ObjectStackProtocolImplementation implements } /** - * The env-wide body of the `view` row `name`, as stored: the active - * env-wide `sys_metadata` row when there is one (the env-wide overlay is - * keyed by its own name, ADR-0005), else the code package's artifact of - * that name. Empty when neither exists. + * The env-wide bodies of the `view` row `name`, as stored, for every + * package that holds the name. [#21934] Resolved per package, the way the + * list read resolves each package's item (ADR-0048): the package's own + * active env-wide `sys_metadata` row (the env-wide overlay is keyed by its + * own name, ADR-0005), else the package-less env-wide row, which stands in + * for every package's row of the name, else that package's artifact of + * the name. So a stored row of one package anchors that package only, and + * every package that ships the name is judged on its own definition, + * whatever the registry order. Empty when no package holds the name. * * Read raw, never through the list read: that serves a container only as * its expansion, whose item names and slots the overlay author chooses, @@ -16099,11 +16127,18 @@ export class ObjectStackProtocolImplementation implements } const stored = this.storedOverlayEntries({ type }, records) .filter((e) => e.name === name && e.organizationId === null) - .map((e) => e.data) - .filter((d): d is Record => !!d && typeof d === 'object' && !Array.isArray(d)); - if (stored.length > 0) return stored; - const artifact = this.lookupArtifactItem(type, name); - return artifact && typeof artifact === 'object' ? [artifact as Record] : []; + .filter((e) => !!e.data && typeof e.data === 'object' && !Array.isArray(e.data)); + const bodies = stored.map((e) => e.data as Record); + const withOwnRow = new Set(stored.map((e) => e.packageId)); + // The package-less row stands in for every package without a row of its own. + if (withOwnRow.has(undefined)) return bodies; + for (const artifact of this.shippedArtifactsOf(type, name)) { + if (!artifact || typeof artifact !== 'object' || Array.isArray(artifact)) continue; + const pkg = (artifact as { _packageId?: unknown })._packageId; + if (typeof pkg === 'string' && withOwnRow.has(pkg)) continue; + bodies.push(artifact as Record); + } + return bodies; } /** @@ -21575,24 +21610,6 @@ export class ObjectStackProtocolImplementation implements ); if (orgRefusal) throw orgRefusal; } - // ADR-0010 L3 — lock blocks publish too (publishing is a write). - // - // [#8594] `lockWriteRefusal`, not `assertLockAllowsWrite`: the row rides - // OUT on the error and each caller records it on its own side of its own - // transaction. See this method's header for why it cannot be written here. - const _publishLockRefusal = await this.lockWriteRefusal({ - type: request.type, - name: request.name, - ...(request.organizationId ? { organizationId: request.organizationId } : {}), - // [#21761] The promotion's address carries its package, as the read's does. - ...(request.packageId ? { packageId: request.packageId } : {}), - operation: 'publish', - ...(request.actor ? { actor: request.actor } : {}), - source: 'protocol.publishMetaItem', - }); - if (_publishLockRefusal) { - throw withPendingAudit(_publishLockRefusal.err, _publishLockRefusal.audit); - } await this.ensureOverlayIndex(); const orgId = request.organizationId ?? null; const repo = this.getOverlayRepo(orgId); @@ -21612,14 +21629,44 @@ export class ObjectStackProtocolImplementation implements // dimension picks either. The key is the caller's stated binding // (spelled exactly as `repo.promoteDraft` receives it); with none // stated, the binding of the draft row this promotion resolves, read - // once and then stated to both the read and the promotion. + // once and then stated to the lock lookup, the read and the promotion. let draftKey: string | null | undefined = 'packageId' in request ? (request.packageId ?? null) : undefined; if (draftKey === undefined) { - const draftRow = await this.engine.findOne('sys_metadata', { - where: { type: singularType, name: request.name, organization_id: orgId, state: 'draft' }, - }); + // [#21934] Read ahead of the lock check, so a store that cannot be + // read is answered here as the lock read answers it (#5706): an + // unprovisioned `sys_metadata` holds no draft, and any other failure + // is the 503 the lock read raised before this read moved above it. + let draftRow: unknown; + try { + draftRow = await this.engine.findOne('sys_metadata', { + where: { type: singularType, name: request.name, organization_id: orgId, state: 'draft' }, + }); + } catch (error) { + this.rethrowUnlessMetadataStoreUnprovisioned(error, 'sys_metadata'); + } if (draftRow) draftKey = (draftRow as { package_id?: string | null }).package_id ?? null; } + // ADR-0010 L3 — lock blocks publish too (publishing is a write). + // + // [#8594] `lockWriteRefusal`, not `assertLockAllowsWrite`: the row rides + // OUT on the error and each caller records it on its own side of its own + // transaction. See this method's header for why it cannot be written here. + const _publishLockRefusal = await this.lockWriteRefusal({ + type: request.type, + name: request.name, + ...(request.organizationId ? { organizationId: request.organizationId } : {}), + // [#21761] The promotion's address carries its package, as the read's does. + // [#21934] It is the key resolved above, the one the gate reads the + // draft under and the promotion writes under: the caller's stated + // binding, else the resolved draft row's own. + ...(draftKey ? { packageId: draftKey } : {}), + operation: 'publish', + ...(request.actor ? { actor: request.actor } : {}), + source: 'protocol.publishMetaItem', + }); + if (_publishLockRefusal) { + throw withPendingAudit(_publishLockRefusal.err, _publishLockRefusal.audit); + } const draftForGate = await repo.get( { type: singularType, name: request.name, org: orgId ?? 'env' } as Parameters[0], { state: 'draft', ...(draftKey !== undefined ? { packageId: draftKey } : {}) }, @@ -21753,6 +21800,11 @@ export class ObjectStackProtocolImplementation implements // draft than the one judged above. Absent only when no draft // was found, where the promotion answers `NO_DRAFT` as before. ...(draftKey !== undefined ? { packageId: draftKey } : {}), + // [#21934] …and the promotion writes only the draft the gate + // judged: the judged draft's hash (`null` when the gate found + // none), so a draft saved after the gate's read is refused as a + // conflict instead of being promoted unjudged. + expectedDraftHash: draftForGate ? draftForGate.hash : null, }); return { singularType, orgId, advisories: runtimeAdvisories, result }; } catch (err: any) { @@ -21761,7 +21813,10 @@ export class ObjectStackProtocolImplementation implements const conflict = await this.metadataConflictRefusal( err, `${request.type}/${request.name}`, - `${request.type}/${request.name} published row advanced while you held the draft.`, + err instanceof DraftConflictError + ? `${request.type}/${request.name} draft changed after this publish judged it, so nothing was ` + + `published. Publish again to judge and promote the current draft.` + : `${request.type}/${request.name} published row advanced while you held the draft.`, ); // [#8594] Attached, not written — same reason as the lock gate // above. The repository's own transaction has already unwound by diff --git a/packages/metadata-protocol/src/sys-metadata-repository.ts b/packages/metadata-protocol/src/sys-metadata-repository.ts index 5793d264aab..80789431dc9 100644 --- a/packages/metadata-protocol/src/sys-metadata-repository.ts +++ b/packages/metadata-protocol/src/sys-metadata-repository.ts @@ -232,6 +232,17 @@ export interface DraftDrainFailure { cause: unknown; } +/** + * [#21934] {@link SysMetadataRepository.promoteDraft}'s refusal when the draft + * row it reads is not the draft its caller judged + * (`opts.expectedDraftHash`): `expectedParent` is the judged draft's hash + * (`null` when the caller judged that no draft exists), `actualHead` the hash + * of the draft row now stored. Raised before anything is written. A + * {@link ConflictError}, so every caller that already answers a conflict + * answers this one; the protocol tells the two apart to word its refusal. + */ +export class DraftConflictError extends ConflictError {} + /** * Sub-set of the ObjectQL engine shape we depend on. Kept narrow so * tests can stub it with a plain mock. Mirrors the real engine's @@ -942,6 +953,19 @@ export class SysMetadataRepository implements MetadataRepository { * what lands in `active`, never which draft is consumed. */ deriveActiveBody?: (draftBody: unknown) => unknown; + /** + * [#21934] The hash of the draft the caller judged before this + * promotion, or `null` when it judged that no draft exists. Stated → the + * draft row this promotion reads must be that draft: a row with another + * hash (a draft saved after the caller's read), or any draft row where + * the caller judged none, is refused with a {@link DraftConflictError} + * before anything is written, so the promotion writes only the body its + * caller judged. A missing draft row still answers `NO_DRAFT`. + * + * Omitted → no expectation: the draft row read here is promoted, as + * before. + */ + expectedDraftHash?: string | null; }, ): Promise<{ version: string; @@ -984,6 +1008,12 @@ export class SysMetadataRepository implements MetadataRepository { } const draftPackageId = (draftRow as { package_id?: string | null }).package_id ?? null; const draft = this.rowToItem(ref, draftRow); + // [#21934] The draft the caller judged, and no other: a caller that read + // the draft to judge it states that draft's hash, and the row read here + // must still be it. + if (opts.expectedDraftHash !== undefined && draft.hash !== opts.expectedDraftHash) { + throw new DraftConflictError(this.fullRef(ref), opts.expectedDraftHash, draft.hash); + } // Read the active row through the SAME package scope we will write, so the // optimistic-lock `parentVersion` matches the exact row `put` upserts. // (Package-less drafts → packageId null → identical to the prior behaviour.)