From e647cfcafe2cd6c7c9785f1f365b7d02eb85a28f Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 5 Oct 2026 03:38:20 +0000 Subject: [PATCH 1/5] fix(metadata-core): a reorder of an object's fields is a content change The canonical hash sorted every map, so a pure reorder of `ObjectSchema.fields` -- whose traversal order the spec declares to be the display order -- hashed equal, the repository's no-op short-circuit read the publish as unchanged, and the object designer's reorder was dropped while the publish answered success. `canonicalize` / `hashSpec` take the metadata type; a per-type table names the maps whose key order is content (today `object.fields`). Every other map, and every key inside a field definition, stays key-order independent, and a call without a type is byte-identical to before. Each repository hashes as `ref.type`. `SysMetadataRepository.put` decides "unchanged" for a type with an ordered map by re-hashing the stored body under the current rule, so a row stamped before this change neither writes a no-change history row on an identical re-save nor drops a reorder into sorted order; a skipped write hands back the stored stamp, the version token readers already hold. Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude --- packages/metadata-core/src/canonicalize.ts | 115 +++++++++-- packages/metadata-core/src/contract-suite.ts | 10 +- .../metadata-core/src/in-memory-repository.ts | 3 +- packages/metadata-core/src/repository.ts | 7 +- packages/metadata-core/src/types.ts | 4 +- .../metadata-core/test/canonicalize.test.ts | 179 +++++++++++++++++- packages/metadata-fs/src/repository.ts | 10 +- .../src/sys-metadata-repository.ts | 69 ++++++- 8 files changed, 364 insertions(+), 33 deletions(-) diff --git a/packages/metadata-core/src/canonicalize.ts b/packages/metadata-core/src/canonicalize.ts index 615107405e8..c2483ce615d 100644 --- a/packages/metadata-core/src/canonicalize.ts +++ b/packages/metadata-core/src/canonicalize.ts @@ -7,7 +7,8 @@ * Guarantees: * * 1. **Key order independence.** Object keys are sorted lexicographically. - * `{a:1, b:2}` and `{b:2, a:1}` produce the same canonical form. + * `{a:1, b:2}` and `{b:2, a:1}` produce the same canonical form — for + * every map EXCEPT the ones guarantee 8 names, whose order is content. * 2. **Whitespace independence.** No incidental whitespace. * 3. **Type preservation.** `undefined` properties are dropped (matching * JSON.stringify), `null` is preserved, arrays preserve order. @@ -40,6 +41,31 @@ * are untouched by this: their canonical form is byte-identical to what * it was before, so no already-stored version changes meaning. * + * 8. **Declared map order (#21790).** A map whose key order the spec + * declares meaningful keeps its insertion order instead of being sorted, + * so a pure reorder of it is a change. Which maps those are is a fact + * about the metadata TYPE, so it applies only when the caller names the + * type (`canonicalize(body, type)`); see {@link orderedMapKeys} for the + * table and the spec text behind each row. Everything else in the body — + * the keys AROUND an ordered map and the keys INSIDE each of its values — + * stays key-order independent (guarantee 1). With no type, or a type with + * no row, the canonical form is byte-identical to what it was before. + * + * Guarantees 5 and 7 hold unchanged: `JSON.stringify` emits own keys in + * the order `Object.keys` reports them and `JSON.parse` recreates them in + * that order, so the order kept here survives every serialisation round + * trip, and the canonical output of an ordered map re-canonicalises to + * itself. + * + * The cost is the hashes already stored: a body whose ordered map is not + * already in sorted key order hashes differently under this rule than + * under the old one. A stamp written before #21790 still names its row — + * it is the row's version token, and the optimistic lock compares a + * caller's token with that stored stamp, never with a re-derivation — so + * the comparison that needed teaching is "is this write a no-op?", which + * must compare CONTENT under the current rule rather than a stored stamp + * taken under an older one (`SysMetadataRepository.put`). + * * Non-goals (deliberately not supported): * * - Functions and symbols. These have no canonical JSON form. Callers must @@ -54,10 +80,57 @@ import { createHash } from 'node:crypto'; -/** Stable JSON serialisation. See module-level doc for guarantees. */ -export function canonicalize(value: unknown): string { +const NO_ORDERED_MAPS: readonly string[] = Object.freeze([]); + +/** + * Guarantee 8's table — per metadata type, the top-level keys of its body + * whose value is a map with MEANINGFUL KEY ORDER. + * + * A row is owed only where `packages/spec` declares the order; ⛔ never an + * order inferred from what a renderer happens to do with a map, because a + * row here turns every reorder of that map into a new version. + * + * - `object` → `fields`. `ObjectSchema.fields` is a name-keyed record whose + * traversal order IS the field order the platform presents: the + * `ObjectFieldGroupSchema` contract (`data/object.zod.ts`) sets "the + * in-group display order equals the traversal order of + * `ObjectSchema.fields`" and lays ungrouped fields out "preserving their + * field declaration order"; `fieldGroups` repeats it ("fields are + * displayed in their `ObjectSchema.fields` declaration order"); the + * `record` field type it is authored through declares "Insertion order = + * display order" (`data/field.zod.ts`); and the display-name derivation + * picks the first title-eligible field "by declaration order" + * (`data/display-name.ts`). A hash blind to that order called a pure + * reorder "no change", and the object designer's publish was dropped. + * + * Keyed by the canonical (singular) type name the repositories write under. + * A `Map`, so a type name that happens to be an `Object.prototype` member + * reads as "no row" rather than as an inherited function. + */ +const ORDERED_MAPS: ReadonlyMap = new Map([ + ['object', Object.freeze(['fields'])], +]); + +/** + * The top-level keys of a `type` body whose map value keeps its key order in + * the canonical form (guarantee 8). Empty when `type` is omitted or has no + * declared ordered map — the case in which canonicalization is exactly the + * type-blind form it has always been. + */ +export function orderedMapKeys(type?: string): readonly string[] { + return (type === undefined ? undefined : ORDERED_MAPS.get(type)) ?? NO_ORDERED_MAPS; +} + +/** + * Stable JSON serialisation. See module-level doc for guarantees. + * + * @param type The metadata type of `value`, when it is a metadata body. + * Decides which maps keep their order (guarantee 8); omit it for + * a value that is not one, which sorts every map. + */ +export function canonicalize(value: unknown, type?: string): string { // `''` is the key `JSON.stringify` hands a root-position `toJSON`. - return JSON.stringify(normalise(value, '')); + return JSON.stringify(normalise(value, '', orderedMapKeys(type), false)); } /** @@ -83,8 +156,19 @@ function resolveToJson(value: unknown, key: string): unknown { * @param key The position's key — `''` at the root, the property name * inside an object, the stringified index inside an array. * Passed to `toJSON` because `JSON.stringify` passes it. + * @param orderedChildren Keys of THIS position's object whose values keep + * their own key order (guarantee 8). Non-empty only at the root; + * every deeper position receives none. + * @param keepOwnOrder Whether THIS position, when it is an object, keeps its + * keys in insertion order instead of sorting them. Applies to + * this level only — its values are normalised as usual. */ -function normalise(rawValue: unknown, key: string): unknown { +function normalise( + rawValue: unknown, + key: string, + orderedChildren: readonly string[], + keepOwnOrder: boolean, +): unknown { const value = resolveToJson(rawValue, key); if (value === null) return null; if (typeof value === 'undefined') return undefined; // caller-dropped @@ -107,16 +191,21 @@ function normalise(rawValue: unknown, key: string): unknown { // the second argument, which is this function's `key` parameter. The // index is the right key, but it has to arrive as the string // `JSON.stringify` would hand a `toJSON`, not as a number. - return value.map((element, index) => normalise(element, String(index))); + return value.map((element, index) => + normalise(element, String(index), NO_ORDERED_MAPS, false), + ); } - // Plain object: sort keys, drop undefineds. + // Plain object: sort keys (unless this is a declared ordered map), drop + // undefineds. if (typeof value === 'object') { const obj = value as Record; const out: Record = {}; - const keys = Object.keys(obj).sort(); + // `Object.keys` order is the order `JSON.stringify` writes, so keeping it + // is what keeps guarantee 7 for an ordered map. + const keys = keepOwnOrder ? Object.keys(obj) : Object.keys(obj).sort(); for (const k of keys) { - const v = normalise(obj[k], k); + const v = normalise(obj[k], k, NO_ORDERED_MAPS, orderedChildren.includes(k)); if (typeof v === 'undefined') continue; out[k] = v; } @@ -129,9 +218,13 @@ function normalise(rawValue: unknown, key: string): unknown { /** * Compute the canonical sha256 hash of a spec, returned as * `"sha256:<64-hex>"`. Equal hashes imply equal canonical forms. + * + * @param type The metadata type of `value` — a repository passes its + * `ref.type`, so a reorder of a declared ordered map is a new + * version (guarantee 8). Omitted, every map is order-blind. */ -export function hashSpec(value: unknown): string { - const json = canonicalize(value); +export function hashSpec(value: unknown, type?: string): string { + const json = canonicalize(value, type); const digest = createHash('sha256').update(json, 'utf8').digest('hex'); return `sha256:${digest}`; } diff --git a/packages/metadata-core/src/contract-suite.ts b/packages/metadata-core/src/contract-suite.ts index cc43d4b043b..21d3563fb8d 100644 --- a/packages/metadata-core/src/contract-suite.ts +++ b/packages/metadata-core/src/contract-suite.ts @@ -12,7 +12,7 @@ * 1. Atomic put * 2. Monotonic seq per branch * 3. Optimistic locking (ConflictError) - * 4. Canonical hashing (hash === hashSpec(body)) + * 4. Canonical hashing (hash === hashSpec(body, ref.type)) * 5. Event ordering (monotonic seq, no gaps) * 6. Resumability (a numeric `since` replays; a bare `watch(filter)` is * owed live events only) @@ -204,7 +204,7 @@ export function runRepositoryContractTests( const ref = refOf(); const res = await repo.put(ref, spec('hello'), { parentVersion: null, actor: 'tester' }); expect(res.version).toMatch(/^sha256:[0-9a-f]{64}$/); - expect(res.version).toBe(hashSpec(spec('hello'))); + expect(res.version).toBe(hashSpec(spec('hello'), ref.type)); expect(res.seq).toBeGreaterThan(0); expect(res.item.parentHash).toBeNull(); expect(res.item.authoredBy).toBe('tester'); @@ -227,12 +227,12 @@ export function runRepositoryContractTests( expect(b.seq).toBeGreaterThan(a.seq); }); - it('canonical hash invariant: item.hash === hashSpec(item.body)', async () => { + it('canonical hash invariant: item.hash === hashSpec(item.body, item.ref.type)', async () => { const repo = await factory(); const ref = refOf(); await repo.put(ref, { z: 1, a: 2, m: [3, 1, 2] }, { parentVersion: null, actor: 't' }); const got = await repo.get(ref); - expect(got!.hash).toBe(hashSpec(got!.body)); + expect(got!.hash).toBe(hashSpec(got!.body, got!.ref.type)); }); // ── #7856 — the version identifies the STORED BYTES ─────────── @@ -267,7 +267,7 @@ export function runRepositoryContractTests( const got = await repo.get(ref); expect(got).not.toBeNull(); expect(got!.hash).toBe(put.version); - expect(got!.hash).toBe(hashSpec(got!.body)); + expect(got!.hash).toBe(hashSpec(got!.body, got!.ref.type)); }); it(`re-putting the same spec is a no-op — ${shape.label}`, async () => { diff --git a/packages/metadata-core/src/in-memory-repository.ts b/packages/metadata-core/src/in-memory-repository.ts index 168fa6e3b10..2f06a32e912 100644 --- a/packages/metadata-core/src/in-memory-repository.ts +++ b/packages/metadata-core/src/in-memory-repository.ts @@ -101,7 +101,8 @@ export class InMemoryRepository implements MetadataRepository { throw new ConflictError(ref, opts.parentVersion ?? null, currentHead); } - const hash = hashSpec(spec); + // [#21790] Hashed as its type — a declared ordered map keeps its order. + const hash = hashSpec(spec, ref.type); // No-op write — same content. Still consumes nothing; no event emitted. if (current && current.hash === hash) { diff --git a/packages/metadata-core/src/repository.ts b/packages/metadata-core/src/repository.ts index 1f325a6d8e9..e8d4903d28f 100644 --- a/packages/metadata-core/src/repository.ts +++ b/packages/metadata-core/src/repository.ts @@ -21,7 +21,12 @@ * scoped to a single org may treat the entire repo as one log.) * 3. **Optimistic locking.** `put` and `delete` throw `ConflictError` * when `parentVersion` does not match the current HEAD. - * 4. **Canonical hashing.** `item.hash === hashSpec(item.body)` — always. + * 4. **Canonical hashing.** `item.hash === hashSpec(item.body, item.ref.type)` + * — always. The type decides which maps keep their key order + * (`canonicalize` guarantee 8). One stated exception, owned by + * `SysMetadataRepository`: a row stamped before its type's canonical form + * changed keeps the stamp it was written with until its content next + * changes — that stamp is its version token. * 5. **Event ordering.** Subscribers to `watch()` receive events in * monotonically-increasing `seq` order with no gaps. * 6. **Resumability, and where it stops.** `watch(_, since)` called with a diff --git a/packages/metadata-core/src/types.ts b/packages/metadata-core/src/types.ts index 2b9d9d426d3..9b6dac1aa5e 100644 --- a/packages/metadata-core/src/types.ts +++ b/packages/metadata-core/src/types.ts @@ -91,13 +91,13 @@ export function refKey(ref: Pick): string { * Full metadata item as stored / returned by the Repository. * * `body` is the **canonical, Zod-normalised** spec (with defaults filled - * in). `hash` is `sha256(canonicalize(body))`. Equal hashes imply equal + * in). `hash` is `sha256(canonicalize(body, ref.type))`. Equal hashes imply equal * specs. */ export const MetadataItemSchema = z.object({ ref: MetaRefSchema, body: z.record(z.string(), z.unknown()).describe('Canonical Zod-normalised spec'), - hash: z.string().regex(/^sha256:[0-9a-f]{64}$/).describe('sha256(canonicalize(body))'), + hash: z.string().regex(/^sha256:[0-9a-f]{64}$/).describe('sha256(canonicalize(body, ref.type))'), parentHash: z.string().nullable().describe('Hash this version was derived from; null for first version'), authoredBy: z.string().nullable().describe('Identity of the writer (user id, "cli", "ai:claude", …); null = system-initiated, no actor'), authoredAt: z.string().describe('ISO-8601 timestamp'), diff --git a/packages/metadata-core/test/canonicalize.test.ts b/packages/metadata-core/test/canonicalize.test.ts index b70716efb85..a714c4fa3cf 100644 --- a/packages/metadata-core/test/canonicalize.test.ts +++ b/packages/metadata-core/test/canonicalize.test.ts @@ -2,7 +2,7 @@ import { describe, it, expect } from 'vitest'; import * as fc from 'fast-check'; -import { canonicalize, hashSpec } from '../src/canonicalize.js'; +import { canonicalize, hashSpec, orderedMapKeys } from '../src/canonicalize.js'; describe('canonicalize', () => { it('orders object keys lexicographically', () => { @@ -230,3 +230,180 @@ describe('hashSpec', () => { }); }); }); + +/** + * Guarantee 8 (#21790): a map the spec declares ordered keeps its order, so a + * pure reorder of it is a change — and NOTHING else in the body moved. + * + * Both halves are pinned on purpose. The defect was a hash blind to the order + * of `ObjectSchema.fields`, so an object-designer reorder published as "no + * change" and was dropped; a fix that stopped sorting EVERY map would close it + * too, and turn every incidental key swap into a new version. So each order + * case below sits beside the unordered case it must not disturb. + */ +describe('declared map order (#21790)', () => { + const field = (type: string, label: string) => ({ type, label, required: false }); + const object = (fields: Record, extra: Record = {}) => ({ + name: 'invoice', + label: 'Invoice', + fields, + ...extra, + }); + // Declared order — deliberately NOT sorted (`amount` < `due` < `title`). + const declared = () => object({ + title: field('text', 'Title'), + amount: field('currency', 'Amount'), + due: field('date', 'Due'), + }); + const reordered = () => object({ + due: field('date', 'Due'), + title: field('text', 'Title'), + amount: field('currency', 'Amount'), + }); + + it('names exactly `object.fields`, and nothing for an unnamed or unknown type', () => { + expect(orderedMapKeys('object')).toEqual(['fields']); + expect(orderedMapKeys()).toEqual([]); + expect(orderedMapKeys('view')).toEqual([]); + // A `Map` lookup: an Object.prototype member is not an inherited row. + expect(orderedMapKeys('constructor')).toEqual([]); + expect(orderedMapKeys('toString')).toEqual([]); + }); + + describe('the order half — a reorder of `fields` is a change', () => { + it('hashes a reordered `fields` map differently, as an object', () => { + expect(hashSpec(reordered(), 'object')).not.toBe(hashSpec(declared(), 'object')); + }); + + it('keeps the declared field order in the canonical form', () => { + expect(canonicalize(declared(), 'object')).toBe( + '{"fields":{' + + '"title":{"label":"Title","required":false,"type":"text"},' + + '"amount":{"label":"Amount","required":false,"type":"currency"},' + + '"due":{"label":"Due","required":false,"type":"date"}' + + '},"label":"Invoice","name":"invoice"}', + ); + }); + + it('a reorder INTO sorted order is still a change', () => { + // The sorted form is exactly what the order-blind rule hashed every + // object to, so this is the reorder a stale comparison would miss. + const sorted = object({ + amount: field('currency', 'Amount'), + due: field('date', 'Due'), + title: field('text', 'Title'), + }); + expect(hashSpec(sorted, 'object')).not.toBe(hashSpec(declared(), 'object')); + }); + }); + + describe('the unordered half — every other map stays key-order independent (H5)', () => { + it('top level: swapping the keys AROUND `fields` hashes equal', () => { + const swapped = { fields: declared().fields, name: 'invoice', label: 'Invoice' }; + expect(Object.keys(swapped)).not.toEqual(Object.keys(declared())); + expect(hashSpec(swapped, 'object')).toBe(hashSpec(declared(), 'object')); + }); + + it('nested: swapping the keys INSIDE a field definition hashes equal', () => { + const inner = object({ + title: { required: false, label: 'Title', type: 'text' }, + amount: field('currency', 'Amount'), + due: field('date', 'Due'), + }); + expect(hashSpec(inner, 'object')).toBe(hashSpec(declared(), 'object')); + }); + + it('nested deeper: a map inside a field definition is not an ordered map', () => { + const withOptions = (options: Record) => + object({ status: { type: 'select', label: 'Status', meta: options } }); + expect(hashSpec(withOptions({ b: 2, a: 1 }), 'object')).toBe( + hashSpec(withOptions({ a: 1, b: 2 }), 'object'), + ); + }); + + it('a `fields` map on a type with no declared order hashes equal under a reorder', () => { + // Guarantee 8 follows the spec's declaration, not the key's name. + expect(hashSpec(reordered(), 'view')).toBe(hashSpec(declared(), 'view')); + }); + + it('with no type, a reorder of `fields` hashes equal — the type-blind form is unchanged', () => { + expect(hashSpec(reordered())).toBe(hashSpec(declared())); + }); + + it('property: under the object type, reordering any key but `fields` hashes equal', () => { + fc.assert( + fc.property( + fc.dictionary( + fc.string({ minLength: 1, maxLength: 8 }).filter((k) => k !== 'fields'), + fc.jsonValue(), + ), + (rest) => { + const body = { ...rest, fields: declared().fields }; + const reversed = Object.fromEntries(Object.entries(body).reverse()); + return hashSpec(body, 'object') === hashSpec(reversed, 'object'); + }, + ), + { numRuns: 200 }, + ); + }); + }); + + describe('what stays byte-identical', () => { + it('an object whose `fields` are already in sorted order hashes exactly as before', () => { + // So a row stored before #21790 with sorted fields keeps a valid stamp. + const sorted = object({ amount: field('currency', 'Amount'), due: field('date', 'Due') }); + expect(canonicalize(sorted, 'object')).toBe(canonicalize(sorted)); + expect(hashSpec(sorted, 'object')).toBe(hashSpec(sorted)); + }); + + it('the empty-spec fixture is unchanged under the object type', () => { + expect(hashSpec({}, 'object')).toBe( + 'sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a', + ); + }); + }); + + describe('guarantees 5 and 7 hold with an ordered map', () => { + const roundTrip = (x: unknown): unknown => JSON.parse(JSON.stringify(x)); + // Field names are snake_case identifiers, never integer-like, so their + // insertion order is their enumeration order. + const fieldName = fc.stringMatching(/^[a-z_][a-z0-9_]{0,7}$/); + const objectBody = fc.record({ + name: fc.string(), + fields: fc.dictionary(fieldName, fc.jsonValue(), { maxKeys: 6 }), + extra: fc.jsonValue(), + }); + + it('guarantee 5 — property: idempotent under the object type', () => { + fc.assert( + fc.property(objectBody, (body) => { + const once = canonicalize(body, 'object'); + return canonicalize(JSON.parse(once), 'object') === once; + }), + { numRuns: 200 }, + ); + }); + + it('guarantee 7 — property: invariant under a JSON round trip, under the object type', () => { + fc.assert( + fc.property(objectBody, (body) => hashSpec(body, 'object') === hashSpec(roundTrip(body), 'object')), + { numRuns: 200 }, + ); + }); + + it('guarantee 7 — a toJSON inside `fields`, and on `fields` itself, hashes as its bytes', () => { + const withDate = object({ + title: field('text', 'Title'), + due: { type: 'date', label: 'Due', defaultValue: new Date('2024-01-01T00:00:00.000Z') }, + }); + expect(hashSpec(withDate, 'object')).toBe(hashSpec(roundTrip(withDate), 'object')); + + const viaToJson = { + name: 'invoice', + fields: { toJSON: () => ({ title: field('text', 'Title'), amount: field('currency', 'Amount') }) }, + }; + expect(hashSpec(viaToJson, 'object')).toBe(hashSpec(roundTrip(viaToJson), 'object')); + expect(canonicalize(viaToJson, 'object')).toContain('"fields":{"title":'); + }); + }); +}); diff --git a/packages/metadata-fs/src/repository.ts b/packages/metadata-fs/src/repository.ts index 888f3920343..f6b4a00f7db 100644 --- a/packages/metadata-fs/src/repository.ts +++ b/packages/metadata-fs/src/repository.ts @@ -258,7 +258,9 @@ export class FileSystemRepository implements MetadataRepository { if (!existsSync(file)) return null; const body = await readJson(file); if (!body) return null; - const hash = hashSpec(body); + // [#21790] Every hash here is taken AS THE ITEM'S TYPE, so a reorder of a + // declared ordered map (an object's `fields`) is a content change. + const hash = hashSpec(body, ref.type); if (ref.version && ref.version !== hash) return null; // Walk back through the log to populate parent/authoredBy/seq. const meta = await this.findMetaForHash(ref, hash); @@ -358,7 +360,7 @@ export class FileSystemRepository implements MetadataRepository { if ((opts.parentVersion ?? null) !== currentHead) { throw new ConflictError(ref, opts.parentVersion ?? null, currentHead); } - const hash = hashSpec(spec); + const hash = hashSpec(spec, ref.type); if (currentHead === hash) { // No-op write — same content. const meta = await this.findMetaForHash(ref, hash); @@ -514,7 +516,7 @@ export class FileSystemRepository implements MetadataRepository { }; const body = await readJson(path.join(dir, file)); if (!body) continue; - this.heads.set(refKey(ref), hashSpec(body)); + this.heads.set(refKey(ref), hashSpec(body, ref.type)); } } } @@ -1001,7 +1003,7 @@ export class FileSystemRepository implements MetadataRepository { } const body = await readJson(absPath); if (!body) return; - const hash = hashSpec(body); + const hash = hashSpec(body, ref.type); const currentHead = this.heads.get(key) ?? null; if (currentHead === hash) return; // No content change. this.heads.set(key, hash); diff --git a/packages/metadata-protocol/src/sys-metadata-repository.ts b/packages/metadata-protocol/src/sys-metadata-repository.ts index 9de1a376cdd..013ffb59ed3 100644 --- a/packages/metadata-protocol/src/sys-metadata-repository.ts +++ b/packages/metadata-protocol/src/sys-metadata-repository.ts @@ -22,9 +22,9 @@ * entry has `allowOrgOverride: false` (Prime Directive #8) * - **M1**: every successful put/delete appends a durable row to * `sys_metadata_history` inside the same engine.transaction() as the - * parent `sys_metadata` write. No-op puts (identical hash) skip the - * history write. Failed optimistic-lock checks abort before any - * write reaches the database. + * parent `sys_metadata` write. No-op puts (identical content — see + * `storedBodyUnchanged`) skip the history write. Failed optimistic-lock + * checks abort before any write reaches the database. * - **M1**: history() yields events from the durable log, ordered by * per-(org,type,name) `version` ASC. * @@ -52,7 +52,7 @@ * switch. */ -import { hashSpec, ConflictError } from '@objectstack/metadata-core'; +import { hashSpec, orderedMapKeys, ConflictError } from '@objectstack/metadata-core'; // #4867 — the SAME discriminator `DatabaseLoader` uses (#4825), imported, not // re-implemented. A second hand-rolled "which driver errors are benign?" here // would be two vocabularies for one question, which is the dual-source debt @@ -534,7 +534,9 @@ export class SysMetadataRepository implements MetadataRepository { const state: OverlayState = opts.state ?? 'active'; const body = (spec ?? {}) as Record; - const hash = hashSpec(body); + // [#21790] Hashed AS ITS TYPE: a map the spec declares ordered (an + // object's `fields`) keeps its key order, so a pure reorder is a change. + const hash = hashSpec(body, ref.type); // ADR-0048 — the ONE row this write targets. A write is not a search: it // upserts exactly one `(org, type, name, package_id)` row, so its scope is @@ -617,9 +619,17 @@ export class SysMetadataRepository implements MetadataRepository { // No-op short-circuit: identical body → no write, no history row, // no event. We re-yield the existing item so callers see the // canonical hash but the seqCounter is unchanged. - if (existing && existingHash === hash) { + // + // [#21790] The version handed back is the row's STORED stamp, not this + // write's hash. The two are one value except on a row stamped before + // its type's canonical form changed (see `storedBodyUnchanged`), where + // the stored stamp is the version every reader was given and the one + // the next write's optimistic lock compares against — answering with + // the re-derived hash instead would turn the caller's next save into a + // conflict with a row nobody touched. + if (existing && this.storedBodyUnchanged(ref, existing, existingHash, hash)) { const item = this.rowToItem(ref, existing); - return { skipped: true as const, version: hash, seq: item.seq, item }; + return { skipped: true as const, version: item.hash, seq: item.seq, item }; } const now = new Date().toISOString(); @@ -1942,7 +1952,7 @@ export class SysMetadataRepository implements MetadataRepository { private rowToItem(ref: Pick, row: any): MetadataItem { const body: Record = typeof row.metadata === 'string' ? JSON.parse(row.metadata) : (row.metadata ?? {}); - const hash: string = row.checksum ?? hashSpec(body); + const hash: string = row.checksum ?? hashSpec(body, ref.type); return { ref: this.fullRef(ref), body, @@ -1959,6 +1969,49 @@ export class SysMetadataRepository implements MetadataRepository { }; } + /** + * [#21790] Does `row` already store the body this write hashes to `hash`? + * The question `put`'s no-op short-circuit asks — about CONTENT. + * + * For a type with no declared ordered map the stored stamp answers it, as it + * always has: those types' canonical form never changed, so a stamp equals + * the hash of the bytes beside it. + * + * A type WITH one (`orderedMapKeys`) is answered by re-hashing the stored + * body under the current rule, because its stored stamp may predate that + * rule — every row written before the canonical form started keeping the + * map's order carries the order-blind stamp. Trusting that stamp misfires + * BOTH ways: + * - an identical re-save looks changed (current hash ≠ old stamp) and + * writes a history row recording no change — so the stamp is not the + * arbiter of "unchanged"; + * - a reorder INTO sorted key order looks unchanged, because the old + * stamp IS the sorted form's hash — the very drop #21790 fixed, + * surviving on exactly the rows stored before the fix. + * Re-hashing the stored bytes is exact in both directions. The stamp itself + * is left as written: it is the row's version token (see `put`). + * + * Stored bytes that do not parse are not the incoming body, so the answer is + * "changed" and the write goes ahead — which is also what lets a save repair + * such a row, exactly as it could before this comparison read the bytes. + */ + private storedBodyUnchanged( + ref: Pick, + row: any, + storedHash: string | null, + hash: string, + ): boolean { + if (orderedMapKeys(ref.type).length === 0) return storedHash === hash; + let storedBody: unknown; + try { + storedBody = + typeof row.metadata === 'string' ? JSON.parse(row.metadata) : (row.metadata ?? {}); + } catch { + return false; + } + return hashSpec(storedBody, ref.type) === hash; + } + private broadcast(evt: MetadataEvent): void { for (const w of Array.from(this.watchers)) { try { w.dispatch(evt); } catch { /* listener errors don't break the repo */ } From 4f1eb9e26b5f11c4f06df9620f35f06498dbbf28 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 5 Oct 2026 03:43:11 +0000 Subject: [PATCH 2/5] test(metadata): pin the ordered-fields hash end to end, and the stamps stored before it - dogfood: an object-designer reorder saves as a draft, publishes and reads back in the new order through `GET /meta/object/:name`; then, over a row rewound to the order-blind stamp, an identical save records no change and keeps the stamp, its receipt token is honoured by If-Match, and a reorder into sorted order -- whose hash IS the stale stamp -- is published. - SysMetadataRepository: the same two halves at the repository, through `promoteDraft`, on the contract file's engine double. - changesets for the three moved packages. Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude --- ...21790-metadata-core-ordered-fields-hash.md | 14 ++ .../21790-metadata-fs-ordered-fields-hash.md | 9 + ...0-metadata-protocol-ordered-fields-hash.md | 12 ++ .../sys-metadata-repository.contract.test.ts | 143 +++++++++++++++ ...ect-designer-field-reorder.dogfood.test.ts | 167 ++++++++++++++++++ 5 files changed, 345 insertions(+) create mode 100644 .changeset/21790-metadata-core-ordered-fields-hash.md create mode 100644 .changeset/21790-metadata-fs-ordered-fields-hash.md create mode 100644 .changeset/21790-metadata-protocol-ordered-fields-hash.md create mode 100644 packages/qa/dogfood/test/object-designer-field-reorder.dogfood.test.ts diff --git a/.changeset/21790-metadata-core-ordered-fields-hash.md b/.changeset/21790-metadata-core-ordered-fields-hash.md new file mode 100644 index 00000000000..241c8de7c3f --- /dev/null +++ b/.changeset/21790-metadata-core-ordered-fields-hash.md @@ -0,0 +1,14 @@ +--- +'@objectstack/metadata-core': minor +--- + +The content hash keeps the order of an object's `fields`, so a pure field reorder is a new version instead of "no change" (#21790). + +Clause-②: no + +- **`canonicalize(value, type?)` and `hashSpec(value, type?)`** take the metadata type. For a type whose body has a map the spec declares ordered, that map keeps its insertion order in the canonical form. Today that is one map: `object.fields`, whose traversal order is the field order the platform presents. Every other map stays key-order independent, including the keys around `fields` and the keys inside each field definition. Called without a type, both functions return exactly what they returned before. +- **`orderedMapKeys(type?)`** is a new export. It returns the top-level keys of a `type` body whose map keeps its order (`['fields']` for `object`, `[]` otherwise). +- `InMemoryRepository` and the repository contract suite hash as `ref.type`. Invariant 4 now reads `item.hash === hashSpec(item.body, item.ref.type)`. +- **Stored hashes.** An object whose `fields` are already in sorted key order hashes exactly as before. Any other object hashes differently from the hash stored before this release. A stored hash is still that row's version token: `@objectstack/metadata-protocol` keeps it as written and compares content to decide whether a save changed anything. + +`minor` because two exports widen: a new parameter and a new function. No metadata key, accepted value, wire payload or error code changes. diff --git a/.changeset/21790-metadata-fs-ordered-fields-hash.md b/.changeset/21790-metadata-fs-ordered-fields-hash.md new file mode 100644 index 00000000000..b3ac7fdaa5b --- /dev/null +++ b/.changeset/21790-metadata-fs-ordered-fields-hash.md @@ -0,0 +1,9 @@ +--- +'@objectstack/metadata-fs': patch +--- + +`FileSystemRepository` hashes each item as its metadata type, so a reorder of an object's `fields` is a change (#21790). A reordered object is written, and an external edit that only reorders `fields` is reported as an update. + +Clause-②: no + +Event-log entries written before this release carry the order-blind hash. For an object whose `fields` are not in sorted key order, `get()` and `list()` no longer find that entry until the item next changes. They fall back to the defaults: no parent hash, sequence `0`, the filesystem actor, and the epoch timestamp. The body and the version hash are unaffected. diff --git a/.changeset/21790-metadata-protocol-ordered-fields-hash.md b/.changeset/21790-metadata-protocol-ordered-fields-hash.md new file mode 100644 index 00000000000..5dc062f6257 --- /dev/null +++ b/.changeset/21790-metadata-protocol-ordered-fields-hash.md @@ -0,0 +1,12 @@ +--- +'@objectstack/metadata-protocol': patch +--- + +Publishing a pure field reorder of an object now saves it. The object designer's drag-to-reorder used to answer success on publish, keep the old order and delete the draft (#21790). + +Clause-②: no + +- `SysMetadataRepository` hashes each body as its type, so a reorder of an object's `fields` is a content change. It is written, recorded in history and served by `GET /api/v1/meta/object/:name`. +- **Rows stored before this release** keep the `checksum` they were written with. That value is still the version token: reads return it, `If-Match` tokens are derived from it, and the optimistic lock compares against it, so upgrading raises no conflict. +- For an object, "is this save a no-op?" is decided by hashing the stored body under the current rule, not by comparing against the stored checksum. An identical re-save of an old row writes no history row and keeps its checksum. A reorder into sorted key order is written too. Its new hash equals the old order-blind checksum, so a checksum comparison would have dropped it. +- A save that changes nothing returns the stored checksum as its version, so the receipt's token is the one the next `If-Match` save must send. diff --git a/packages/metadata-protocol/src/sys-metadata-repository.contract.test.ts b/packages/metadata-protocol/src/sys-metadata-repository.contract.test.ts index a2be88e8c61..e2793c7151b 100644 --- a/packages/metadata-protocol/src/sys-metadata-repository.contract.test.ts +++ b/packages/metadata-protocol/src/sys-metadata-repository.contract.test.ts @@ -580,3 +580,146 @@ describe('SysMetadataRepository — close() terminates every live watcher (#1102 expect(viaClose).toEqual({ value: undefined, done: true }); }); }); + +/** + * #21790 — `ObjectSchema.fields` is a map whose KEY ORDER the spec declares to + * be the field order, and the content hash used to sort it away: a pure + * reorder hashed equal, `put`'s no-op short-circuit read the publish as + * "unchanged", and the object designer's reorder was dropped while the publish + * answered success. `hashSpec(body, ref.type)` now keeps that order (see + * `canonicalize` guarantee 8). + * + * The second block is the upgrade half. Every row written before the change + * carries the ORDER-BLIND stamp, and that stamp is the row's version token — + * readers were handed it, and the optimistic lock compares against it. So the + * stamp is accepted as written, and "is this write a no-op?" is answered from + * the stored CONTENT under the current rule. Both directions a stale stamp + * misfires in are pinned: an identical re-save must not record a change, and a + * reorder into sorted order — whose hash IS the stale stamp — must not be + * dropped. + */ +describe('SysMetadataRepository — a declared ordered map: object `fields` (#21790)', () => { + const ref = { org: 'system', type: 'object' as const, name: 'invoice' }; + // `object` is `allowRuntimeCreate`, not `allowOrgOverride`: the runtime door. + const intent = 'runtime-only' as const; + const FIELDS: Record = { + title: { type: 'text', label: 'Title' }, + amount: { type: 'currency', label: 'Amount' }, + due: { type: 'date', label: 'Due' }, + }; + const DECLARED = ['title', 'amount', 'due']; // deliberately not sorted + const MOVED = ['due', 'title', 'amount']; + const SORTED = ['amount', 'due', 'title']; + const body = (order: readonly string[]) => ({ + name: 'invoice', + label: 'Invoice', + fields: Object.fromEntries(order.map((k) => [k, FIELDS[k]])), + }); + + type Engine = ReturnType; + const activeRow = (engine: Engine) => + engine.rows().find((r) => r.type === 'object' && r.state === 'active')!; + const storedOrder = (engine: Engine) => + Object.keys((JSON.parse(activeRow(engine).metadata as string) as { fields: object }).fields); + const repoOn = (engine: Engine) => { + const repo = new SysMetadataRepository({ engine, organizationId: null, orgLabel: 'system' }); + created.push(repo); + return repo; + }; + + it('a pure reorder, saved as a draft and published, is written', async () => { + const engine = makeFakeEngine(); + const repo = repoOn(engine); + const first = await repo.put(ref, body(DECLARED), { parentVersion: null, actor: 't', intent }); + + await repo.put(ref, body(MOVED), { parentVersion: null, actor: 't', intent, state: 'draft' }); + const published = await repo.promoteDraft(ref, { actor: 't', intent }); + + expect(published.version).not.toBe(first.version); + expect(published.version).toBe(hashSpec(body(MOVED), 'object')); + expect(storedOrder(engine)).toEqual(MOVED); + expect(Object.keys((await repo.get(ref))!.body.fields as object)).toEqual(MOVED); + expect(engine.historyRows().map((r) => r.operation_type)).toEqual(['create', 'publish']); + }); + + it('…while an identical re-save is still the no-op it was', async () => { + const engine = makeFakeEngine(); + const repo = repoOn(engine); + const first = await repo.put(ref, body(DECLARED), { parentVersion: null, actor: 't', intent }); + const again = await repo.put(ref, body(DECLARED), { parentVersion: first.version, actor: 't', intent }); + + expect(again.version).toBe(first.version); + expect(again.seq).toBe(first.seq); + expect(engine.historyRows()).toHaveLength(1); + }); + + describe('a row stamped before #21790 — the order-blind stamp, accepted as written', () => { + async function legacyRow(order: readonly string[]) { + const engine = makeFakeEngine(); + const stored = body(order); + // The stamp every pre-#21790 write took: `hashSpec` with no type. + const stamp = hashSpec(stored); + const now = new Date().toISOString(); + await engine.insert('sys_metadata', { + type: 'object', + name: 'invoice', + organization_id: null, + package_id: null, + state: 'active', + version: 1, + metadata: JSON.stringify(stored), + checksum: stamp, + created_at: now, + updated_at: now, + }); + return { engine, repo: repoOn(engine), stamp }; + } + + it('the fixture is a stale stamp: the current rule hashes the same bytes differently', async () => { + const { stamp } = await legacyRow(DECLARED); + expect(stamp).not.toBe(hashSpec(body(DECLARED), 'object')); + }); + + it('reads back with the stamp it was written with', async () => { + const { repo, stamp } = await legacyRow(DECLARED); + expect((await repo.get(ref))!.hash).toBe(stamp); + }); + + it('an identical re-save: no conflict, no history row, the stored stamp handed back', async () => { + const { engine, repo, stamp } = await legacyRow(DECLARED); + const res = await repo.put(ref, body(DECLARED), { parentVersion: stamp, actor: 't', intent }); + + expect(res.version).toBe(stamp); + expect(engine.historyRows()).toEqual([]); + expect(activeRow(engine).checksum).toBe(stamp); + // The version handed back is the one the next write's lock accepts. + await expect( + repo.put(ref, body(MOVED), { parentVersion: res.version, actor: 't', intent }), + ).resolves.toMatchObject({ version: hashSpec(body(MOVED), 'object') }); + expect(storedOrder(engine)).toEqual(MOVED); + }); + + it('a reorder published through a draft is written, chained to the stored stamp', async () => { + const { engine, repo, stamp } = await legacyRow(DECLARED); + await repo.put(ref, body(MOVED), { parentVersion: null, actor: 't', intent, state: 'draft' }); + await repo.promoteDraft(ref, { actor: 't', intent }); + + expect(storedOrder(engine)).toEqual(MOVED); + const [publish] = engine.historyRows(); + expect(publish).toMatchObject({ operation_type: 'publish', previous_checksum: stamp }); + }); + + it('a reorder INTO sorted order is written, though its hash IS the stale stamp', async () => { + const { engine, repo, stamp } = await legacyRow(DECLARED); + // The trap: the order-blind stamp is the sorted form's hash, so a + // comparison of hash against stamp reads this reorder as "unchanged". + expect(hashSpec(body(SORTED), 'object')).toBe(stamp); + + await repo.put(ref, body(SORTED), { parentVersion: null, actor: 't', intent, state: 'draft' }); + await repo.promoteDraft(ref, { actor: 't', intent }); + + expect(storedOrder(engine)).toEqual(SORTED); + expect(engine.historyRows().map((r) => r.operation_type)).toEqual(['publish']); + }); + }); +}); diff --git a/packages/qa/dogfood/test/object-designer-field-reorder.dogfood.test.ts b/packages/qa/dogfood/test/object-designer-field-reorder.dogfood.test.ts new file mode 100644 index 00000000000..f41a275a7dc --- /dev/null +++ b/packages/qa/dogfood/test/object-designer-field-reorder.dogfood.test.ts @@ -0,0 +1,167 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// GOLDEN REGRESSION — #21790: "a field reorder made in the object designer +// survives draft save -> publish -> reload", exercised end-to-end through the +// real HTTP + metadata stack on the showcase. +// +// `ObjectSchema.fields` is a name-keyed map whose traversal order the spec +// declares to BE the field order. The content hash sorted every map, so a pure +// reorder hashed equal to the published row, `SysMetadataRepository.put`'s +// no-op short-circuit read the publish as "unchanged", and the draft was +// drained — the publish answered success, the served object kept the old +// order, and the draft was gone. Reproduced on 17.6.0 and the 17.7 +// pre-release. +// +// The second block is the upgrade half. Every object row written before the +// fix carries the ORDER-BLIND stamp, and that stamp is the version token its +// readers hold. It is accepted as written: an identical save must record no +// change, a receipt's token must still be honoured by `If-Match`, and a reorder +// INTO sorted order — whose current hash IS the stale stamp — must not be +// dropped. + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import showcaseStack from '@objectstack/example-showcase'; +import { hashSpec } from '@objectstack/metadata-core'; +import { bootStack, type VerifyStack } from '@objectstack/verify'; + +const OBJ = 'dogfood_field_reorder'; +const SYSTEM_CTX = { isSystem: true }; + +const FIELDS: Record = { + title: { type: 'text', label: 'Title' }, + amount: { type: 'number', label: 'Amount' }, + due: { type: 'date', label: 'Due' }, +}; +const DECLARED = ['title', 'amount', 'due']; // deliberately not sorted +const MOVED = ['due', 'title', 'amount']; // the designer drag +const SORTED = ['amount', 'due', 'title']; // the order the old hash sorted to + +/** The designer's document, its `fields` in `order`. */ +const objectBody = (order: readonly string[]) => ({ + name: OBJ, + label: 'Field Reorder', + // The runtime object door refuses an unauthored OWD at publish (#8310). + sharingModel: 'private', + fields: Object.fromEntries(order.map((k) => [k, FIELDS[k]])), +}); + +interface Row { + id: string; + metadata: string; + checksum: string | null; +} +interface Ql { + find(object: string, query: Record): Promise; + update(object: string, data: Record, opts: Record): Promise; +} + +describe('dogfood: an object designer field reorder publishes and reads back in the new order (#21790)', () => { + let stack: VerifyStack; + let token: string; + let ql: Ql; + + beforeAll(async () => { + stack = await bootStack(showcaseStack); + token = await stack.signIn(); + ql = (await stack.kernel.getServiceAsync('objectql')) as unknown as Ql; + }, 90_000); + + afterAll(async () => { + await stack?.stop(); + }); + + /** `GET /meta/object/:name` — the authored fields, in the order served. */ + const servedOrder = async (): Promise => { + const res = await stack.apiAs(token, 'GET', `/meta/object/${OBJ}`); + expect(res.status).toBe(200); + const body = (await res.json()) as { type?: string; name?: string; item?: { fields?: object } }; + expect(body).toMatchObject({ type: 'object', name: OBJ }); + // The registry may serve injected system columns beside the authored ones; + // the order under test is the authored fields' relative order. + return Object.keys(body.item?.fields ?? {}).filter((k) => k in FIELDS); + }; + const saveDraft = (order: readonly string[]) => + stack.apiAs(token, 'PUT', `/meta/object/${OBJ}?mode=draft`, objectBody(order)); + const publish = () => stack.apiAs(token, 'POST', `/meta/object/${OBJ}/publish`, {}); + const activeRow = async (): Promise => { + const rows = await ql.find('sys_metadata', { + where: { type: 'object', name: OBJ, state: 'active' }, + context: SYSTEM_CTX, + }); + expect(rows).toHaveLength(1); + return rows[0]!; + }; + const historyRows = async () => + (await ql.find('sys_metadata_history', { where: { type: 'object', name: OBJ }, context: SYSTEM_CTX })).length; + + describe('the designer round trip', () => { + it('creates and publishes the object in its declared order', async () => { + expect((await saveDraft(DECLARED)).status).toBe(200); + expect((await publish()).status).toBe(200); + expect(await servedOrder()).toEqual(DECLARED); + }); + + it('a pure reorder, saved as a draft and published, is served — and still served on reload', async () => { + expect((await saveDraft(MOVED)).status).toBe(200); + // The defect: this answered 200 and dropped the reorder. + expect((await publish()).status).toBe(200); + expect(await servedOrder()).toEqual(MOVED); + // The designer's reload is a second read of the same door. + expect(await servedOrder()).toEqual(MOVED); + // …and the stored row holds it, not just a registry entry. + const stored = JSON.parse((await activeRow()).metadata) as { fields: object }; + expect(Object.keys(stored.fields)).toEqual(MOVED); + }); + }); + + describe('an object row stamped before the fix (the order-blind stamp)', () => { + let legacyStamp: string; + let receipt: string; + + it('rewinds the stored stamp to the one the order-blind rule wrote', async () => { + const row = await activeRow(); + const stored = JSON.parse(row.metadata) as Record; + legacyStamp = hashSpec(stored); + // A real stale stamp: the current rule hashes these bytes differently. + expect(row.checksum).toBe(hashSpec(stored, 'object')); + expect(legacyStamp).not.toBe(row.checksum); + await ql.update('sys_metadata', { id: row.id, checksum: legacyStamp }, { context: SYSTEM_CTX }); + expect((await activeRow()).checksum).toBe(legacyStamp); + expect(await servedOrder()).toEqual(MOVED); + }); + + it('an identical save records no change and leaves the stamp as written', async () => { + const before = await historyRows(); + const stored = JSON.parse((await activeRow()).metadata) as Record; + const res = await stack.apiAs(token, 'PUT', `/meta/object/${OBJ}`, stored); + expect(res.status).toBe(200); + receipt = ((await res.json()) as { version: string }).version; + expect(typeof receipt).toBe('string'); + expect(await historyRows()).toBe(before); + expect((await activeRow()).checksum).toBe(legacyStamp); + }); + + it('the receipt\'s token is honoured by If-Match, and a reorder into sorted order is published', async () => { + // The trap: the stale stamp IS the sorted form's current hash. + const stored = JSON.parse((await activeRow()).metadata) as { fields: Record }; + const sorted = { ...stored, fields: Object.fromEntries(SORTED.map((k) => [k, stored.fields[k]])) }; + expect(hashSpec(sorted, 'object')).toBe(legacyStamp); + + const before = await historyRows(); + const res = await stack.api(`/meta/object/${OBJ}`, { + method: 'PUT', + headers: { + 'Content-Type': 'application/json', + Authorization: `Bearer ${token}`, + 'If-Match': `"${receipt}"`, + }, + body: JSON.stringify(sorted), + }); + expect(res.status).toBe(200); + expect(await servedOrder()).toEqual(SORTED); + expect(await historyRows()).toBe(before + 1); + const written = JSON.parse((await activeRow()).metadata) as { fields: object }; + expect(Object.keys(written.fields)).toEqual(SORTED); + }); + }); +}); From bc6102dddb6d54924c4ad4ca5a17a5f3fd0e44c5 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 5 Oct 2026 03:49:29 +0000 Subject: [PATCH 3/5] test(metadata-protocol): a draft save appends its own history row; assert the publish row by kind Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude --- .../src/sys-metadata-repository.contract.test.ts | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/packages/metadata-protocol/src/sys-metadata-repository.contract.test.ts b/packages/metadata-protocol/src/sys-metadata-repository.contract.test.ts index e2793c7151b..fd0c001b22d 100644 --- a/packages/metadata-protocol/src/sys-metadata-repository.contract.test.ts +++ b/packages/metadata-protocol/src/sys-metadata-repository.contract.test.ts @@ -639,7 +639,9 @@ describe('SysMetadataRepository — a declared ordered map: object `fields` (#21 expect(published.version).toBe(hashSpec(body(MOVED), 'object')); expect(storedOrder(engine)).toEqual(MOVED); expect(Object.keys((await repo.get(ref))!.body.fields as object)).toEqual(MOVED); - expect(engine.historyRows().map((r) => r.operation_type)).toEqual(['create', 'publish']); + // The active create, the draft's own row, and the publish — the write the + // order-blind hash used to skip. + expect(engine.historyRows().map((r) => r.operation_type)).toEqual(['create', 'create', 'publish']); }); it('…while an identical re-save is still the no-op it was', async () => { @@ -705,8 +707,8 @@ describe('SysMetadataRepository — a declared ordered map: object `fields` (#21 await repo.promoteDraft(ref, { actor: 't', intent }); expect(storedOrder(engine)).toEqual(MOVED); - const [publish] = engine.historyRows(); - expect(publish).toMatchObject({ operation_type: 'publish', previous_checksum: stamp }); + const publish = engine.historyRows().find((r) => r.operation_type === 'publish'); + expect(publish).toMatchObject({ previous_checksum: stamp }); }); it('a reorder INTO sorted order is written, though its hash IS the stale stamp', async () => { @@ -719,7 +721,8 @@ describe('SysMetadataRepository — a declared ordered map: object `fields` (#21 await repo.promoteDraft(ref, { actor: 't', intent }); expect(storedOrder(engine)).toEqual(SORTED); - expect(engine.historyRows().map((r) => r.operation_type)).toEqual(['publish']); + // The draft's own row, then the publish the stale stamp would have skipped. + expect(engine.historyRows().map((r) => r.operation_type)).toEqual(['create', 'publish']); }); }); }); From 187b5a0ad6960ed994c3a3326910e1d44209e484 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 5 Oct 2026 05:02:29 +0000 Subject: [PATCH 4/5] =?UTF-8?q?chore(changeset):=20#21790=20declares=20Cla?= =?UTF-8?q?use-=E2=91=A1=20yes=20--=20metadata-core=20widens=20a=20publish?= =?UTF-8?q?ed=20surface?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit orderedMapKeys is a new export and canonicalize/hashSpec take a new parameter, so the PR widens @objectstack/metadata-core's public surface. Levels unchanged: metadata-core minor, metadata-protocol and metadata-fs patch; the level axis accepts them under yes. Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude --- .changeset/21790-metadata-core-ordered-fields-hash.md | 2 +- .changeset/21790-metadata-fs-ordered-fields-hash.md | 2 +- .changeset/21790-metadata-protocol-ordered-fields-hash.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.changeset/21790-metadata-core-ordered-fields-hash.md b/.changeset/21790-metadata-core-ordered-fields-hash.md index 241c8de7c3f..8c947c9b72d 100644 --- a/.changeset/21790-metadata-core-ordered-fields-hash.md +++ b/.changeset/21790-metadata-core-ordered-fields-hash.md @@ -4,7 +4,7 @@ The content hash keeps the order of an object's `fields`, so a pure field reorder is a new version instead of "no change" (#21790). -Clause-②: no +Clause-②: yes - **`canonicalize(value, type?)` and `hashSpec(value, type?)`** take the metadata type. For a type whose body has a map the spec declares ordered, that map keeps its insertion order in the canonical form. Today that is one map: `object.fields`, whose traversal order is the field order the platform presents. Every other map stays key-order independent, including the keys around `fields` and the keys inside each field definition. Called without a type, both functions return exactly what they returned before. - **`orderedMapKeys(type?)`** is a new export. It returns the top-level keys of a `type` body whose map keeps its order (`['fields']` for `object`, `[]` otherwise). diff --git a/.changeset/21790-metadata-fs-ordered-fields-hash.md b/.changeset/21790-metadata-fs-ordered-fields-hash.md index b3ac7fdaa5b..564aa9f0248 100644 --- a/.changeset/21790-metadata-fs-ordered-fields-hash.md +++ b/.changeset/21790-metadata-fs-ordered-fields-hash.md @@ -4,6 +4,6 @@ `FileSystemRepository` hashes each item as its metadata type, so a reorder of an object's `fields` is a change (#21790). A reordered object is written, and an external edit that only reorders `fields` is reported as an update. -Clause-②: no +Clause-②: yes Event-log entries written before this release carry the order-blind hash. For an object whose `fields` are not in sorted key order, `get()` and `list()` no longer find that entry until the item next changes. They fall back to the defaults: no parent hash, sequence `0`, the filesystem actor, and the epoch timestamp. The body and the version hash are unaffected. diff --git a/.changeset/21790-metadata-protocol-ordered-fields-hash.md b/.changeset/21790-metadata-protocol-ordered-fields-hash.md index 5dc062f6257..75412518d80 100644 --- a/.changeset/21790-metadata-protocol-ordered-fields-hash.md +++ b/.changeset/21790-metadata-protocol-ordered-fields-hash.md @@ -4,7 +4,7 @@ Publishing a pure field reorder of an object now saves it. The object designer's drag-to-reorder used to answer success on publish, keep the old order and delete the draft (#21790). -Clause-②: no +Clause-②: yes - `SysMetadataRepository` hashes each body as its type, so a reorder of an object's `fields` is a content change. It is written, recorded in history and served by `GET /api/v1/meta/object/:name`. - **Rows stored before this release** keep the `checksum` they were written with. That value is still the version token: reads return it, `If-Match` tokens are derived from it, and the optimistic lock compares against it, so upgrading raises no conflict. From 046c425d62db1c00fade07cde87f632c1bd7f899 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 5 Oct 2026 05:07:57 +0000 Subject: [PATCH 5/5] docs(concepts): the metadata hash keeps an object's field order The canonical form sorts keys except in the maps the spec declares ordered (today an object's fields), put() still short-circuits on an unchanged body, and a row written before this release keeps its stored hash as its version token until its content next changes. Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude --- content/docs/concepts/metadata-lifecycle.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/concepts/metadata-lifecycle.mdx b/content/docs/concepts/metadata-lifecycle.mdx index c2eee1b41a4..e5e3270300f 100644 --- a/content/docs/concepts/metadata-lifecycle.mdx +++ b/content/docs/concepts/metadata-lifecycle.mdx @@ -146,7 +146,7 @@ The switches are operator-only where one install serves several organizations. T Every `put()` requires a `parentVersion` (or `null` for fresh creates). If the head has moved since the caller read it, the repository throws `ConflictError(ref, expectedParent, actualHead)` with code `METADATA_CONFLICT`. `protocol.ts` catches it at every write entry point (save, publish, rollback, overlay-delete) and re-throws an API error with code `metadata_conflict`, **HTTP 409 Conflict**, and the `expectedParent` / `actualHead` pair attached. Clients are expected to re-read the item and retry; the console does not yet surface a dedicated conflict-resolution UI. -The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON serialization of the body. Identical bodies produce identical hashes — `put()` short-circuits on no-op writes. +The hash is `sha256:` + 64-hex of a canonical JSON serialization of the body, with `undefined` values dropped. The canonical form sorts object keys, except in the maps the spec declares ordered. Today that is one map: an `object`'s `fields`, which keeps its declared order, so a pure reorder of fields is a new version. `put()` still short-circuits on a write whose body is unchanged. A row written before this release keeps its stored hash as its version token until its content next changes. ---