Skip to content

Commit e1790fd

Browse files
fix(metadata-core): a reorder of an object's fields is a content change, so the designer's publish is no longer dropped (#21790) (#21814)
Fixes #21790 Clause-②: yes ## What was wrong `ObjectSchema.fields` is a name-keyed map, and the spec declares its traversal order to be the field order. `canonicalize` sorted every map, so a pure reorder of `fields` hashed equal to the published row. `SysMetadataRepository.put` treats an equal hash as a no-op and skipped the write. `promoteDraft` then drained the draft anyway. So the publish answered success, the served object kept the old order, and the draft was gone. That is the card's reproduction. Under the reverse verification below, the new dogfood pin reproduces it: it reads `title, amount, due` back after publishing `due, title, amount`. ## The fix - **`@objectstack/metadata-core`**: `canonicalize(value, type?)` and `hashSpec(value, type?)` take the metadata type. A per-type table names the maps whose key order is content. Today it has one row, `object` maps to `fields`, and the table's docblock cites the spec text behind it. That map keeps its insertion order in the canonical form. Everything else stays key-order independent: the keys around `fields`, the keys inside each field definition, and every map of every other type. Without a type, the output is byte-identical to before. The table is exported read-only as `orderedMapKeys(type?)`. `InMemoryRepository` and the shared contract suite hash as `ref.type`, and invariant 4 is restated as `item.hash === hashSpec(item.body, item.ref.type)`. - **`@objectstack/metadata-protocol`**: `SysMetadataRepository` hashes as `ref.type` (in `put`, and in the `rowToItem` fallback for a null checksum). For a type with an ordered map, its no-op short-circuit compares content: it re-hashes the stored body under the current rule rather than trusting the stored stamp. A skipped write returns the stored stamp, which is the version every reader already holds (see H4). - **`@objectstack/metadata-fs`**: `FileSystemRepository` hashes as `ref.type` at all four sites (get, put, boot scan, watcher). Landing: the dispatch named `canonicalize.ts` and the `hashSpec` call site, and measurement kept it there. The type has to reach `canonicalize`, so the two other repository implementations that call `hashSpec` (`InMemoryRepository`, `FileSystemRepository`) pass it too. Otherwise the shared repository contract would hold two hash rules. No `packages/spec/src/**` edit: the order is already declared in spec prose (H2), and the table implements that declaration. `@objectstack/metadata-core` is graded `minor`: two exports widen (a new optional parameter and a new function). That follows the "WHICH LEVEL" rule in `pr-automation.yml`, under which a fix that widens an index is `minor`. Because the diff widens that published surface, the declaration is `Clause-②: yes`. The claim predicted `no` before the diff existed, and the seat's review corrected it. No payload key, accepted value or error code moves. `metadata-protocol` and `metadata-fs` are `patch`. ## H1: every place a content hash or stored checksum decides "unchanged" (at `18c2ddc1ec`) | site | compares | what equality drops | |---|---|---| | `metadata-protocol/src/sys-metadata-repository.ts:620` (`put`) | stored `checksum` vs `hashSpec(incoming)` | the row write, the history row and the watch event; it re-yields the stored item. **The defect site.** | | `sys-metadata-repository.ts:886` / `:972` (`promoteDraft`) | delegates to `put` above with `parentVersion` = the stored active stamp | the publish (through `:620`); the draft drain still runs, which is why "the draft is gone" | | `sys-metadata-repository.ts:613` / `:786` (`put` / `delete` lock) | caller `parentVersion` vs stored `checksum` | nothing; inequality throws `ConflictError` (a lock, not a no-op) | | `sys-metadata-repository.ts:1945` (`rowToItem`) | none; re-derives only when `checksum` is null | n/a (the read returns the stored stamp) | | `metadata-protocol/src/protocol.ts:16901` (`storedParentForToken`) | caller `If-Match` token vs keyed digest of the stored head | nothing; inequality answers 409 | | `protocol.ts:14082` (`getMetaItem` ETag) | `simpleHash` of the served JSON (order-sensitive already) vs `If-None-Match` | the body (304); not a write decision | | `metadata-core/src/in-memory-repository.ts:107` (`put`) | current head hash vs `hashSpec(incoming)` | the write and the event | | `metadata-fs/src/repository.ts:362` (`put`) / `:1006` (watcher) | head index (re-derived at boot) vs `hashSpec(body)` | the file write / the external-change event | | `metadata/src/loaders/database-loader.ts:1414` (`save`) / `:728` (history) | its own key-sorted `calculateChecksum`, a separate vocabulary | the write / the history row. Reached only through `MetadataManager.register` / `save`; the Studio save and publish path does not call them (no caller of `MetadataManager.register` in `packages/*/src`). Unchanged; see Acceptance notes. | | `objectql` | `src/` has no comparison. `scripts/dry-run-hash-compat.ts` compares stored `checksum` with type-blind `hashSpec(body)` in an offline audit probe | nothing; it reports. Unchanged; see Acceptance notes. | ## H2: maps a stored metadata body declares ordered | map | declared ordered? | in the table | |---|---|---| | `object.fields` | **yes**. `ObjectFieldGroupSchema` (`data/object.zod.ts` ~1080-1084): "the in-group display order equals the traversal order of `ObjectSchema.fields`", and ungrouped fields keep "their field declaration order". `fieldGroups` (~2090-2093): fields are displayed "in their `ObjectSchema.fields` declaration order". The `record` field type it is authored through (`data/field.zod.ts:90`, `object.form.ts:114`): "Insertion order = display order". `data/display-name.ts:50`: declaration order is `Object.keys` order, and the first title-eligible field "by declaration order" is picked | yes | | `object.listViews` | no order statement (rendered as tabs, keyed by view name) | no | | `ObjectExtension.fields` | no statement, and not a stored metadata type (code-only `defineObjectExtension`) | no | | nested `fields` records (`security/permission.zod.ts:877`, `system/translation.zod.ts:96/206`, flow node configs) | no statement | no | | inline locale maps (`I18nLabel`) | the resolver breaks ties by key insertion order (`ui/i18n-label-resolver.ts:44`, limbs 3 and 6, mirroring objectui's `pickLocalized`). That is a tie-break rule, not a declared authoring order, and these maps sit at arbitrary depths | no (raised in the report) | | `AnalyticsQuery.order` / `DatasetSelection.order` | declared ("applied by key in insertion order"), but these are request bodies, not stored metadata. Reports store `order` as an array because "JSON object key order is not a contract" (`ui/report.zod.ts:48`) | n/a | The only row is `object.fields`, which the spec already declares. No new spec declaration was needed. ## H3: where the order is kept `canonicalize` takes the metadata type. That is the least surface that keeps one hash rule. A pre-pass in `hashSpec`'s caller would leave `hashSpec(body)` disagreeing with the repository's hash, which breaks contract invariant 4, and every repository implementation would need its own copy. The type is in hand at every call site (`ref.type`), and an omitted type is exactly the old rule. Guarantees 5 (idempotence) and 7 (serialized-form identity, #7856) hold for an ordered map because `JSON.stringify` writes own keys in `Object.keys` order and `JSON.parse` recreates them in that order. Both are pinned as properties under the `object` type. ## H4: stored hashes after the change (measured) - **Does a stored checksum stop matching the recomputed hash?** Yes, for an object whose `fields` are not in sorted key order. Objects with sorted `fields` hash byte-identically (pinned). - **Read / ETag / If-Match / conflict:** unaffected. `rowToItem` returns the stored stamp. The ETag hashes the served document, not the checksum. `If-Match` tokens are keyed digests of the stored stamp, and the repository lock compares against the stored stamp. Nothing re-derives a stamp to judge a lock, so upgrading raises no conflict. Pinned end to end: a receipt token taken over a row with an old stamp is honoured by `If-Match`. - **Misfire 1 (handled):** with only the type-aware hash, an identical re-save of an old row would compare current hash against old stamp, see a difference, and write a history row recording no change. `storedBodyUnchanged` re-hashes the stored bytes under the current rule. The save is a no-op, and the stamp stays as written. - **Misfire 2 (handled):** a reorder into sorted key order hashes, under the current rule, to exactly the old order-blind stamp. A stamp comparison therefore drops it: the original defect, surviving on exactly the rows stored before the fix. The content comparison writes it. - **Boot re-derivation:** `SysMetadataRepository` re-derives nothing. `FileSystemRepository` re-derives its head index at boot under the new rule, and the watcher and resync compare new against new, so there are no spurious events. Its JSONL log keeps pre-change hashes, so provenance on `get` / `list` for such an object falls back to defaults until the item next changes. That is measured but not handled; see Acceptance notes. ## H5: the negative Pinned: swapping the keys around `fields`, the keys inside a field definition, and a map inside a field definition all hash equal under `object`. A property test holds every non-`fields` key order-independent. A `fields` map under a type with no row (`view`), and under no type, hashes equal across a reorder. ## Pins 1. **Dogfood** (`packages/qa/dogfood/test/object-designer-field-reorder.dogfood.test.ts`, showcase, real HTTP). It creates and publishes an object, saves a reorder as a draft, publishes it, and reads the new order back through `GET /api/v1/meta/object/:name` twice (the designer reload), plus the stored row. Then it rewinds the row to the order-blind stamp. An identical save records no history row and keeps the stamp. That save's receipt token is honoured by `If-Match`, and a reorder into sorted order is published. 2. **Hash** (`packages/metadata-core/test/canonicalize.test.ts`): the order half and the unordered half (H5), side by side. 3. **Guarantees 5 and 7** with the ordered map: properties under `object`, plus `toJSON` inside `fields` and on `fields` itself. 4. **H4 at the repository** (`packages/metadata-protocol/src/sys-metadata-repository.contract.test.ts`, on that file's existing engine double; no new double). The draft-and-publish reorder through `promoteDraft`. Over an old-stamp row: read, identical re-save, draft-and-publish reorder (`previous_checksum` = the old stamp), and the reorder into sorted order. ## Reverse verification (both run from committed state; every mutation through `scripts/ablation-replace.mjs`, `dist/` proven with `scripts/ablation-dist-preflight.mjs`) **A. Order-blind `fields` restored** (the `object` row deleted from the table). Pre-mutation `dist/` reading: marker `Object.freeze(["fields"])` present in 2 files. Mutate: anchor 1 to 0, blob `c2483ce615dd` to `169e16f653bf`. `metadata-core` rebuilt, preflight `--absent` green. - pin 2: **5 red / 38 green**. Red: the table row, "hashes a reordered `fields` map differently", "keeps the declared field order", "a reorder INTO sorted order is still a change", and the `toJSON`-on-`fields` case (it asserts the kept order). Green: every H5 case, both byte-identical cases, and both guarantee properties. - pin 1: **3 red / 2 green**. The reorder publish reads back `title, amount, due`, the card's defect. The stale-stamp precondition and the sorted-order publish are also red. The declared-order create and the identical save stay green. - pin 4: **5 red / 50 green** (every order-dependent case). - Restore: blob back to `c2483ce615dd`, `git diff HEAD` empty, rebuilt, marker present again, tree clean. **B. Content comparison reverted to a stamp comparison** (`storedBodyUnchanged`), with the type-aware hash kept. Mutate: anchor 1 to 0, plant 0 to 1, blob `013ffb59ed3d` to `195cf07493ae`. - pin 4: **2 red / 53 green**. Exactly "an identical re-save: no conflict, no history row" and "a reorder INTO sorted order is written". - pin 1, after `metadata-protocol` was rebuilt: **1 red / 4 green** ("an identical save records no change"). The type-check of the deliberately ill-typed mutation failed the d.ts step after the JS was emitted, and preflight proved the marker in 2 built JS files. - Restore: blob back to `013ffb59ed3d`, `git diff HEAD` empty, rebuilt, marker absent, tree clean. ## Local verification All readings were taken at `071959e91e` (this branch merged with `origin/main` `75ddcd1b41`) on a shared 4-core box. Lock seconds are shared-box figures. - **Build**: `turbo run build --filter=@objectstack/dogfood^...` plus the eight packages `check:dual-build-cjs-loads` reads: 71/71 tasks. - **`@objectstack/metadata-core`**: `test` 389/389 (18 files); `typecheck` exit 0. - **`@objectstack/metadata-fs`**: `test` 70/70 (10 files); `typecheck` exit 0. - **`@objectstack/metadata-protocol`**: `typecheck` exit 0, and the pin file 55/55. The full suite ran before the merge, at `bc6102dddb`: 19959 passed, 19 skipped (213 files passed, 3 skipped). Of this PR's packages, the merge touched only `metadata-core/src/protocol-handshake.ts`, a file this diff does not touch. - **`@objectstack/dogfood`**: the new file 5/5; `typecheck` exit 0. The rest of the dogfood suite is CI's. - **`@objectstack/objectql`** `src/dry-run-hash-compat.test.ts`, the one other `hashSpec` caller: 17/17. - **ESLint, narrowed to the 10 changed `.ts` files**: `eslint --no-inline-config --format json` read 10 files and reported 0 errors, 0 warnings and none ignored. Population: the config's `**/*.{ts,…}` blocks, which contain all 10. Invariance: the config enables no type-aware linting (no `parserOptions.project`), so this diff cannot move a verdict on any untouched file. - **Gates**: `dispatch-gates --commands`, re-derived at this head, gave 67 commands. Its artifact-roster block added 54 and the four symbol-anchor sweeps 4: 124 run, 121 exit 0. The `--ran` reconciliation reads 67 derived, 67 run, 0 NOT-MEASURED, 0 UNRUN. The 3 non-zero exits are exit 2, from gates that read pull-request context and judged nothing locally: `check-closing-target-claim`, `check-partof-closing-keyword` and `check-single-claim-paths`. Re-run with this body as `PR_BODY`, `check-partof-closing-keyword` is green. In that run, `check-changeset-no-major --event` (over the body as it then read) found no major and read `Clause-②: no`. See the re-run below for the corrected declaration. `check:engine-double-contract`: OK, no new double. - **Re-run at `6fe110968c`**, after the declaration was corrected to `Clause-②: yes` in this body and all three changesets, and `origin/main` `18c7dfd2e6` was merged (docs/qa only): `check-changeset-no-major --base origin/main --event` over this body exits 0. It finds no major, and its level axis reads that the PR declares clause-② `yes` and grades `@objectstack/metadata-core` `minor`; `metadata-fs` and `metadata-protocol` are `patch` and not refused. `check-adr-0087-registration` exits 0, reporting no declared-breaking changeset. `check:adr-symbol-anchors`, `check:scripts-symbol-anchors`, `check:spec-docblock-symbol-anchors` and `check:adr-anchors` all exit 0. The `metadata-core`, `metadata-protocol` and `metadata-fs` typechecks all exit 0. No source changed between `071959e91e` and this head, so the test readings above stand. - **NOT MEASURED here (CI's)**: `pnpm lint` over the repo, the workspace type-check lane, the Test Core shards, the rest of the Dogfood Regression Gate, and Temporal Conformance. ## Acceptance notes - **Not changed, observed:** `DatabaseLoader` (`packages/metadata`) keeps its own key-sorted `calculateChecksum` and skips a save on equality. It is a second hash vocabulary over the same `sys_metadata` table. It is reached only through `MetadataManager.register` / `save`, which no Studio save or publish path calls (no caller in `packages/*/src`). Carrier: none. - **Not changed, observed:** the offline probe `packages/objectql/scripts/dry-run-hash-compat.ts` (PR-10d.1) hashes type-blind. On an object row stamped by the new rule with unsorted `fields`, its `checksum_drift` arm fires. It is not published (`files` is `dist` only) and has no runner beyond its own test. Carrier: none. - **Measured, not handled:** `FileSystemRepository`'s JSONL log keeps pre-change hashes. For an object with unsorted `fields`, provenance on `get` / `list` falls back to defaults: no parent hash, seq `0`, the filesystem actor and the epoch timestamp. This lasts until the item next changes. Production consumer census: `MetadataManager.getRepository()` has no caller in `packages/*/src`, and the manager consumes only `watch()` events, which are unaffected. Stated in the `metadata-fs` changeset. - **Residue, filed as #21821:** `revertCommit`'s restore limb passes the commit item's stored type to `put` unfolded, so a legacy row stored under a manifest plural such as `objects` reaches the repository under that spelling. The ordered-`fields` rule applies only to the canonical `object` spelling. A reorder-only restore of such a row is therefore skipped as unchanged. Only rows written before the #4432 boundary fold are affected; every `/meta` door folds today. - `FileSystemRepository` takes the metadata type from the directory name verbatim (`parseItemPath`, no validation). An `objects/` directory would hash order-blind. Its `put` has no production caller; it is attached only as an event source. - **Carried over from this thread.** The seat's REWORK (comment 5988361846): the declaration is corrected to `Clause-②: yes` here and in the three changesets, and the plural-spelling measurement it asked for is answered in the os-dev-report on #21790. The docs drift check (comment 5988314939): `content/docs/concepts/metadata-lifecycle.mdx:149` says the hash is a sorted-keys serialization of the body. After this PR that is true except for `object.fields`, whose order is kept. Done at `046c425d62`: that paragraph now says the canonical form sorts keys except in the maps the spec declares ordered (today an `object`'s `fields`), that `put()` still short-circuits on an unchanged body, and that a row written before this release keeps its stored hash as its version token until its content next changes. `dispatch-gates --commands` for that path derives 41 gates. All 41 exit 0, `check:doc-authoring` included, and `--ran` reconciles 41 derived, 41 run, 0 NOT-MEASURED. `check:skill-examples` first answered PREREQUISITE NOT MET because `client-react` was unbuilt; it passed once that package was built. `origin/main` moved to `ba57588665` (service-settings and the spec crypto-provider contract) after the pre-push check, so this head does not merge it. - A reorder into sorted order over an old-stamp row is written with `checksum` equal to its `previous_checksum`: the two hashes are taken under different rules. `getByHash` on that value can find either history row. Objects are not `executionPinned`, so no pinned execution resolves through it. --- _Generated by [Claude Code](https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 088428f commit e1790fd

14 files changed

Lines changed: 713 additions & 34 deletions
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
'@objectstack/metadata-core': minor
3+
---
4+
5+
The content hash keeps the order of an object's `fields`, so a pure field reorder is a new version instead of "no change" (#21790).
6+
7+
Clause-②: yes
8+
9+
- **`canonicalize(value, type?)` and `hashSpec(value, type?)`** take the metadata type. For a type whose body has a map the spec declares ordered, that map keeps its insertion order in the canonical form. Today that is one map: `object.fields`, whose traversal order is the field order the platform presents. Every other map stays key-order independent, including the keys around `fields` and the keys inside each field definition. Called without a type, both functions return exactly what they returned before.
10+
- **`orderedMapKeys(type?)`** is a new export. It returns the top-level keys of a `type` body whose map keeps its order (`['fields']` for `object`, `[]` otherwise).
11+
- `InMemoryRepository` and the repository contract suite hash as `ref.type`. Invariant 4 now reads `item.hash === hashSpec(item.body, item.ref.type)`.
12+
- **Stored hashes.** An object whose `fields` are already in sorted key order hashes exactly as before. Any other object hashes differently from the hash stored before this release. A stored hash is still that row's version token: `@objectstack/metadata-protocol` keeps it as written and compares content to decide whether a save changed anything.
13+
14+
`minor` because two exports widen: a new parameter and a new function. No metadata key, accepted value, wire payload or error code changes.
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
---
2+
'@objectstack/metadata-fs': patch
3+
---
4+
5+
`FileSystemRepository` hashes each item as its metadata type, so a reorder of an object's `fields` is a change (#21790). A reordered object is written, and an external edit that only reorders `fields` is reported as an update.
6+
7+
Clause-②: yes
8+
9+
Event-log entries written before this release carry the order-blind hash. For an object whose `fields` are not in sorted key order, `get()` and `list()` no longer find that entry until the item next changes. They fall back to the defaults: no parent hash, sequence `0`, the filesystem actor, and the epoch timestamp. The body and the version hash are unaffected.
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
---
2+
'@objectstack/metadata-protocol': patch
3+
---
4+
5+
Publishing a pure field reorder of an object now saves it. The object designer's drag-to-reorder used to answer success on publish, keep the old order and delete the draft (#21790).
6+
7+
Clause-②: yes
8+
9+
- `SysMetadataRepository` hashes each body as its type, so a reorder of an object's `fields` is a content change. It is written, recorded in history and served by `GET /api/v1/meta/object/:name`.
10+
- **Rows stored before this release** keep the `checksum` they were written with. That value is still the version token: reads return it, `If-Match` tokens are derived from it, and the optimistic lock compares against it, so upgrading raises no conflict.
11+
- For an object, "is this save a no-op?" is decided by hashing the stored body under the current rule, not by comparing against the stored checksum. An identical re-save of an old row writes no history row and keeps its checksum. A reorder into sorted key order is written too. Its new hash equals the old order-blind checksum, so a checksum comparison would have dropped it.
12+
- A save that changes nothing returns the stored checksum as its version, so the receipt's token is the one the next `If-Match` save must send.

‎content/docs/concepts/metadata-lifecycle.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@ The switches are operator-only where one install serves several organizations. T
146146

147147
Every `put()` requires a `parentVersion` (or `null` for fresh creates). If the head has moved since the caller read it, the repository throws `ConflictError(ref, expectedParent, actualHead)` with code `METADATA_CONFLICT`. `protocol.ts` catches it at every write entry point (save, publish, rollback, overlay-delete) and re-throws an API error with code `metadata_conflict`, **HTTP 409 Conflict**, and the `expectedParent` / `actualHead` pair attached. Clients are expected to re-read the item and retry; the console does not yet surface a dedicated conflict-resolution UI.
148148

149-
The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON serialization of the body. Identical bodies produce identical hashes — `put()` short-circuits on no-op writes.
149+
The hash is `sha256:` + 64-hex of a canonical JSON serialization of the body, with `undefined` values dropped. The canonical form sorts object keys, except in the maps the spec declares ordered. Today that is one map: an `object`'s `fields`, which keeps its declared order, so a pure reorder of fields is a new version. `put()` still short-circuits on a write whose body is unchanged. A row written before this release keeps its stored hash as its version token until its content next changes.
150150

151151
---
152152

‎packages/metadata-core/src/canonicalize.ts‎

Lines changed: 104 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,8 @@
77
* Guarantees:
88
*
99
* 1. **Key order independence.** Object keys are sorted lexicographically.
10-
* `{a:1, b:2}` and `{b:2, a:1}` produce the same canonical form.
10+
* `{a:1, b:2}` and `{b:2, a:1}` produce the same canonical form — for
11+
* every map EXCEPT the ones guarantee 8 names, whose order is content.
1112
* 2. **Whitespace independence.** No incidental whitespace.
1213
* 3. **Type preservation.** `undefined` properties are dropped (matching
1314
* JSON.stringify), `null` is preserved, arrays preserve order.
@@ -40,6 +41,31 @@
4041
* are untouched by this: their canonical form is byte-identical to what
4142
* it was before, so no already-stored version changes meaning.
4243
*
44+
* 8. **Declared map order (#21790).** A map whose key order the spec
45+
* declares meaningful keeps its insertion order instead of being sorted,
46+
* so a pure reorder of it is a change. Which maps those are is a fact
47+
* about the metadata TYPE, so it applies only when the caller names the
48+
* type (`canonicalize(body, type)`); see {@link orderedMapKeys} for the
49+
* table and the spec text behind each row. Everything else in the body —
50+
* the keys AROUND an ordered map and the keys INSIDE each of its values —
51+
* stays key-order independent (guarantee 1). With no type, or a type with
52+
* no row, the canonical form is byte-identical to what it was before.
53+
*
54+
* Guarantees 5 and 7 hold unchanged: `JSON.stringify` emits own keys in
55+
* the order `Object.keys` reports them and `JSON.parse` recreates them in
56+
* that order, so the order kept here survives every serialisation round
57+
* trip, and the canonical output of an ordered map re-canonicalises to
58+
* itself.
59+
*
60+
* The cost is the hashes already stored: a body whose ordered map is not
61+
* already in sorted key order hashes differently under this rule than
62+
* under the old one. A stamp written before #21790 still names its row —
63+
* it is the row's version token, and the optimistic lock compares a
64+
* caller's token with that stored stamp, never with a re-derivation — so
65+
* the comparison that needed teaching is "is this write a no-op?", which
66+
* must compare CONTENT under the current rule rather than a stored stamp
67+
* taken under an older one (`SysMetadataRepository.put`).
68+
*
4369
* Non-goals (deliberately not supported):
4470
*
4571
* - Functions and symbols. These have no canonical JSON form. Callers must
@@ -54,10 +80,57 @@
5480

5581
import { createHash } from 'node:crypto';
5682

57-
/** Stable JSON serialisation. See module-level doc for guarantees. */
58-
export function canonicalize(value: unknown): string {
83+
const NO_ORDERED_MAPS: readonly string[] = Object.freeze([]);
84+
85+
/**
86+
* Guarantee 8's table — per metadata type, the top-level keys of its body
87+
* whose value is a map with MEANINGFUL KEY ORDER.
88+
*
89+
* A row is owed only where `packages/spec` declares the order; ⛔ never an
90+
* order inferred from what a renderer happens to do with a map, because a
91+
* row here turns every reorder of that map into a new version.
92+
*
93+
* - `object` → `fields`. `ObjectSchema.fields` is a name-keyed record whose
94+
* traversal order IS the field order the platform presents: the
95+
* `ObjectFieldGroupSchema` contract (`data/object.zod.ts`) sets "the
96+
* in-group display order equals the traversal order of
97+
* `ObjectSchema.fields`" and lays ungrouped fields out "preserving their
98+
* field declaration order"; `fieldGroups` repeats it ("fields are
99+
* displayed in their `ObjectSchema.fields` declaration order"); the
100+
* `record` field type it is authored through declares "Insertion order =
101+
* display order" (`data/field.zod.ts`); and the display-name derivation
102+
* picks the first title-eligible field "by declaration order"
103+
* (`data/display-name.ts`). A hash blind to that order called a pure
104+
* reorder "no change", and the object designer's publish was dropped.
105+
*
106+
* Keyed by the canonical (singular) type name the repositories write under.
107+
* A `Map`, so a type name that happens to be an `Object.prototype` member
108+
* reads as "no row" rather than as an inherited function.
109+
*/
110+
const ORDERED_MAPS: ReadonlyMap<string, readonly string[]> = new Map([
111+
['object', Object.freeze(['fields'])],
112+
]);
113+
114+
/**
115+
* The top-level keys of a `type` body whose map value keeps its key order in
116+
* the canonical form (guarantee 8). Empty when `type` is omitted or has no
117+
* declared ordered map — the case in which canonicalization is exactly the
118+
* type-blind form it has always been.
119+
*/
120+
export function orderedMapKeys(type?: string): readonly string[] {
121+
return (type === undefined ? undefined : ORDERED_MAPS.get(type)) ?? NO_ORDERED_MAPS;
122+
}
123+
124+
/**
125+
* Stable JSON serialisation. See module-level doc for guarantees.
126+
*
127+
* @param type The metadata type of `value`, when it is a metadata body.
128+
* Decides which maps keep their order (guarantee 8); omit it for
129+
* a value that is not one, which sorts every map.
130+
*/
131+
export function canonicalize(value: unknown, type?: string): string {
59132
// `''` is the key `JSON.stringify` hands a root-position `toJSON`.
60-
return JSON.stringify(normalise(value, ''));
133+
return JSON.stringify(normalise(value, '', orderedMapKeys(type), false));
61134
}
62135

63136
/**
@@ -83,8 +156,19 @@ function resolveToJson(value: unknown, key: string): unknown {
83156
* @param key The position's key — `''` at the root, the property name
84157
* inside an object, the stringified index inside an array.
85158
* Passed to `toJSON` because `JSON.stringify` passes it.
159+
* @param orderedChildren Keys of THIS position's object whose values keep
160+
* their own key order (guarantee 8). Non-empty only at the root;
161+
* every deeper position receives none.
162+
* @param keepOwnOrder Whether THIS position, when it is an object, keeps its
163+
* keys in insertion order instead of sorting them. Applies to
164+
* this level only — its values are normalised as usual.
86165
*/
87-
function normalise(rawValue: unknown, key: string): unknown {
166+
function normalise(
167+
rawValue: unknown,
168+
key: string,
169+
orderedChildren: readonly string[],
170+
keepOwnOrder: boolean,
171+
): unknown {
88172
const value = resolveToJson(rawValue, key);
89173
if (value === null) return null;
90174
if (typeof value === 'undefined') return undefined; // caller-dropped
@@ -107,16 +191,21 @@ function normalise(rawValue: unknown, key: string): unknown {
107191
// the second argument, which is this function's `key` parameter. The
108192
// index is the right key, but it has to arrive as the string
109193
// `JSON.stringify` would hand a `toJSON`, not as a number.
110-
return value.map((element, index) => normalise(element, String(index)));
194+
return value.map((element, index) =>
195+
normalise(element, String(index), NO_ORDERED_MAPS, false),
196+
);
111197
}
112198

113-
// Plain object: sort keys, drop undefineds.
199+
// Plain object: sort keys (unless this is a declared ordered map), drop
200+
// undefineds.
114201
if (typeof value === 'object') {
115202
const obj = value as Record<string, unknown>;
116203
const out: Record<string, unknown> = {};
117-
const keys = Object.keys(obj).sort();
204+
// `Object.keys` order is the order `JSON.stringify` writes, so keeping it
205+
// is what keeps guarantee 7 for an ordered map.
206+
const keys = keepOwnOrder ? Object.keys(obj) : Object.keys(obj).sort();
118207
for (const k of keys) {
119-
const v = normalise(obj[k], k);
208+
const v = normalise(obj[k], k, NO_ORDERED_MAPS, orderedChildren.includes(k));
120209
if (typeof v === 'undefined') continue;
121210
out[k] = v;
122211
}
@@ -129,9 +218,13 @@ function normalise(rawValue: unknown, key: string): unknown {
129218
/**
130219
* Compute the canonical sha256 hash of a spec, returned as
131220
* `"sha256:<64-hex>"`. Equal hashes imply equal canonical forms.
221+
*
222+
* @param type The metadata type of `value` — a repository passes its
223+
* `ref.type`, so a reorder of a declared ordered map is a new
224+
* version (guarantee 8). Omitted, every map is order-blind.
132225
*/
133-
export function hashSpec(value: unknown): string {
134-
const json = canonicalize(value);
226+
export function hashSpec(value: unknown, type?: string): string {
227+
const json = canonicalize(value, type);
135228
const digest = createHash('sha256').update(json, 'utf8').digest('hex');
136229
return `sha256:${digest}`;
137230
}

‎packages/metadata-core/src/contract-suite.ts‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
* 1. Atomic put
1313
* 2. Monotonic seq per branch
1414
* 3. Optimistic locking (ConflictError)
15-
* 4. Canonical hashing (hash === hashSpec(body))
15+
* 4. Canonical hashing (hash === hashSpec(body, ref.type))
1616
* 5. Event ordering (monotonic seq, no gaps)
1717
* 6. Resumability (a numeric `since` replays; a bare `watch(filter)` is
1818
* owed live events only)
@@ -204,7 +204,7 @@ export function runRepositoryContractTests(
204204
const ref = refOf();
205205
const res = await repo.put(ref, spec('hello'), { parentVersion: null, actor: 'tester' });
206206
expect(res.version).toMatch(/^sha256:[0-9a-f]{64}$/);
207-
expect(res.version).toBe(hashSpec(spec('hello')));
207+
expect(res.version).toBe(hashSpec(spec('hello'), ref.type));
208208
expect(res.seq).toBeGreaterThan(0);
209209
expect(res.item.parentHash).toBeNull();
210210
expect(res.item.authoredBy).toBe('tester');
@@ -227,12 +227,12 @@ export function runRepositoryContractTests(
227227
expect(b.seq).toBeGreaterThan(a.seq);
228228
});
229229

230-
it('canonical hash invariant: item.hash === hashSpec(item.body)', async () => {
230+
it('canonical hash invariant: item.hash === hashSpec(item.body, item.ref.type)', async () => {
231231
const repo = await factory();
232232
const ref = refOf();
233233
await repo.put(ref, { z: 1, a: 2, m: [3, 1, 2] }, { parentVersion: null, actor: 't' });
234234
const got = await repo.get(ref);
235-
expect(got!.hash).toBe(hashSpec(got!.body));
235+
expect(got!.hash).toBe(hashSpec(got!.body, got!.ref.type));
236236
});
237237

238238
// ── #7856 — the version identifies the STORED BYTES ───────────
@@ -267,7 +267,7 @@ export function runRepositoryContractTests(
267267
const got = await repo.get(ref);
268268
expect(got).not.toBeNull();
269269
expect(got!.hash).toBe(put.version);
270-
expect(got!.hash).toBe(hashSpec(got!.body));
270+
expect(got!.hash).toBe(hashSpec(got!.body, got!.ref.type));
271271
});
272272

273273
it(`re-putting the same spec is a no-op — ${shape.label}`, async () => {

‎packages/metadata-core/src/in-memory-repository.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,8 @@ export class InMemoryRepository implements MetadataRepository {
101101
throw new ConflictError(ref, opts.parentVersion ?? null, currentHead);
102102
}
103103

104-
const hash = hashSpec(spec);
104+
// [#21790] Hashed as its type — a declared ordered map keeps its order.
105+
const hash = hashSpec(spec, ref.type);
105106

106107
// No-op write — same content. Still consumes nothing; no event emitted.
107108
if (current && current.hash === hash) {

‎packages/metadata-core/src/repository.ts‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,12 @@
2121
* scoped to a single org may treat the entire repo as one log.)
2222
* 3. **Optimistic locking.** `put` and `delete` throw `ConflictError`
2323
* when `parentVersion` does not match the current HEAD.
24-
* 4. **Canonical hashing.** `item.hash === hashSpec(item.body)` — always.
24+
* 4. **Canonical hashing.** `item.hash === hashSpec(item.body, item.ref.type)`
25+
* — always. The type decides which maps keep their key order
26+
* (`canonicalize` guarantee 8). One stated exception, owned by
27+
* `SysMetadataRepository`: a row stamped before its type's canonical form
28+
* changed keeps the stamp it was written with until its content next
29+
* changes — that stamp is its version token.
2530
* 5. **Event ordering.** Subscribers to `watch()` receive events in
2631
* monotonically-increasing `seq` order with no gaps.
2732
* 6. **Resumability, and where it stops.** `watch(_, since)` called with a

‎packages/metadata-core/src/types.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -91,13 +91,13 @@ export function refKey(ref: Pick<MetaRef, 'org' | 'type' | 'name'>): string {
9191
* Full metadata item as stored / returned by the Repository.
9292
*
9393
* `body` is the **canonical, Zod-normalised** spec (with defaults filled
94-
* in). `hash` is `sha256(canonicalize(body))`. Equal hashes imply equal
94+
* in). `hash` is `sha256(canonicalize(body, ref.type))`. Equal hashes imply equal
9595
* specs.
9696
*/
9797
export const MetadataItemSchema = z.object({
9898
ref: MetaRefSchema,
9999
body: z.record(z.string(), z.unknown()).describe('Canonical Zod-normalised spec'),
100-
hash: z.string().regex(/^sha256:[0-9a-f]{64}$/).describe('sha256(canonicalize(body))'),
100+
hash: z.string().regex(/^sha256:[0-9a-f]{64}$/).describe('sha256(canonicalize(body, ref.type))'),
101101
parentHash: z.string().nullable().describe('Hash this version was derived from; null for first version'),
102102
authoredBy: z.string().nullable().describe('Identity of the writer (user id, "cli", "ai:claude", …); null = system-initiated, no actor'),
103103
authoredAt: z.string().describe('ISO-8601 timestamp'),

0 commit comments

Comments
 (0)