Repository navigation
fix(metadata-core): a reorder of an object's fields is a content change, so the designer's publish is no longer dropped (#21790) - #21814
Conversation
The canonical hash sorted every map, so a pure reorder of `ObjectSchema.fields` -- whose traversal order the spec declares to be the display order -- hashed equal, the repository's no-op short-circuit read the publish as unchanged, and the object designer's reorder was dropped while the publish answered success. `canonicalize` / `hashSpec` take the metadata type; a per-type table names the maps whose key order is content (today `object.fields`). Every other map, and every key inside a field definition, stays key-order independent, and a call without a type is byte-identical to before. Each repository hashes as `ref.type`. `SysMetadataRepository.put` decides "unchanged" for a type with an ordered map by re-hashing the stored body under the current rule, so a row stamped before this change neither writes a no-change history row on an identical re-save nor drops a reorder into sorted order; a skipped write hands back the stored stamp, the version token readers already hold. Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…s stored before it - dogfood: an object-designer reorder saves as a draft, publishes and reads back in the new order through `GET /meta/object/:name`; then, over a row rewound to the order-blind stamp, an identical save records no change and keeps the stamp, its receipt token is honoured by If-Match, and a reorder into sorted order -- whose hash IS the stale stamp -- is published. - SysMetadataRepository: the same two halves at the repository, through `promoteDraft`, on the contract file's engine double. - changesets for the three moved packages. Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…sert the publish row by kind Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…dered-fields-hash
📓 Docs Drift CheckThis PR changes 3 package(s): 1 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
What this run could not see
Coarse fallback — 13 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 50c4601d4e8e193ad185ec6b05f5ebb6b7e21063 && git checkout 50c4601d4e8e193ad185ec6b05f5ebb6b7e21063
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin ba57588665fd846eb95b682b3660f52a697370a9 046c425d62db1c00fade07cde87f632c1bd7f899 && git checkout -B drift-repro ba57588665fd846eb95b682b3660f52a697370a9 && git merge --no-ff 046c425d62db1c00fade07cde87f632c1bd7f899
node scripts/docs-audit/affected-docs.mjs --json ba57588665fd846eb95b682b3660f52a697370a9
|
REWORK (seat review, narrow) — PR #21814 at head
|
…s a published surface orderedMapKeys is a new export and canonicalize/hashSpec take a new parameter, so the PR widens @objectstack/metadata-core's public surface. Levels unchanged: metadata-core minor, metadata-protocol and metadata-fs patch; the level axis accepts them under yes. Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…dered-fields-hash
The canonical form sorts keys except in the maps the spec declares ordered (today an object's fields), put() still short-circuits on an unchanged body, and a row written before this release keeps its stored hash as its version token until its content next changes. Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
ACCEPT (seat review) — PR #21814 at head
|
Contract reviewServed-tier: Inputs: card #21790 (body, comments 5987334318, 5987515805, 5988341325, 5988409457, 5988464990, 5988540414), PR #21814 (body, 14-file list, net diff against the merge-base Gate verdicts on the head: every completed check-run is ① Derived judgmentsPublic surface of
Docs: Done-when of the card: "a pure reorder is hashed as a change" holds ( ② Semver level
③ Boundary flags
Implemented-by: VERDICT: PASS Adopted by Generated by Claude Code |
Fixes #21790
Clause-②: yes
What was wrong
ObjectSchema.fieldsis a name-keyed map, and the spec declares its traversal order to be the field order.canonicalizesorted every map, so a pure reorder offieldshashed equal to the published row.SysMetadataRepository.puttreats an equal hash as a no-op and skipped the write.promoteDraftthen drained the draft anyway. So the publish answered success, the served object kept the old order, and the draft was gone. That is the card's reproduction. Under the reverse verification below, the new dogfood pin reproduces it: it readstitle, amount, dueback after publishingdue, title, amount.The fix
@objectstack/metadata-core:canonicalize(value, type?)andhashSpec(value, type?)take the metadata type. A per-type table names the maps whose key order is content. Today it has one row,objectmaps tofields, and the table's docblock cites the spec text behind it. That map keeps its insertion order in the canonical form. Everything else stays key-order independent: the keys aroundfields, the keys inside each field definition, and every map of every other type. Without a type, the output is byte-identical to before. The table is exported read-only asorderedMapKeys(type?).InMemoryRepositoryand the shared contract suite hash asref.type, and invariant 4 is restated asitem.hash === hashSpec(item.body, item.ref.type).@objectstack/metadata-protocol:SysMetadataRepositoryhashes asref.type(input, and in therowToItemfallback for a null checksum). For a type with an ordered map, its no-op short-circuit compares content: it re-hashes the stored body under the current rule rather than trusting the stored stamp. A skipped write returns the stored stamp, which is the version every reader already holds (see H4).@objectstack/metadata-fs:FileSystemRepositoryhashes asref.typeat all four sites (get, put, boot scan, watcher).Landing: the dispatch named
canonicalize.tsand thehashSpeccall site, and measurement kept it there. The type has to reachcanonicalize, so the two other repository implementations that callhashSpec(InMemoryRepository,FileSystemRepository) pass it too. Otherwise the shared repository contract would hold two hash rules. Nopackages/spec/src/**edit: the order is already declared in spec prose (H2), and the table implements that declaration.@objectstack/metadata-coreis gradedminor: two exports widen (a new optional parameter and a new function). That follows the "WHICH LEVEL" rule inpr-automation.yml, under which a fix that widens an index isminor. Because the diff widens that published surface, the declaration isClause-②: yes. The claim predictednobefore the diff existed, and the seat's review corrected it. No payload key, accepted value or error code moves.metadata-protocolandmetadata-fsarepatch.H1: every place a content hash or stored checksum decides "unchanged" (at
18c2ddc1ec)metadata-protocol/src/sys-metadata-repository.ts:620(put)checksumvshashSpec(incoming)sys-metadata-repository.ts:886/:972(promoteDraft)putabove withparentVersion= the stored active stamp:620); the draft drain still runs, which is why "the draft is gone"sys-metadata-repository.ts:613/:786(put/deletelock)parentVersionvs storedchecksumConflictError(a lock, not a no-op)sys-metadata-repository.ts:1945(rowToItem)checksumis nullmetadata-protocol/src/protocol.ts:16901(storedParentForToken)If-Matchtoken vs keyed digest of the stored headprotocol.ts:14082(getMetaItemETag)simpleHashof the served JSON (order-sensitive already) vsIf-None-Matchmetadata-core/src/in-memory-repository.ts:107(put)hashSpec(incoming)metadata-fs/src/repository.ts:362(put) /:1006(watcher)hashSpec(body)metadata/src/loaders/database-loader.ts:1414(save) /:728(history)calculateChecksum, a separate vocabularyMetadataManager.register/save; the Studio save and publish path does not call them (no caller ofMetadataManager.registerinpackages/*/src). Unchanged; see Acceptance notes.objectqlsrc/has no comparison.scripts/dry-run-hash-compat.tscompares storedchecksumwith type-blindhashSpec(body)in an offline audit probeH2: maps a stored metadata body declares ordered
object.fieldsObjectFieldGroupSchema(data/object.zod.ts~1080-1084): "the in-group display order equals the traversal order ofObjectSchema.fields", and ungrouped fields keep "their field declaration order".fieldGroups(~2090-2093): fields are displayed "in theirObjectSchema.fieldsdeclaration order". Therecordfield type it is authored through (data/field.zod.ts:90,object.form.ts:114): "Insertion order = display order".data/display-name.ts:50: declaration order isObject.keysorder, and the first title-eligible field "by declaration order" is pickedobject.listViewsObjectExtension.fieldsdefineObjectExtension)fieldsrecords (security/permission.zod.ts:877,system/translation.zod.ts:96/206, flow node configs)I18nLabel)ui/i18n-label-resolver.ts:44, limbs 3 and 6, mirroring objectui'spickLocalized). That is a tie-break rule, not a declared authoring order, and these maps sit at arbitrary depthsAnalyticsQuery.order/DatasetSelection.orderorderas an array because "JSON object key order is not a contract" (ui/report.zod.ts:48)The only row is
object.fields, which the spec already declares. No new spec declaration was needed.H3: where the order is kept
canonicalizetakes the metadata type. That is the least surface that keeps one hash rule. A pre-pass inhashSpec's caller would leavehashSpec(body)disagreeing with the repository's hash, which breaks contract invariant 4, and every repository implementation would need its own copy. The type is in hand at every call site (ref.type), and an omitted type is exactly the old rule. Guarantees 5 (idempotence) and 7 (serialized-form identity, #7856) hold for an ordered map becauseJSON.stringifywrites own keys inObject.keysorder andJSON.parserecreates them in that order. Both are pinned as properties under theobjecttype.H4: stored hashes after the change (measured)
fieldsare not in sorted key order. Objects with sortedfieldshash byte-identically (pinned).rowToItemreturns the stored stamp. The ETag hashes the served document, not the checksum.If-Matchtokens are keyed digests of the stored stamp, and the repository lock compares against the stored stamp. Nothing re-derives a stamp to judge a lock, so upgrading raises no conflict. Pinned end to end: a receipt token taken over a row with an old stamp is honoured byIf-Match.storedBodyUnchangedre-hashes the stored bytes under the current rule. The save is a no-op, and the stamp stays as written.SysMetadataRepositoryre-derives nothing.FileSystemRepositoryre-derives its head index at boot under the new rule, and the watcher and resync compare new against new, so there are no spurious events. Its JSONL log keeps pre-change hashes, so provenance onget/listfor such an object falls back to defaults until the item next changes. That is measured but not handled; see Acceptance notes.H5: the negative
Pinned: swapping the keys around
fields, the keys inside a field definition, and a map inside a field definition all hash equal underobject. A property test holds every non-fieldskey order-independent. Afieldsmap under a type with no row (view), and under no type, hashes equal across a reorder.Pins
packages/qa/dogfood/test/object-designer-field-reorder.dogfood.test.ts, showcase, real HTTP). It creates and publishes an object, saves a reorder as a draft, publishes it, and reads the new order back throughGET /api/v1/meta/object/:nametwice (the designer reload), plus the stored row. Then it rewinds the row to the order-blind stamp. An identical save records no history row and keeps the stamp. That save's receipt token is honoured byIf-Match, and a reorder into sorted order is published.packages/metadata-core/test/canonicalize.test.ts): the order half and the unordered half (H5), side by side.object, plustoJSONinsidefieldsand onfieldsitself.packages/metadata-protocol/src/sys-metadata-repository.contract.test.ts, on that file's existing engine double; no new double). The draft-and-publish reorder throughpromoteDraft. Over an old-stamp row: read, identical re-save, draft-and-publish reorder (previous_checksum= the old stamp), and the reorder into sorted order.Reverse verification (both run from committed state; every mutation through
scripts/ablation-replace.mjs,dist/proven withscripts/ablation-dist-preflight.mjs)A. Order-blind
fieldsrestored (theobjectrow deleted from the table). Pre-mutationdist/reading: markerObject.freeze(["fields"])present in 2 files. Mutate: anchor 1 to 0, blobc2483ce615ddto169e16f653bf.metadata-corerebuilt, preflight--absentgreen.fieldsmap differently", "keeps the declared field order", "a reorder INTO sorted order is still a change", and thetoJSON-on-fieldscase (it asserts the kept order). Green: every H5 case, both byte-identical cases, and both guarantee properties.title, amount, due, the card's defect. The stale-stamp precondition and the sorted-order publish are also red. The declared-order create and the identical save stay green.c2483ce615dd,git diff HEADempty, rebuilt, marker present again, tree clean.B. Content comparison reverted to a stamp comparison (
storedBodyUnchanged), with the type-aware hash kept. Mutate: anchor 1 to 0, plant 0 to 1, blob013ffb59ed3dto195cf07493ae.metadata-protocolwas rebuilt: 1 red / 4 green ("an identical save records no change"). The type-check of the deliberately ill-typed mutation failed the d.ts step after the JS was emitted, and preflight proved the marker in 2 built JS files.013ffb59ed3d,git diff HEADempty, rebuilt, marker absent, tree clean.Local verification
All readings were taken at
071959e91e(this branch merged withorigin/main75ddcd1b41) on a shared 4-core box. Lock seconds are shared-box figures.turbo run build --filter=@objectstack/dogfood^...plus the eight packagescheck:dual-build-cjs-loadsreads: 71/71 tasks.@objectstack/metadata-core:test389/389 (18 files);typecheckexit 0.@objectstack/metadata-fs:test70/70 (10 files);typecheckexit 0.@objectstack/metadata-protocol:typecheckexit 0, and the pin file 55/55. The full suite ran before the merge, atbc6102dddb: 19959 passed, 19 skipped (213 files passed, 3 skipped). Of this PR's packages, the merge touched onlymetadata-core/src/protocol-handshake.ts, a file this diff does not touch.@objectstack/dogfood: the new file 5/5;typecheckexit 0. The rest of the dogfood suite is CI's.@objectstack/objectqlsrc/dry-run-hash-compat.test.ts, the one otherhashSpeccaller: 17/17..tsfiles:eslint --no-inline-config --format jsonread 10 files and reported 0 errors, 0 warnings and none ignored. Population: the config's**/*.{ts,…}blocks, which contain all 10. Invariance: the config enables no type-aware linting (noparserOptions.project), so this diff cannot move a verdict on any untouched file.dispatch-gates --commands, re-derived at this head, gave 67 commands. Its artifact-roster block added 54 and the four symbol-anchor sweeps 4: 124 run, 121 exit 0. The--ranreconciliation reads 67 derived, 67 run, 0 NOT-MEASURED, 0 UNRUN. The 3 non-zero exits are exit 2, from gates that read pull-request context and judged nothing locally:check-closing-target-claim,check-partof-closing-keywordandcheck-single-claim-paths. Re-run with this body asPR_BODY,check-partof-closing-keywordis green. In that run,check-changeset-no-major --event(over the body as it then read) found no major and readClause-②: no. See the re-run below for the corrected declaration.check:engine-double-contract: OK, no new double.6fe110968c, after the declaration was corrected toClause-②: yesin this body and all three changesets, andorigin/main18c7dfd2e6was merged (docs/qa only):check-changeset-no-major --base origin/main --eventover this body exits 0. It finds no major, and its level axis reads that the PR declares clause-②yesand grades@objectstack/metadata-coreminor;metadata-fsandmetadata-protocolarepatchand not refused.check-adr-0087-registrationexits 0, reporting no declared-breaking changeset.check:adr-symbol-anchors,check:scripts-symbol-anchors,check:spec-docblock-symbol-anchorsandcheck:adr-anchorsall exit 0. Themetadata-core,metadata-protocolandmetadata-fstypechecks all exit 0. No source changed between071959e91eand this head, so the test readings above stand.pnpm lintover the repo, the workspace type-check lane, the Test Core shards, the rest of the Dogfood Regression Gate, and Temporal Conformance.Acceptance notes
DatabaseLoader(packages/metadata) keeps its own key-sortedcalculateChecksumand skips a save on equality. It is a second hash vocabulary over the samesys_metadatatable. It is reached only throughMetadataManager.register/save, which no Studio save or publish path calls (no caller inpackages/*/src). Carrier: none.packages/objectql/scripts/dry-run-hash-compat.ts(PR-10d.1) hashes type-blind. On an object row stamped by the new rule with unsortedfields, itschecksum_driftarm fires. It is not published (filesisdistonly) and has no runner beyond its own test. Carrier: none.FileSystemRepository's JSONL log keeps pre-change hashes. For an object with unsortedfields, provenance onget/listfalls back to defaults: no parent hash, seq0, the filesystem actor and the epoch timestamp. This lasts until the item next changes. Production consumer census:MetadataManager.getRepository()has no caller inpackages/*/src, and the manager consumes onlywatch()events, which are unaffected. Stated in themetadata-fschangeset.revertCommit's restore limb passes the commit item's stored type toputunfolded, so a legacy row stored under a manifest plural such asobjectsreaches the repository under that spelling. The ordered-fieldsrule applies only to the canonicalobjectspelling. A reorder-only restore of such a row is therefore skipped as unchanged. Only rows written before the meta overlays: unnormalized type segment creates phantom rows that shadow the code-authored listing and cannot be deleted #4432 boundary fold are affected; every/metadoor folds today.FileSystemRepositorytakes the metadata type from the directory name verbatim (parseItemPath, no validation). Anobjects/directory would hash order-blind. Itsputhas no production caller; it is attached only as an event source.Clause-②: yeshere and in the three changesets, and the plural-spelling measurement it asked for is answered in the os-dev-report on metadata: publishing a field reorder from the object designer is a silent no-op — the content hash is key-order-insensitive, so a reorderedfieldsmap hashes equal and the draft is dropped #21790. The docs drift check (comment 5988314939):content/docs/concepts/metadata-lifecycle.mdx:149says the hash is a sorted-keys serialization of the body. After this PR that is true except forobject.fields, whose order is kept. Done at046c425d62: that paragraph now says the canonical form sorts keys except in the maps the spec declares ordered (today anobject'sfields), thatput()still short-circuits on an unchanged body, and that a row written before this release keeps its stored hash as its version token until its content next changes.dispatch-gates --commandsfor that path derives 41 gates. All 41 exit 0,check:doc-authoringincluded, and--ranreconciles 41 derived, 41 run, 0 NOT-MEASURED.check:skill-examplesfirst answered PREREQUISITE NOT MET becauseclient-reactwas unbuilt; it passed once that package was built.origin/mainmoved toba57588665(service-settings and the spec crypto-provider contract) after the pre-push check, so this head does not merge it.checksumequal to itsprevious_checksum: the two hashes are taken under different rules.getByHashon that value can find either history row. Objects are notexecutionPinned, so no pinned execution resolves through it.Generated by Claude Code