Merge remote-tracking branch 'origin/main' into claude/issue-21776-reseed-intact-baseline

Author: claude

.changeset/21785-email-template-overlay-survives-boot.md

@@ -0,0 +1,13 @@
+---
+"@objectstack/plugin-email": patch
+---
+
+An email template edited through `PUT /api/v1/meta/email_template/:name` (the Studio editor's door) now keeps the admin's wording in `sys_email_template` across a restart. Before, the boot sweep wrote the package wording back over the sending row while `GET /meta` kept serving the admin's, so mail went out with the package wording after every boot.
+
+Clause-②: no
+
+- The cause: on a deployment with a Default Organization the admin's save is an org-scoped overlay, and boot hydration keeps org-scoped overlays out of the registry the sweep read. An env-wide overlay was already kept.
+- `EmailServicePlugin`'s boot sweep now projects the effective template: the layered list `protocol.getMetaItems` serves, read in the organization `tenancy.defaultOrgId()` names. That is the Default Organization under the `single` posture. A host without a `protocol` service reads the registry as before.
+- A failed effective read projects nothing for that boot, so the rows keep their last projection. It does not fall back to the package wording.
+- Seed-not-clobber is unchanged. A row an admin created (`managed_by: 'admin'`) or edited through the data API (`customized: true`) is still never overwritten.
+- The published API is unchanged. The exported `bootstrapDeclaredEmailTemplates` keeps its signature and still reads the registry, so a caller outside the plugin sees the same behaviour as before.

.changeset/21790-metadata-core-ordered-fields-hash.md

@@ -0,0 +1,14 @@
+---
+'@objectstack/metadata-core': minor
+---
+
+The content hash keeps the order of an object's `fields`, so a pure field reorder is a new version instead of "no change" (#21790).
+
+Clause-②: yes
+
+- **`canonicalize(value, type?)` and `hashSpec(value, type?)`** take the metadata type. For a type whose body has a map the spec declares ordered, that map keeps its insertion order in the canonical form. Today that is one map: `object.fields`, whose traversal order is the field order the platform presents. Every other map stays key-order independent, including the keys around `fields` and the keys inside each field definition. Called without a type, both functions return exactly what they returned before.
+- **`orderedMapKeys(type?)`** is a new export. It returns the top-level keys of a `type` body whose map keeps its order (`['fields']` for `object`, `[]` otherwise).
+- `InMemoryRepository` and the repository contract suite hash as `ref.type`. Invariant 4 now reads `item.hash === hashSpec(item.body, item.ref.type)`.
+- **Stored hashes.** An object whose `fields` are already in sorted key order hashes exactly as before. Any other object hashes differently from the hash stored before this release. A stored hash is still that row's version token: `@objectstack/metadata-protocol` keeps it as written and compares content to decide whether a save changed anything.
+
+`minor` because two exports widen: a new parameter and a new function. No metadata key, accepted value, wire payload or error code changes.

.changeset/21790-metadata-fs-ordered-fields-hash.md

@@ -0,0 +1,9 @@
+---
+'@objectstack/metadata-fs': patch
+---
+
+`FileSystemRepository` hashes each item as its metadata type, so a reorder of an object's `fields` is a change (#21790). A reordered object is written, and an external edit that only reorders `fields` is reported as an update.
+
+Clause-②: yes
+
+Event-log entries written before this release carry the order-blind hash. For an object whose `fields` are not in sorted key order, `get()` and `list()` no longer find that entry until the item next changes. They fall back to the defaults: no parent hash, sequence `0`, the filesystem actor, and the epoch timestamp. The body and the version hash are unaffected.

.changeset/21790-metadata-protocol-ordered-fields-hash.md

@@ -0,0 +1,12 @@
+---
+'@objectstack/metadata-protocol': patch
+---
+
+Publishing a pure field reorder of an object now saves it. The object designer's drag-to-reorder used to answer success on publish, keep the old order and delete the draft (#21790).
+
+Clause-②: yes
+
+- `SysMetadataRepository` hashes each body as its type, so a reorder of an object's `fields` is a content change. It is written, recorded in history and served by `GET /api/v1/meta/object/:name`.
+- **Rows stored before this release** keep the `checksum` they were written with. That value is still the version token: reads return it, `If-Match` tokens are derived from it, and the optimistic lock compares against it, so upgrading raises no conflict.
+- For an object, "is this save a no-op?" is decided by hashing the stored body under the current rule, not by comparing against the stored checksum. An identical re-save of an old row writes no history row and keeps its checksum. A reorder into sorted key order is written too. Its new hash equals the old order-blind checksum, so a checksum comparison would have dropped it.
+- A save that changes nothing returns the stored checksum as its version, so the receipt's token is the one the next `If-Match` save must send.

.changeset/21806-runtime-ai-patch-mount.md

@@ -0,0 +1,11 @@
+---
+'@objectstack/runtime': patch
+---
+
+A declared `PATCH` AI route is reachable over HTTP: the dispatcher's `/ai/*` method wildcards mount `patch` beside `get`, `post`, `put` and `delete`, so `PATCH /api/v1/ai/conversations/:id` (the SDK's `ai.conversations.update`, the console's conversation rename) reaches its handler instead of answering `405` (#21806).
+
+Clause-②: no
+
+- **Where it failed.** On a host where the wildcards are the only door into `/ai/**`, a `PATCH` never reached the dispatcher. The server adapter answered `405 METHOD_NOT_ALLOWED` with `Allow: DELETE, GET, HEAD, POST, PUT`, because the path matched the wildcards under the four other verbs. Both bases the wildcards serve are fixed: `/api/v1` and `/api/v1/environments/:environmentId`.
+- **An undeclared method still answers `405`.** The AI route table now tells its two misses apart. A method the table does not declare on a path it does declare answers `405 METHOD_NOT_ALLOWED`, with an `Allow` header that names exactly the declared methods, and no handler runs. A path the table declares under no method still answers `404 ROUTE_NOT_FOUND`. The rule is the same for every verb.
+- **What a caller sees change.** A `GET`, `POST`, `PUT` or `DELETE` that names a declared AI path under the wrong method used to answer `404 ROUTE_NOT_FOUND`. It now answers `405` with `Allow`. A `PATCH` to an AI path the table does not declare at all used to answer the adapter's `405`. It now answers `404 ROUTE_NOT_FOUND`. No request that was refused before is served now, except a `PATCH` to a route the table declares.

content/docs/concepts/metadata-lifecycle.mdx

@@ -146,7 +146,7 @@ The switches are operator-only where one install serves several organizations. T
 
 Every `put()` requires a `parentVersion` (or `null` for fresh creates). If the head has moved since the caller read it, the repository throws `ConflictError(ref, expectedParent, actualHead)` with code `METADATA_CONFLICT`. `protocol.ts` catches it at every write entry point (save, publish, rollback, overlay-delete) and re-throws an API error with code `metadata_conflict`, **HTTP 409 Conflict**, and the `expectedParent` / `actualHead` pair attached. Clients are expected to re-read the item and retry; the console does not yet surface a dedicated conflict-resolution UI.
 
-The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON serialization of the body. Identical bodies produce identical hashes — `put()` short-circuits on no-op writes.
+The hash is `sha256:` + 64-hex of a canonical JSON serialization of the body, with `undefined` values dropped. The canonical form sorts object keys, except in the maps the spec declares ordered. Today that is one map: an `object`'s `fields`, which keeps its declared order, so a pure reorder of fields is a new version. `put()` still short-circuits on a write whose body is unchanged. A row written before this release keeps its stored hash as its version token until its content next changes.
 
 ---
 

packages/metadata-core/src/canonicalize.ts

@@ -7,7 +7,8 @@
  * Guarantees:
  *
  * 1. **Key order independence.** Object keys are sorted lexicographically.
- *    `{a:1, b:2}` and `{b:2, a:1}` produce the same canonical form.
+ *    `{a:1, b:2}` and `{b:2, a:1}` produce the same canonical form — for
+ *    every map EXCEPT the ones guarantee 8 names, whose order is content.
  * 2. **Whitespace independence.** No incidental whitespace.
  * 3. **Type preservation.** `undefined` properties are dropped (matching
  *    JSON.stringify), `null` is preserved, arrays preserve order.
@@ -40,6 +41,31 @@
  *    are untouched by this: their canonical form is byte-identical to what
  *    it was before, so no already-stored version changes meaning.
  *
+ * 8. **Declared map order (#21790).** A map whose key order the spec
+ *    declares meaningful keeps its insertion order instead of being sorted,
+ *    so a pure reorder of it is a change. Which maps those are is a fact
+ *    about the metadata TYPE, so it applies only when the caller names the
+ *    type (`canonicalize(body, type)`); see {@link orderedMapKeys} for the
+ *    table and the spec text behind each row. Everything else in the body —
+ *    the keys AROUND an ordered map and the keys INSIDE each of its values —
+ *    stays key-order independent (guarantee 1). With no type, or a type with
+ *    no row, the canonical form is byte-identical to what it was before.
+ *
+ *    Guarantees 5 and 7 hold unchanged: `JSON.stringify` emits own keys in
+ *    the order `Object.keys` reports them and `JSON.parse` recreates them in
+ *    that order, so the order kept here survives every serialisation round
+ *    trip, and the canonical output of an ordered map re-canonicalises to
+ *    itself.
+ *
+ *    The cost is the hashes already stored: a body whose ordered map is not
+ *    already in sorted key order hashes differently under this rule than
+ *    under the old one. A stamp written before #21790 still names its row —
+ *    it is the row's version token, and the optimistic lock compares a
+ *    caller's token with that stored stamp, never with a re-derivation — so
+ *    the comparison that needed teaching is "is this write a no-op?", which
+ *    must compare CONTENT under the current rule rather than a stored stamp
+ *    taken under an older one (`SysMetadataRepository.put`).
+ *
  * Non-goals (deliberately not supported):
  *
  * - Functions and symbols. These have no canonical JSON form. Callers must
@@ -54,10 +80,57 @@
 
 import { createHash } from 'node:crypto';
 
-/** Stable JSON serialisation. See module-level doc for guarantees. */
-export function canonicalize(value: unknown): string {
+const NO_ORDERED_MAPS: readonly string[] = Object.freeze([]);
+
+/**
+ * Guarantee 8's table — per metadata type, the top-level keys of its body
+ * whose value is a map with MEANINGFUL KEY ORDER.
+ *
+ * A row is owed only where `packages/spec` declares the order; ⛔ never an
+ * order inferred from what a renderer happens to do with a map, because a
+ * row here turns every reorder of that map into a new version.
+ *
+ *   - `object` → `fields`. `ObjectSchema.fields` is a name-keyed record whose
+ *     traversal order IS the field order the platform presents: the
+ *     `ObjectFieldGroupSchema` contract (`data/object.zod.ts`) sets "the
+ *     in-group display order equals the traversal order of
+ *     `ObjectSchema.fields`" and lays ungrouped fields out "preserving their
+ *     field declaration order"; `fieldGroups` repeats it ("fields are
+ *     displayed in their `ObjectSchema.fields` declaration order"); the
+ *     `record` field type it is authored through declares "Insertion order =
+ *     display order" (`data/field.zod.ts`); and the display-name derivation
+ *     picks the first title-eligible field "by declaration order"
+ *     (`data/display-name.ts`). A hash blind to that order called a pure
+ *     reorder "no change", and the object designer's publish was dropped.
+ *
+ * Keyed by the canonical (singular) type name the repositories write under.
+ * A `Map`, so a type name that happens to be an `Object.prototype` member
+ * reads as "no row" rather than as an inherited function.
+ */
+const ORDERED_MAPS: ReadonlyMap<string, readonly string[]> = new Map([
+  ['object', Object.freeze(['fields'])],
+]);
+
+/**
+ * The top-level keys of a `type` body whose map value keeps its key order in
+ * the canonical form (guarantee 8). Empty when `type` is omitted or has no
+ * declared ordered map — the case in which canonicalization is exactly the
+ * type-blind form it has always been.
+ */
+export function orderedMapKeys(type?: string): readonly string[] {
+  return (type === undefined ? undefined : ORDERED_MAPS.get(type)) ?? NO_ORDERED_MAPS;
+}
+
+/**
+ * Stable JSON serialisation. See module-level doc for guarantees.
+ *
+ * @param type The metadata type of `value`, when it is a metadata body.
+ *             Decides which maps keep their order (guarantee 8); omit it for
+ *             a value that is not one, which sorts every map.
+ */
+export function canonicalize(value: unknown, type?: string): string {
   // `''` is the key `JSON.stringify` hands a root-position `toJSON`.
-  return JSON.stringify(normalise(value, ''));
+  return JSON.stringify(normalise(value, '', orderedMapKeys(type), false));
 }
 
 /**
@@ -83,8 +156,19 @@ function resolveToJson(value: unknown, key: string): unknown {
  * @param key   The position's key — `''` at the root, the property name
  *              inside an object, the stringified index inside an array.
  *              Passed to `toJSON` because `JSON.stringify` passes it.
+ * @param orderedChildren Keys of THIS position's object whose values keep
+ *              their own key order (guarantee 8). Non-empty only at the root;
+ *              every deeper position receives none.
+ * @param keepOwnOrder Whether THIS position, when it is an object, keeps its
+ *              keys in insertion order instead of sorting them. Applies to
+ *              this level only — its values are normalised as usual.
  */
-function normalise(rawValue: unknown, key: string): unknown {
+function normalise(
+  rawValue: unknown,
+  key: string,
+  orderedChildren: readonly string[],
+  keepOwnOrder: boolean,
+): unknown {
   const value = resolveToJson(rawValue, key);
   if (value === null) return null;
   if (typeof value === 'undefined') return undefined; // caller-dropped
@@ -107,16 +191,21 @@ function normalise(rawValue: unknown, key: string): unknown {
     // the second argument, which is this function's `key` parameter. The
     // index is the right key, but it has to arrive as the string
     // `JSON.stringify` would hand a `toJSON`, not as a number.
-    return value.map((element, index) => normalise(element, String(index)));
+    return value.map((element, index) =>
+      normalise(element, String(index), NO_ORDERED_MAPS, false),
+    );
   }
 
-  // Plain object: sort keys, drop undefineds.
+  // Plain object: sort keys (unless this is a declared ordered map), drop
+  // undefineds.
   if (typeof value === 'object') {
     const obj = value as Record<string, unknown>;
     const out: Record<string, unknown> = {};
-    const keys = Object.keys(obj).sort();
+    // `Object.keys` order is the order `JSON.stringify` writes, so keeping it
+    // is what keeps guarantee 7 for an ordered map.
+    const keys = keepOwnOrder ? Object.keys(obj) : Object.keys(obj).sort();
     for (const k of keys) {
-      const v = normalise(obj[k], k);
+      const v = normalise(obj[k], k, NO_ORDERED_MAPS, orderedChildren.includes(k));
       if (typeof v === 'undefined') continue;
       out[k] = v;
     }
@@ -129,9 +218,13 @@ function normalise(rawValue: unknown, key: string): unknown {
 /**
  * Compute the canonical sha256 hash of a spec, returned as
  * `"sha256:<64-hex>"`. Equal hashes imply equal canonical forms.
+ *
+ * @param type The metadata type of `value` — a repository passes its
+ *             `ref.type`, so a reorder of a declared ordered map is a new
+ *             version (guarantee 8). Omitted, every map is order-blind.
  */
-export function hashSpec(value: unknown): string {
-  const json = canonicalize(value);
+export function hashSpec(value: unknown, type?: string): string {
+  const json = canonicalize(value, type);
   const digest = createHash('sha256').update(json, 'utf8').digest('hex');
   return `sha256:${digest}`;
 }

packages/metadata-core/src/contract-suite.ts

@@ -12,7 +12,7 @@
  *   1. Atomic put
  *   2. Monotonic seq per branch
  *   3. Optimistic locking (ConflictError)
- *   4. Canonical hashing (hash === hashSpec(body))
+ *   4. Canonical hashing (hash === hashSpec(body, ref.type))
  *   5. Event ordering (monotonic seq, no gaps)
  *   6. Resumability (a numeric `since` replays; a bare `watch(filter)` is
  *      owed live events only)
@@ -204,7 +204,7 @@ export function runRepositoryContractTests(
         const ref = refOf();
         const res = await repo.put(ref, spec('hello'), { parentVersion: null, actor: 'tester' });
         expect(res.version).toMatch(/^sha256:[0-9a-f]{64}$/);
-        expect(res.version).toBe(hashSpec(spec('hello')));
+        expect(res.version).toBe(hashSpec(spec('hello'), ref.type));
         expect(res.seq).toBeGreaterThan(0);
         expect(res.item.parentHash).toBeNull();
         expect(res.item.authoredBy).toBe('tester');
@@ -227,12 +227,12 @@ export function runRepositoryContractTests(
         expect(b.seq).toBeGreaterThan(a.seq);
       });
 
-      it('canonical hash invariant: item.hash === hashSpec(item.body)', async () => {
+      it('canonical hash invariant: item.hash === hashSpec(item.body, item.ref.type)', async () => {
         const repo = await factory();
         const ref = refOf();
         await repo.put(ref, { z: 1, a: 2, m: [3, 1, 2] }, { parentVersion: null, actor: 't' });
         const got = await repo.get(ref);
-        expect(got!.hash).toBe(hashSpec(got!.body));
+        expect(got!.hash).toBe(hashSpec(got!.body, got!.ref.type));
       });
 
       // ── #7856 — the version identifies the STORED BYTES ───────────
@@ -267,7 +267,7 @@ export function runRepositoryContractTests(
             const got = await repo.get(ref);
             expect(got).not.toBeNull();
             expect(got!.hash).toBe(put.version);
-            expect(got!.hash).toBe(hashSpec(got!.body));
+            expect(got!.hash).toBe(hashSpec(got!.body, got!.ref.type));
           });
 
           it(`re-putting the same spec is a no-op — ${shape.label}`, async () => {

packages/metadata-core/src/in-memory-repository.ts

@@ -101,7 +101,8 @@ export class InMemoryRepository implements MetadataRepository {
       throw new ConflictError(ref, opts.parentVersion ?? null, currentHead);
     }
 
-    const hash = hashSpec(spec);
+    // [#21790] Hashed as its type — a declared ordered map keeps its order.
+    const hash = hashSpec(spec, ref.type);
 
     // No-op write — same content. Still consumes nothing; no event emitted.
     if (current && current.hash === hash) {

packages/metadata-core/src/repository.ts

@@ -21,7 +21,12 @@
  *    scoped to a single org may treat the entire repo as one log.)
  * 3. **Optimistic locking.** `put` and `delete` throw `ConflictError`
  *    when `parentVersion` does not match the current HEAD.
- * 4. **Canonical hashing.** `item.hash === hashSpec(item.body)` — always.
+ * 4. **Canonical hashing.** `item.hash === hashSpec(item.body, item.ref.type)`
+ *    — always. The type decides which maps keep their key order
+ *    (`canonicalize` guarantee 8). One stated exception, owned by
+ *    `SysMetadataRepository`: a row stamped before its type's canonical form
+ *    changed keeps the stamp it was written with until its content next
+ *    changes — that stamp is its version token.
  * 5. **Event ordering.** Subscribers to `watch()` receive events in
  *    monotonically-increasing `seq` order with no gaps.
  * 6. **Resumability, and where it stops.** `watch(_, since)` called with a