fix(service-datasource): the admin door reads a datasource's origin from provenance, and a metadata-door write reaches it in the same boot

[objectstack-fleet]

Fixes #21923
Clause-②: no

What changes

A runtime datasource is one record that two doors serve: the datasource admin door (/api/v1/datasources) and the metadata door (/api/v1/meta/datasource). They disagreed about it in three ways. All three land in packages/services/service-datasource/src/datasource-admin-plugin.ts.

1. Origin comes from provenance, not from the record. listDatasourceRecords and getDatasourceRecord served origin: r.origin ?? 'code'. A metadata-door body has no origin, or it asserts origin: 'code'. So after a restart, the boot restore registered the row and the admin door served it as code-defined and refused its PATCH. A new servedOrigin(ctx, name) returns code only for a name in the host's code-datasource set, and runtime for every other name, whatever the record says. That set is the code-datasource-names kernel service PR fix(service-datasource,runtime,metadata-protocol)!: a stored datasource row no longer displaces a code-defined datasource at boot, and the metadata door refuses edits to the host default #21965 landed, which the boot restore and the metadata door's refusal already read. Boot pool rehydration filters on the served origin, so such a datasource also gets its live pool after a restart.

2. Same-boot reach. The metadata door persists to sys_metadata and the SchemaRegistry. It never registers the record in the MetadataService slot the admin door lists. At start(), the plugin now registers the protocol's awaited datasource mutation projector (ADR-0094, registerMutationProjector; no metadata-protocol edit). After each metadata-door save, publish, revert, rollback or delete, and before that door answers, the projector does three things:

   • it re-reads the stored row (sys_metadata, the same read the boot restore makes);
   • it registers or unregisters the MetadataService slot to match that row;
   • it converges the live pool through the existing convergePool.

   A name in the code set is skipped, because code wins. So the metadata door's repair DELETE of a stored shadow row under a code name keeps the code definition served.

3. The reverse direction. persistDatasourceRow wrote the sys_metadata row with no checksum. The metadata door's reads serve row.checksum ?? hashSpec(body) as the version, but its writes compare the parent against the raw column. So every metadata-door PUT and DELETE of an admin-created datasource answered 409 METADATA_CONFLICT. The row now carries hashSpec(record, 'datasource'), imported from @objectstack/metadata-core. That is the same computation SysMetadataRepository.put stamps (hashSpec(body, ref.type)). @objectstack/metadata-core moves from devDependencies to dependencies; in the lockfile only the importer entry moves.

convergePool is the receive half of the datasource cluster bridge, and the projector's pool step. It now decides "code" from the same set instead of row.origin !== 'runtime'. It pools every other stored row as runtime, and stamps the record origin: 'runtime' before the record reaches the connect context.

Editing a code-defined datasource is still refused at both doors.

Why Clause-②: no

The origin docblock in packages/spec/src/data/datasource.zod.ts reads: "runtime — created via the Studio wizard, persisted in the runtime metadata store, environment-scoped, editable", and "Never accepted from client input". The published contract already says such a datasource is editable, so the admin door's refusal of a metadata-door datasource as code-defined was a false refusal. No key, export or stored shape moves.

projectionApplied now appears on the answer to a datasource write through the metadata door. That key is already declared optional on the protocol's write answer, so this adds no new key to a published payload.

Measured on the base (c9761cd2fb): the new door pin is red

datasource-meta-door-reaches-admin-door.dogfood.test.ts, run on the unmodified base, gave 4 failed and 2 passed:

• :174 same boot, after PUT /meta/datasource/dogfood_meta_none_21923 answered 200: expected undefined to match object { origin: 'runtime', …(1) }. The admin door did not list it.
• :193 an admin-created datasource, then PUT /meta/datasource/dogfood_admin_rt_21923: 409 METADATA_CONFLICT, "Expected parent hmac-sha256:… but current is null."
• :206 after a restart: received "origin": "code", expected "runtime".

Mechanism hypotheses, measured

1. Does any code registration reach the MetadataService without its name in the set? This was measured with bootStack on HEAD 493c13dbb3, comparing metadata.list('datasource') against code-datasource-names:

   composition	listed at boot	code set	listed but outside the set
   showcase	default, showcase_external	default, showcase_external	none
   crm	crm_analytics, crm_primary, default	crm_analytics, crm_primary, default	none
   multi-package	default	default	none

   A package installed after boot does not register datasources in the MetadataService at all: registerApp and the sys_packages rehydrate write only the engine registry. So the admin door never lists one. The three residual paths, read and not measured on a boot, are under Acceptance notes. The fix never falls back to ?? 'code'.

2. Same-boot reach. Verified on main (the :174 failure above). The seam is the awaited projector, not onMetadataMutation: the metadata door answers only after the admin door lists the record, and a projection failure is reported on that answer as well as logged.

   • A live pool is needed. The admin door's create calls registerPool, which connects. The dogfood pin asserts connected for a metadata-door save in the same boot and again after a restart. Before the fix there was no pool, and after a restart the served code kept the datasource out of pool rehydration.

3. Checksum. The fix lands inside service-datasource by importing the repository's own computation. The residual, rows already stored without a checksum, is under Acceptance notes.

4. Carrier notes.

   • convergePool is covered by the provenance rule (above).
   • The restoreRuntimeDatasources and rehydratePools warnings that go only to options.logger are not changed. The bounded in-place-fix condition "same defect class" does not hold for them; see Acceptance notes.

Tests (head 493c13dbb3)

• pnpm --filter @objectstack/service-datasource typecheck: exit 0. The package's tsc --listFiles includes the edited test file.
• pnpm --filter @objectstack/service-datasource test: 41 files and 760 tests pass.
  • 7 new cases in datasource-admin-plugin.test.ts: the served origin, the projector registration, a metadata-door save, an edit and a delete reaching the admin door with their pool, a write under a code name changing nothing, a peer signal pooling by provenance, and the checksum.
  • One fixture re-judged: the "artefact (code) datasource" case now carries the code set, as the runtime fills it, instead of relying on a missing origin.
  • datasource-system-context.pin.test.ts follows convergePool's new ctx parameter.
• pnpm --filter @objectstack/dogfood typecheck: exit 0.
• Four dogfood files pass, 24 tests (after pnpm --filter @objectstack/service-datasource build, dist marker preflight present): the new pin, datasource-restore-code-wins, meta-door-code-datasource, and external-import-code-datasource-namespace.
• The rejection pin asserts code plus status: the admin PATCH of showcase_external still answers 400 DATASOURCE_ADMIN_ERROR, with the message's first sentence.

Ablations (each mutation through scripts/ablation-replace.mjs, rebuilt, ablation-dist-preflight on both legs, restored to the HEAD blob a7f28b52b967, git diff HEAD empty, and the restore leg rebuilt)

put back	unit (26 in file)	dogfood
origin ?? 'code' in list and get	3 red	new pin 3 red (:174, :206, :222)
the record's own origin first (r.origin ?? servedOrigin(…))	2 red	new pin 2 red (:175, the body asserting code served as code)
no projector (void this.projectMetadataDoorWrites;)	4 red	new pin 4 red (:174, :194, :206, :222)
convergePool reads row.origin !== 'runtime'	3 red	new pin 2 red (:178, no pool)
no checksum (const checksum = null)	1 red	new pin 3 red (:193 and :221, 409 METADATA_CONFLICT)
projector without its code-name guard	1 red	datasource-restore-code-wins 1 red (:271, the repair DELETE unregisters the code definition)

Three first attempts did not run: the no-projector, convergePool and no-checksum mutations each failed the DTS build with an unused symbol (TS6133), so nothing was measured. Each was redone with a mutation that compiles. In every void attempt the restore leg was proven the same way.

Gates (head 493c13dbb3)

node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack re-derived 78 families on this change. That is a superset of the 49 the dispatch listed; the additions come from the changeset, the lockfile and package.json. All 78 exit 0. --ran reconciles: 78 derived, 78 run, 0 NOT-MEASURED, each with a recorded exit code.

One gate needed a rerun. pnpm check:dual-build-cjs-loads first exited 3 with PREREQUISITE NOT MET (8 unrelated packages had no dist/). After those packages were built, it exited 0: 106 require entry points across 66 packages load.

Narrowed lint (eslint --no-inline-config --format json) over the 4 touched .ts files:

• every one resolves a config (--print-config);
• the JSON output counts 4 files, 0 errors and 0 warnings;
• eslint.config.mjs enables no type-aware linting, so this diff cannot move a verdict on an untouched file.

The repo-wide pnpm lint is left to CI.

Acceptance notes

• Rows stored before this release. An admin-created datasource row written before this release has no checksum, so the metadata door still answers it 409 until the admin door next writes it; one admin edit is the remedy. The asymmetry itself is in metadata-protocol's sys-metadata-repository.ts: rowToItem serves row.checksum ?? hashSpec(body) while put and delete compare the raw column. It holds for any type's null-checksum row, and is reported to the seat rather than edited here.

• Code datasources the code set does not cover (read, not measured on a boot):

  • Dev artifact HMR (artifactWatch) re-registers the artifact's datasources through the MetadataPlugin door. A datasource added to the artifact after boot is not in the set, because AppPlugin memoizes its owners, so the admin door serves it as runtime until a restart.
  • The legacy FilesystemLoader (a datasource/ directory under the metadata root) lists datasources nothing adds to the set.
  • A host that composes the artifact door without AppPlugin or DefaultDatasourcePlugin registers no set, so nothing is code there; the boot restore already treats such a host that way.

  The remedy for each is on the producer side: contribute to the set. It is not a consumer default.

• Cluster. The projector runs on the writing replica only; the protocol's cluster channel replays mutation listeners, not projectors. So a metadata-door datasource write does not converge peer replicas' MetadataService or pools until they restart. Before this change no replica converged. Cross-replica MetadataService coherence is a separate, open measurement.

• Lost warnings under os serve. The existing restoreRuntimeDatasources and rehydratePools warnings still go only to options.logger, which os serve does not pass. This change widens the population that reaches rehydratePools, because metadata-door datasources now rehydrate. The new projector adds no log line of its own: a projection failure is reported on the metadata door's answer (projectionApplied) and logged by the protocol.

• Not this card. The metadata door's own GET serves a stored row under a code-defined name (the overlay read); finding(service-datasource): a stored datasource row overrides a code-defined datasource at boot, so after a restart the admin door serves and edits it at runtime (restoreRuntimeDatasources has no code-collision check) #21922 remains open for that half.

Files

• packages/services/service-datasource/src/datasource-admin-plugin.ts
• packages/services/service-datasource/src/__tests__/datasource-admin-plugin.test.ts
• packages/services/service-datasource/src/__tests__/datasource-system-context.pin.test.ts
• packages/services/service-datasource/package.json, pnpm-lock.yaml (@objectstack/metadata-core becomes a dependency)
• packages/qa/dogfood/test/datasource-meta-door-reaches-admin-door.dogfood.test.ts (the door pin, declared on [PM seat] domain:cli — 🟢 os-warren · session_01RWZbGvPFcRKvUqASZtunCU #6024)
• .changeset/21923-datasource-origin-from-provenance.md (@objectstack/service-datasource patch)

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/service-datasource, touching 7 documentable anchor(s). ⚠️ 1 changed file(s) yielded no anchor (packages/services/service-datasource/package.json), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

1 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

• content/docs/data-modeling/external-datasources.mdx (via DatasourceAdminServicePlugin (symbol, a top-level class))

What this run could not see

• 1 changed file(s) yielded no anchor (packages/services/service-datasource/package.json) — pages documenting those are invisible to this run
• 1 name(s) were too generic to anchor anything (single lowercase words)
• the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
• a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 1 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json f76c6221acd3997dd778fdd3e8e7d43e0bec4851 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 91bb125dbf8d2cb3ffe6140b40a0c12b706ee395 — the merge of head 493c13dbb312ec5123605bfd2bcf20522bf55d30 into base f76c6221acd3997dd778fdd3e8e7d43e0bec4851, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 91bb125dbf8d2cb3ffe6140b40a0c12b706ee395 && git checkout 91bb125dbf8d2cb3ffe6140b40a0c12b706ee395
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin f76c6221acd3997dd778fdd3e8e7d43e0bec4851 493c13dbb312ec5123605bfd2bcf20522bf55d30 && git checkout -B drift-repro f76c6221acd3997dd778fdd3e8e7d43e0bec4851 && git merge --no-ff 493c13dbb312ec5123605bfd2bcf20522bf55d30

node scripts/docs-audit/affected-docs.mjs --json f76c6221acd3997dd778fdd3e8e7d43e0bec4851

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs f76c6221acd3997dd778fdd3e8e7d43e0bec4851 → pass the list as
args.docs, on the commit named under Which tree this was computed on.