Skip to content

Commit 26d710e

Browse files
fix(objectql): runtime syncSchemas() binds federated objects instead of sending them DDL (#21796)
Fixes #21777 Clause-②: no ## What this changes `ObjectQL.syncSchemas()` is the schema sync that install-local installs, rehydrates and template seeding run after they register objects at runtime. It now routes a federated object (ADR-0015, `external` set) by the same predicate the boot sync (`ObjectQLPlugin.syncRegisteredSchemas`) uses, and gives it the same treatment: - it binds the object to its remote table with the DDL-free `registerExternalObject` and logs that at `debug`; - it never calls `syncSchema` for the object. Before this, `syncSchemas()` had no federated branch. It sent DDL to every federated object in the registry, the external-schema datasource refused it as designed, and the refusal was logged as the #4632 durability ERROR. The ERROR is unchanged for every other object whose sync fails. Files: - `packages/objectql/src/engine.ts`: `syncSchemas()` gets the federated branch. This is the expected landing point. The file's four other inline `external != null` spellings now call the shared predicate; each is a pure rename. They are in `buildDriverOptions`, the tenant-audit exemption, the related-object own-read branch and `syncObjectSchema`. - `packages/objectql/src/federated-object.ts` (new): `isFederatedObject(schema)`, the one predicate. It is internal and not re-exported from either entry (`index`, `core`), so the public surface is unchanged. - `packages/objectql/src/plugin.ts`: the boot path's four inline spellings adopt the same predicate. They are in `syncRegisteredSchemas` (two sites), `registerSchemasWithoutDdl` and `reconcileFederatedBindings`. These are pure renames with no behaviour change, plus one comment. This file is outside the claim's declared surface. It is the half that makes "cannot drift from the boot path" structural (see H3). - `packages/objectql/src/sync-schemas-federated-object.test.ts` (new): the three pins. - `.changeset/21777-sync-schemas-federated-object.md`: `patch` for `@objectstack/objectql`. There is no edit in `packages/cloud-connection/src/marketplace-install-local-plugin.ts` (no plugin-side filter), none in `packages/spec/src/**`, and none on a governed surface. ## Measurements (at base `ebfe658c72` unless stated) ### H1: every engine path that sends schema DDL to a driver | Path | Where | What it does with an object whose `external` is set | |---|---|---| | Boot sync, `ObjectQLPlugin.syncRegisteredSchemas` (start phases 1 and 3, and `metadata:reloaded`) | `plugin.ts` | With a driver: `registerExternalObject`, and a throw logs `warn`. Without a driver: `debug`, deferred to the `kernel:ready` reconciliation. With a driver that has no `registerExternalObject`: `debug` skip. Never `syncSchema` or `syncSchemasBatch`. | | DDL-free boot (`skipSchemaSync`), `registerSchemasWithoutDdl` | `plugin.ts` | Skip, counted as `federated`. The binding is left to the reconciliation. | | `kernel:ready`, `reconcileFederatedBindings` | `plugin.ts` | `registerExternalObject`. Unbound, unsupported or failed objects are reported at ERROR. | | `ObjectQL.syncObjectSchema`, also reached from the `DatasourceConnectionService` connect re-drive, the `metadata-protocol` publish path and the CLI schema-migration plugins | `engine.ts` | `registerExternalObject`, then return. A throw propagates. | | **`ObjectQL.syncSchemas`** (install-local install and rehydrate, template seeding) | `engine.ts` | **`syncSchema`, refused, then the #4632 ERROR. This is the defect.** | The lifecycle archive pass in `lifecycle-service.ts` also calls `syncSchema`, but against the archive target datasource, not the object's own. It is outside this census. ### H2: what "skip" must mean (real `SqlDriver` on SQLite) A scratch script (not committed) set up a `schemaMode: 'external'` SqlDriver with a pre-existing `customers` table. After `init()`, which is the install-local shape, it registered an object with `external: { remoteName: 'customers' }` and then read it: | Variant | ERROR lines | Read | |---|---|---| | A: `syncSchemas()` at base | 1, "Schema sync FAILED for object 'ext_customer' …" | fails with `no such table: ext_customer` | | B: skip only (nothing after registering) | 0 | fails with `no such table: ext_customer` | | C: `registerExternalObject` (the boot path's treatment) | 0 | 1 row from `customers` | | D: `syncObjectSchema` | 0 | 1 row from `customers` | | A: `syncSchemas()` with this PR | 0 | 1 row from `customers` | A bare skip would silence the alarm and leave a runtime-registered federated object unreadable. So `syncSchemas()` follows the boot path and binds without DDL. A failed binding is logged at `warn`, exactly as the boot path logs it. ### H3: one predicate This PR extracts ONE named predicate, `isFederatedObject` (`external != null`), and adopts it at every inline site in `engine.ts` and `plugin.ts`. The alternative was to route `syncSchemas()` through `syncObjectSchema()`'s branch. It was not chosen because it ties `syncSchemas()` only to `syncObjectSchema()`. The boot path lives in `ObjectQLPlugin` and never goes through `syncObjectSchema()`, so the runtime and boot syncs could still drift apart. Every adoption outside `syncSchemas()` is a pure rename, with the same expression and the same null-safety. No other call site's behaviour changes. ### H4: the showcase reach Both objects named in the card's log declare `external` on datasource `showcase_external` (`schemaMode: 'external'`): - `showcase_ext_customer`: `external: { remoteName: 'customers' }`; - `showcase_ext_order`: `external: { remoteName: 'orders' }`. The H2 script measured that `external` survives registration on the registry entry. The predicate covers both objects. ### H5: the ERROR stays loud This diff leaves the #4632 `logger.error` block in `syncSchemas()` byte-for-byte unchanged. Pin 2 holds it, checking the level, the subject, the Error slot and the context fields, in two cases: - an internal object whose driver refuses; - an object WITHOUT `external` that lands on the external-schema datasource. ### Reach through the public door (dogfood: real showcase boot plus install-local) `packages/qa/dogfood/test/install-local-purge-sample-data.dogfood.test.ts` boots the showcase and installs CRM through install-local. It resolves `@objectstack/objectql` through `dist/`. | `dist/` | `Schema sync FAILED` lines | Objects named | Control: `syncSchemas() ran after registering` lines | Tests | |---|---|---|---|---| | `syncSchemas()` federated branch disabled (marker planted and rebuilt; `ablation-dist-preflight` found it in 4 built files) | 4 | `showcase_ext_customer` ×2, `showcase_ext_order` ×2 | 2 | 8/8 | | This PR | 0 | none | 2 | 8/8 | Restore leg: - `git checkout HEAD -- PATH`; the blob equals the HEAD blob and `git diff HEAD` is 0 bytes; - rebuilt, and `ablation-dist-preflight --absent` is clean over 14 built files; - `git status --porcelain` is empty. The suite stays green in both legs, because nothing in it reads the false alarm. ## Pins The pins are in `packages/objectql/src/sync-schemas-federated-object.test.ts`, on the real `ObjectQL` engine. The driver double records its calls and, on the external datasource, refuses DDL with `ExternalSchemaModeViolationError`. 1. `syncSchemas()` over a registry holding the two showcase-shaped federated objects: - logs no ERROR; - makes zero `syncSchema` calls on the external datasource; - makes one `registerExternalObject` call per federated object; - still syncs the managed object beside them. 2. The #4632 ERROR still fires in two cases: (a) an internal object whose driver refuses; (b) an object without `external` routed to the external-schema datasource. 3. On one fixture, the boot sync and `syncSchemas()` produce the identical driver-call list and report the identical objects at ERROR. ### Reverse verification (committed first, at `ef5099f4df`) - **Mutation.** `syncSchemas()`'s `if (isFederatedObject(obj)) {` became `if (false as boolean) {`, written through `scripts/ablation-replace.mjs`: anchor 1 to 0, blob `397b5ffb636d` to `06523188d217`. The test imports the subject relatively, so it resolves to `src/` and no rebuild leg was needed. - **Predicted and observed:** - pin 1 red: "expected … to have a length of +0 but got 2"; - both pin-3 cases red: the call lists differ, and the runtime's errored set is `showcase_ext_customer, showcase_ext_order, stray_ledger` against the boot's `stray_ledger`; - both pin-2 cases green. - **Restore.** `git checkout HEAD -- PATH`; the blob equals HEAD `397b5ffb636d`, `git diff HEAD` is 0 bytes, and porcelain is empty. ## Verification (HEAD `260392d6a6`, after merging `origin/main` `e83c9f6154`) - **Package suite.** `pnpm --filter @objectstack/objectql test` passed 373 files and 7463 tests. - **Typecheck.** `pnpm --filter @objectstack/objectql typecheck` exited 0. `--listFiles` confirms the new test file and module are inside the `tsconfig.test.json` program, with zero errors of their own. - **Build refresh.** After the merge: `pnpm install --frozen-lockfile`, then a full turbo build (72/72), then `pnpm --filter @objectstack/spec check:generated` (15/15 artifacts up to date). - **Derived gates.** `node scripts/pm/dispatch-gates.mjs --commands` with no paths derived 67 families. All 67 ran, and `--ran` reconciled 67/67 with 0 NOT-MEASURED and 0 UNRUN. - **Artifact-roster block.** All 54 rows ran. 51 exited 0. Three bare scripts are PR-context guards that exit 2 NOT WIRED without a PR (nothing measured): `check-closing-target-claim.mjs`, `check-partof-closing-keyword.mjs` and `check-single-claim-paths.mjs`. Their workflows supply that context on this PR. - **Symbol-anchor sweeps.** All four exited 0: `check:adr-symbol-anchors`, `check:scripts-symbol-anchors`, `check:spec-docblock-symbol-anchors` and `check:adr-anchors`. - **Engine split ratio.** `check-engine-split-ratio --days 90` is report-only: 98.4% over a complete-clone horizon. It raises no objection to the lines added in `engine.ts`. ## Acceptance notes - **Observation: dead code.** `syncSchemas()` keeps an empty block, `if (... syncSchemasBatch ... batchSchemaSync) { }`, commented "Already handled per-driver below". It does nothing and is untouched here. - **Observation: warn-only binding failure at runtime.** - At boot, the `warn` on a failed `registerExternalObject` is backed by the `kernel:ready` reconciliation, which reports any still-unbound federated object at ERROR. - A runtime `syncSchemas()` has no reconciliation after it, so a binding failure there is reported only at `warn`. - This was not measured as reachable: `SqlDriver.registerExternalObject` is synchronous metadata assignment with no observed throw path. - **Observation: one inline spelling left.** `packages/objectql/src/search-companion.ts` still spells the same test inline (`schema.external == null`), and its docblock requires it to equal the sync seam's predicate. It is semantically identical. Adopting `isFederatedObject` there would be a pure rename, left out to keep this diff to the sync seams. - **Observation: the dogfood suite cannot see the alarm.** The install-local dogfood suite stays green while the false ERROR lines are present, because nothing in it reads the server log. --- _Generated by [Claude Code](https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 045f764 commit 26d710e

5 files changed

Lines changed: 322 additions & 10 deletions

File tree

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
---
2+
"@objectstack/objectql": patch
3+
---
4+
5+
A runtime schema sync (`ObjectQL.syncSchemas()`) no longer sends DDL to an object with `external` set. That sync runs on an install-local install, a rehydrate and template seeding. It also no longer logs the durability ERROR "Schema sync FAILED … not durable" for such an object. The ERROR still fires for any other object whose sync fails.
6+
7+
Clause-②: no
8+
9+
- **What was wrong.** A federated object (ADR-0015) lives on a datasource whose schema the remote database owns. The boot sync never sends it DDL. It binds the object to its remote table with the driver's DDL-free `registerExternalObject`. The runtime sync had no such branch, so it called `syncSchema` on every federated object in the registry. On an external-schema datasource the driver refuses that DDL, as designed. The refusal was then logged as a durability failure, although nothing durable was lost. A showcase-based host printed two false ERROR lines on every install-local install, one each for `showcase_ext_customer` and `showcase_ext_order`. A false alarm on every run teaches operators to skip the one line that, for any other object, means its data is not on disk.
10+
- **What it does now.** `syncSchemas()` treats a federated object exactly as the boot sync does. It binds the object without DDL and logs a `debug` line. If the driver has no `registerExternalObject`, it skips the object at `debug`. A binding that throws is logged at `warn`. It never calls `syncSchema` for the object. The binding matters for an object registered at runtime: without it, every read resolves to a table named after the object instead of the remote table, and fails with "no such table".
11+
- **One predicate.** The boot sync, `syncSchemas()` and `syncObjectSchema()` now ask one shared predicate, `external != null`, so the runtime and boot syncs cannot drift apart again. The predicate reads the object's own `external` block, not its datasource's `schemaMode`. An object without `external` that lands on an external-schema datasource still gets the ERROR when its DDL is refused, because it expected a table it did not get.
12+
- No accepted input, key, export, status or error code changes.

‎packages/objectql/src/engine.ts‎

Lines changed: 42 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -298,6 +298,8 @@ import {
298298
rowsWithDeclaredColumnsOnly,
299299
withDeclaredColumnsOnly,
300300
} from './declared-read-columns.js';
301+
// [#21777] "Is this schema the remote's?" One predicate, shared with the boot sync.
302+
import { isFederatedObject } from './federated-object.js';
301303
import { applyInMemoryAggregation } from './in-memory-aggregation.js';
302304
import {
303305
resolveEngineDeleteDispatch,
@@ -5483,10 +5485,10 @@ export class ObjectQL implements IObjectQLEngine {
54835485
// of their own and were riding the same wrong connection.
54845486
const hasTx = tx !== undefined && this.transactionCoversDriverFor(object, tx);
54855487
const objectSchema = this._registry.getObject(object) as any;
5486-
// `external != null` is the same predicate `syncObjectSchema` routes a
5488+
// `isFederatedObject` is the same predicate every schema-sync seam routes a
54875489
// federated object by — one spelling of "this schema is the remote's",
54885490
// not a second reading of it.
5489-
const isFederated = objectSchema?.external != null;
5491+
const isFederated = isFederatedObject(objectSchema);
54905492
const hasTenant =
54915493
execCtx?.tenantId !== undefined &&
54925494
!isTenancyDisabled(objectSchema) &&
@@ -5730,7 +5732,7 @@ export class ObjectQL implements IObjectQLEngine {
57305732
// A federated object's schema is the REMOTE's (ADR-0015); the platform's
57315733
// injected column says nothing about it, which is the same reason
57325734
// `buildDriverOptions` withholds `tenantId` there.
5733-
if (objectSchema?.external != null) return undefined;
5735+
if (isFederatedObject(objectSchema)) return undefined;
57345736
const tenantField = resolveTenantFieldName(objectSchema);
57355737
if (!tenantField) return undefined;
57365738
// A row that names its own organization has carried one explicitly. Only a
@@ -8410,7 +8412,7 @@ export class ObjectQL implements IObjectQLEngine {
84108412
// caller that is not SYSTEM its rows are the ones its OWN read returns,
84118413
// through every enforcement layer; any other id is 'unreadable', stored or not.
84128414
let ownRead: Set<string> | undefined;
8413-
if (bound && (targetSchema?.external != null || resolveTenantFieldName(targetSchema) === null)) {
8415+
if (bound && (isFederatedObject(targetSchema) || resolveTenantFieldName(targetSchema) === null)) {
84148416
const own = await this.find(target, {
84158417
where: { id: { $in: [...ids] } }, fields: ['id'], context: caller as EngineQueryOptions['context'],
84168418
}) as Array<Record<string, unknown>>;
@@ -18304,12 +18306,47 @@ export class ObjectQL implements IObjectQLEngine {
1830418306
* Call this after dynamically registering new objects at runtime
1830518307
* (e.g. after template seeding) to ensure tables/collections exist
1830618308
* before inserting seed data.
18309+
*
18310+
* A federated object (ADR-0015 `external`) is not a DDL target. It is bound
18311+
* to its remote table without DDL, the way the boot sync
18312+
* (`ObjectQLPlugin.syncRegisteredSchemas`) binds it.
1830718313
*/
1830818314
async syncSchemas(): Promise<void> {
1830918315
const allObjects = this._registry.getAllObjects();
1831018316
for (const obj of allObjects) {
1831118317
const driver = this.getDriverForObject(obj.name);
1831218318
if (!driver) continue;
18319+
// [#21777] Same predicate and same treatment as the boot sync. The remote
18320+
// owns this object's schema, so `syncSchema` would be refused
18321+
// (`ExternalSchemaModeViolationError` on an external-schema datasource).
18322+
// Logging that refusal at the #4632 ERROR below was a false durability
18323+
// alarm, because nothing was meant to be created. The DDL-free binding
18324+
// still runs, because an object registered at runtime has no other way
18325+
// to its remote table: without it every read resolves to a table named
18326+
// after the object.
18327+
if (isFederatedObject(obj)) {
18328+
if (typeof (driver as any).registerExternalObject !== 'function') {
18329+
this.logger.debug('Driver does not support registerExternalObject, skipping external object', {
18330+
object: obj.name,
18331+
driver: (driver as any).name,
18332+
});
18333+
continue;
18334+
}
18335+
try {
18336+
await (driver as any).registerExternalObject(obj);
18337+
this.logger.debug('Federated object is not a DDL target — bound to its remote table without DDL', {
18338+
object: obj.name,
18339+
driver: (driver as any).name,
18340+
});
18341+
} catch (e: unknown) {
18342+
this.logger.warn('Failed to register external object metadata', {
18343+
object: obj.name,
18344+
driver: (driver as any).name,
18345+
error: e instanceof Error ? e.message : String(e),
18346+
});
18347+
}
18348+
continue;
18349+
}
1831318350
const tableName = StorageNameMapping.resolveTableName(obj);
1831418351
if (typeof (driver as any).syncSchemasBatch === 'function' && (driver as any).supports?.batchSchemaSync) {
1831518352
// Already handled per-driver below; skip individual call
@@ -18358,7 +18395,7 @@ export class ObjectQL implements IObjectQLEngine {
1835818395
// (its remote schema is owned externally). This is what an app's onEnable
1835918396
// calls after registering a late external driver so coercion maps + the
1836018397
// physical-table mapping exist for queries. See SqlDriver.registerExternalObject.
18361-
if (obj.external != null) {
18398+
if (isFederatedObject(obj)) {
1836218399
if (typeof (driver as any).registerExternalObject === 'function') {
1836318400
await (driver as any).registerExternalObject(obj);
1836418401
}
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* Is `schema` a federated object (ADR-0015 `external`), one whose schema is
5+
* owned by the REMOTE database?
6+
*
7+
* This is the ONE spelling of that question for this package's schema-sync
8+
* seams. A federated object is never a DDL target: its storage belongs to the
9+
* remote, so a sync binds it with the driver's DDL-free
10+
* `registerExternalObject` (the object -> remote-table mapping and the
11+
* coercion maps a read needs) and never calls `syncSchema`. Three seams make
12+
* that decision, and all three ask this predicate, so they cannot drift apart:
13+
*
14+
* - the boot sync, `ObjectQLPlugin.syncRegisteredSchemas` (and its DDL-free
15+
* sibling and the `kernel:ready` reconciliation);
16+
* - the runtime sync, `ObjectQL.syncSchemas`, which install-local installs,
17+
* rehydrates and template seeding run after registering objects;
18+
* - the single-object sync, `ObjectQL.syncObjectSchema`.
19+
*
20+
* They did drift (#21777). The runtime sync had no federated branch, so it sent
21+
* DDL to every federated object in the registry. The driver refused it, as
22+
* designed for an external-schema datasource, and the refusal was logged as
23+
* the #4632 durability ERROR, although nothing durable was lost. Every
24+
* install-local install on a showcase host printed that false alarm.
25+
*
26+
* It is a PRESENCE test, `external != null`, and deliberately not a reading of
27+
* the datasource's `schemaMode`. An object with no `external` block is not
28+
* federated, whatever datasource it lands on. If its DDL is refused, that is a
29+
* real lost sync, and it keeps the ERROR.
30+
*/
31+
export function isFederatedObject(schema: unknown): boolean {
32+
return (schema as { external?: unknown } | null | undefined)?.external != null;
33+
}

‎packages/objectql/src/plugin.ts‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ import { Plugin, PluginContext } from '@objectstack/core';
77
import { resolveArtifactPackageOrder, artifactPackageId } from '@objectstack/core';
88
import { applyConversionsToStoredItem } from '@objectstack/spec';
99
import { StorageNameMapping } from '@objectstack/spec/system';
10+
// [#21777] The ONE "is this schema the remote's?" predicate, shared with `ObjectQL.syncSchemas`.
11+
import { isFederatedObject } from './federated-object.js';
1012
import { LifecycleService } from './lifecycle/lifecycle-service.js';
1113
import { lifecycleSettingsManifest } from './lifecycle/lifecycle-settings.js';
1214
import type { DanglingReferenceAuditOptions } from './integrity/dangling-reference-audit.js';
@@ -1492,7 +1494,7 @@ export class ObjectQLPlugin implements Plugin {
14921494
);
14931495
return;
14941496
}
1495-
const federated = allObjects.filter((o: any) => o?.external != null);
1497+
const federated = allObjects.filter((o: any) => isFederatedObject(o));
14961498
if (federated.length === 0) return;
14971499

14981500
let bound = 0;
@@ -1614,7 +1616,7 @@ export class ObjectQLPlugin implements Plugin {
16141616
let unbound = 0;
16151617

16161618
for (const obj of allObjects) {
1617-
if ((obj as any).external != null) {
1619+
if (isFederatedObject(obj)) {
16181620
federated++;
16191621
continue;
16201622
}
@@ -1757,7 +1759,7 @@ export class ObjectQLPlugin implements Plugin {
17571759
// That is the point at which "still no driver" is final and is a real
17581760
// defect, and it is reported as one — this skip is no longer the last
17591761
// word on a declared external object.
1760-
if (obj.external != null) {
1762+
if (isFederatedObject(obj)) {
17611763
ctx.logger.debug(
17621764
'No driver yet for federated object — deferring its remote-table binding to the kernel:ready reconciliation',
17631765
{ object: obj.name, datasource: this.ql.resolveEffectiveDatasource?.(obj.name) },
@@ -1776,8 +1778,10 @@ export class ObjectQLPlugin implements Plugin {
17761778
// remote database, so DDL (syncSchema/initObjects) is forbidden and would
17771779
// throw. Register read metadata (physical remote table + coercion maps)
17781780
// without DDL so the query path resolves to the remote table, then skip
1779-
// the DDL grouping below.
1780-
if (obj.external != null) {
1781+
// the DDL grouping below. `ObjectQL.syncSchemas` (the runtime sync) routes
1782+
// by this SAME predicate and gives the same treatment, so the two passes
1783+
// cannot disagree about which objects are DDL targets (#21777).
1784+
if (isFederatedObject(obj)) {
17811785
if (typeof driver.registerExternalObject === 'function') {
17821786
try {
17831787
await driver.registerExternalObject(obj);

0 commit comments

Comments
 (0)