Merge remote-tracking branch 'origin/main' into claude/issue-21918-federated-injected-anchors

Author: claude

.changeset/21889-code-datasource-namespace.md

@@ -0,0 +1,14 @@
+---
+'@objectstack/runtime': patch
+'@objectstack/service-datasource': patch
+---
+
+An import over a code-defined datasource is held to the namespace of the package that declares it (ADR-0028), and the draft door answers the prefixed name (#21889).
+
+Clause-②: no
+
+- **Before.** `POST /api/v1/datasources/:name/external/tables/:remote/import` with an explicit `name` that carried no namespace prefix answered `201` and saved an unprefixed federated object, and `POST …/external/tables/:remote/draft` answered the bare remote table name with a `TODO(namespace)` note. Measured on the showcase's `showcase_external` under `objectstack dev` and `objectstack start`.
+- **`@objectstack/runtime`.** `AppPlugin` registers each code-defined datasource through `applyProtection` with the id and version of the package body that declares it, so the item carries `_packageId`, `_packageVersion` and `_provenance: 'package'`. On an ADR-0130 `packages[]` artifact each datasource takes its own body's id, never the artifact's top-level manifest id. A top-level datasource that no body declares keeps its registration under the artifact's own id, and a warning names it.
+- **`@objectstack/service-datasource`.** The federation service reads the datasource's package record from the engine registry (`registry.getPackage` on the `objectql` service), the store the runtime publish gate reads for the same check. It used to ask the `metadata` service, which holds no package records in any composition, so no datasource resolved a namespace. The package id still comes only from the stamped `_packageId`.
+- **What a caller sees now.** On a datasource whose package declares `manifest.namespace`, an unprefixed import `name` answers `400 EXTERNAL_IMPORT_ERROR` with ADR-0028's message, which names the prefixed name to use. An import with no `name` override saves the prefixed name the draft derives (for example `showcase_customers` instead of `customers`). `GET /api/v1/meta/datasource` lists the three provenance keys on a code-defined datasource; all three are declared on `DatasourceSchema`. The datasource admin list (`GET /api/v1/datasources`) is unchanged, and the admin door still refuses to edit or remove a code-defined datasource.
+- **Unchanged.** A datasource that carries no `_packageId` (the host `default` is one) and a package that declares no namespace resolve no namespace, so their imports and drafts answer as before.

packages/qa/dogfood/test/external-import-code-datasource-namespace.dogfood.test.ts

@@ -0,0 +1,177 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+//
+// [#21889, ADR-0028] An import over a CODE-DEFINED datasource is held to the
+// namespace of the package that declares the datasource, over the real
+// showcase composition.
+//
+// ## What was broken
+//
+// The showcase declares `showcase_external` in its own package, whose
+// `manifest.namespace` is `showcase`. `POST
+// /api/v1/datasources/showcase_external/external/tables/orders/import` with an
+// explicit `name` that carried no prefix answered `201` and persisted an
+// unprefixed federated object, and the draft door answered the bare remote
+// table name with its `TODO(namespace)` note. Two links were missing:
+// `AppPlugin` registered the code-defined datasource with no `_packageId`, and
+// the federation service then looked the package record up on the `metadata`
+// service, which holds none in any composition. The datasource is now stamped
+// with the package body that declares it, and the namespace is read from the
+// engine registry, the store the publish gate reads.
+//
+// ## What each case pins
+//
+// The ruling's two pins: an unprefixed import name is refused with ADR-0028's
+// message and saves nothing, and the draft door answers the prefixed name with
+// no `TODO(namespace)` note. Then the controls that show the stamp moved
+// nothing else: a prefixed import is saved and serves the remote rows, the
+// host `default` datasource carries no package, the admin service keeps
+// refusing edits to a code-defined datasource and lists it as before, and
+// validate and the existing federated objects answer as before.
+//
+// The verify harness does not mount the federation service, so this file
+// mounts `ExternalDatasourceServicePlugin` itself, as `objectstack dev` and
+// `serve` do. The working directory is a temporary one because the showcase's
+// external datasource and its fixture both name a cwd-relative SQLite file.
+
+import { describe, it, expect, beforeAll, afterAll } from 'vitest';
+import showcaseStack, { onEnable } from '@objectstack/example-showcase';
+import { ExternalDatasourceServicePlugin } from '@objectstack/service-datasource';
+import { bootStack, type VerifyStack } from '@objectstack/verify';
+import { mkdtempSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+
+/** The showcase's code-defined external datasource (`showcase-external.datasource.ts`). */
+const DATASOURCE = 'showcase_external';
+/** The package that declares it, and that package's ADR-0028 namespace. */
+const PACKAGE_ID = 'com.example.showcase';
+const NAMESPACE = 'showcase';
+/** An explicit import name with no namespace prefix: the defect's shape. */
+const UNPREFIXED = 'dogfood_ext_order_21889';
+const PREFIXED = `${NAMESPACE}_${UNPREFIXED}`;
+
+const ext = `/datasources/${DATASOURCE}/external`;
+
+interface Envelope {
+  success?: boolean;
+  error?: { code?: string; message?: string };
+  item?: Record<string, unknown>;
+  data?: unknown;
+  records?: Array<Record<string, unknown>>;
+}
+
+describe('an import over a code-defined datasource is held to its package\'s namespace (showcase)', () => {
+  let stack: VerifyStack;
+  let token: string;
+  let prevCwd: string;
+  let dir: string;
+
+  const call = async (method: string, path: string, body?: unknown) => {
+    const res = await stack.apiAs(token, method, path, body);
+    const json = (await res.json().catch(() => ({}))) as Envelope;
+    return { status: res.status, json };
+  };
+
+  beforeAll(async () => {
+    prevCwd = process.cwd();
+    dir = mkdtempSync(join(tmpdir(), 'dogfood-21889-'));
+    process.chdir(dir);
+    // Provision the remote tables, exactly as `os dev` does at boot (the
+    // harness imports only the stack's default export, so `onEnable` never
+    // runs on its own).
+    await onEnable({ logger: { info() {}, warn() {} } } as never);
+    stack = await bootStack(showcaseStack, {
+      databaseFile: join(dir, 'showcase.db'),
+      extraPlugins: [new ExternalDatasourceServicePlugin()],
+    });
+    token = await stack.signIn();
+  }, 180_000);
+
+  afterAll(async () => {
+    await stack?.stop();
+    if (prevCwd) process.chdir(prevCwd);
+    if (dir) rmSync(dir, { recursive: true, force: true });
+  });
+
+  it('premise: the code-defined datasource carries the provenance of the package that declares it', async () => {
+    const version = (showcaseStack as unknown as { manifest?: { version?: string } }).manifest?.version;
+    expect(version).toMatch(/^\d+\.\d+\.\d+/);
+
+    const read = await call('GET', `/meta/datasource/${DATASOURCE}`);
+    expect(read.status, JSON.stringify(read.json)).toBe(200);
+    expect(read.json.item).toMatchObject({
+      origin: 'code',
+      _packageId: PACKAGE_ID,
+      _packageVersion: version,
+      _provenance: 'package',
+    });
+  });
+
+  it('control: validate and the code-defined federated objects answer as before, ahead of any import', async () => {
+    const validated = await call('POST', `${ext}/validate`);
+    expect(validated.status, JSON.stringify(validated.json)).toBe(200);
+    const report = validated.json.data as { ok?: boolean; results?: Array<{ object: string }> };
+    expect(report.ok).toBe(true);
+    expect(report.results?.map((r) => r.object).sort()).toEqual(['showcase_ext_customer', 'showcase_ext_order']);
+
+    const rows = await call('GET', '/data/showcase_ext_order');
+    expect(rows.status, JSON.stringify(rows.json)).toBe(200);
+    expect(rows.json.records).toHaveLength(4);
+  });
+
+  it('an explicit import name without the prefix is refused with ADR-0028\'s message, and nothing is saved', async () => {
+    const refused = await call('POST', `${ext}/tables/orders/import`, { name: UNPREFIXED });
+
+    expect(refused.status, JSON.stringify(refused.json)).toBe(400);
+    expect(refused.json.error?.code).toBe('EXTERNAL_IMPORT_ERROR');
+    expect(refused.json.error?.message).toContain('missing the package namespace prefix');
+    expect(refused.json.error?.message).toContain(`'${PREFIXED}'`);
+
+    const stored = await call('GET', `/meta/object/${UNPREFIXED}`);
+    expect(stored.status, JSON.stringify(stored.json)).toBe(404);
+  });
+
+  it('the draft door answers the prefixed name, with no TODO(namespace) note', async () => {
+    const drafted = await call('POST', `${ext}/tables/orders/draft`, {});
+
+    expect(drafted.status, JSON.stringify(drafted.json)).toBe(200);
+    const draft = (drafted.json.data as { draft?: { name?: string; source?: string } } | undefined)?.draft;
+    expect(draft?.name).toBe(`${NAMESPACE}_orders`);
+    expect(draft?.source).not.toContain('TODO(namespace)');
+  });
+
+  it('control: a prefixed import name is saved and serves the remote rows', async () => {
+    const imported = await call('POST', `${ext}/tables/orders/import`, { name: PREFIXED });
+    expect(imported.status, JSON.stringify(imported.json)).toBe(201);
+
+    const rows = await call('GET', `/data/${PREFIXED}`);
+    expect(rows.status, JSON.stringify(rows.json)).toBe(200);
+    expect(rows.json.records).toHaveLength(4);
+  });
+
+  it('control: the host `default` datasource carries no package, so no namespace is demanded on it', async () => {
+    const read = await call('GET', '/meta/datasource/default');
+    expect(read.status, JSON.stringify(read.json)).toBe(200);
+    expect(read.json.item).not.toHaveProperty('_packageId');
+    expect(read.json.item).not.toHaveProperty('_provenance');
+  });
+
+  it('control: the admin service still refuses to edit or remove a code-defined datasource, and lists no envelope key', async () => {
+    // The harness mounts no `/api/v1/datasources` admin routes (the CLI's
+    // `serve` does), so this reads the service those routes relay: a refusal
+    // here is the door's `400 DATASOURCE_ADMIN_ERROR`. The refusals key on
+    // `origin`, which the stamp leaves as it was.
+    const admin = stack.kernel.getService<{
+      updateDatasource(name: string, patch: Record<string, unknown>): Promise<unknown>;
+      removeDatasource(name: string): Promise<void>;
+      listDatasources(): Promise<Array<Record<string, unknown>>>;
+    }>('datasource-admin');
+
+    await expect(admin.updateDatasource(DATASOURCE, { label: 'Renamed 21889' })).rejects.toBeInstanceOf(Error);
+    await expect(admin.removeDatasource(DATASOURCE)).rejects.toBeInstanceOf(Error);
+
+    const entry = (await admin.listDatasources()).find((d) => d.name === DATASOURCE);
+    expect(entry).toMatchObject({ origin: 'code', label: 'External Analytics (SQLite)' });
+    expect(entry).not.toHaveProperty('_packageId');
+  });
+});

packages/qa/dogfood/test/external-import-destructive-remedy.dogfood.test.ts

@@ -30,6 +30,10 @@
 // mounts `ExternalDatasourceServicePlugin` itself, as `objectstack dev` and
 // `serve` do. The working directory is a temporary one because the showcase's
 // external datasource and its fixture both name a cwd-relative SQLite file.
+//
+// [#21889] Both imported names carry the showcase's ADR-0028 prefix:
+// `showcase_external` is declared by the showcase package (namespace
+// `showcase`), so an import over it is held to that namespace.
 
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';
 import showcaseStack, { onEnable } from '@objectstack/example-showcase';
@@ -43,9 +47,9 @@ import { join } from 'node:path';
 const DATASOURCE = 'showcase_external';
 const REMOTE = 'customers';
 /** The object the first import creates and the re-import would shrink. */
-const NAME = 'dogfood_ext_cust_21841';
+const NAME = 'showcase_dogfood_ext_cust_21841';
 /** The remedy's "new `name`". */
-const NEW_NAME = 'dogfood_ext_cust_21841_v2';
+const NEW_NAME = 'showcase_dogfood_ext_cust_21841_v2';
 /** The column the re-import leaves out, so the stored object would lose its field. */
 const DROPPED = 'region';
 /** The re-import's options: the same table, one column fewer. */

packages/qa/dogfood/test/external-import-saves-like-meta.dogfood.test.ts

@@ -31,6 +31,13 @@
 // `serve` do. The working directory is a temporary one because the showcase's
 // external datasource and its fixture both name a cwd-relative SQLite file:
 // this file's remote database is its own, not one a parallel file writes.
+//
+// [#21889] Both imported names carry the showcase's ADR-0028 prefix:
+// `showcase_external` is declared by the showcase package (namespace
+// `showcase`), so an import over it is held to that namespace. That retires the
+// shape this file's control first pinned, an object named exactly as its
+// remote table (`orders`), which the namespace now refuses; the control imports
+// with no `name` override instead, under the prefixed name the draft derives.
 
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';
 import showcaseStack, { onEnable } from '@objectstack/example-showcase';
@@ -43,9 +50,9 @@ import { join } from 'node:path';
 /** The showcase's external datasource and its two remote tables (`external-fixture.ts`). */
 const DATASOURCE = 'showcase_external';
 /** Imported under a name that differs from its remote table — the defect's shape. */
-const RENAMED = 'dogfood_ext_cust_21788';
-/** Imported under the remote table's own name — the shape that hid the defect. */
-const SAME_NAME = 'orders';
+const RENAMED = 'showcase_dogfood_ext_cust_21788';
+/** Imported with no `name` override: the draft's name, the remote table's prefixed with the namespace. */
+const DERIVED = 'showcase_orders';
 
 const importPath = (remote: string) => `/datasources/${DATASOURCE}/external/tables/${remote}/import`;
 
@@ -101,11 +108,12 @@ describe('Import as Object persists like the metadata door (showcase, cold boot)
     );
   });
 
-  it('control: an object imported under its remote table\'s own name serves the remote rows', async () => {
-    const imported = await call('POST', importPath('orders'), { name: SAME_NAME });
+  it('control: an object imported with no name override is saved under the prefixed name and serves the remote rows', async () => {
+    const imported = await call('POST', importPath('orders'), {});
     expect(imported.status, JSON.stringify(imported.json)).toBe(201);
+    expect((imported.json as { data?: { object?: { name?: string } } }).data?.object?.name).toBe(DERIVED);
 
-    const read = await call('GET', `/data/${SAME_NAME}`);
+    const read = await call('GET', `/data/${DERIVED}`);
     expect(read.status, JSON.stringify(read.json)).toBe(200);
     expect(recordsOf(read.json)).toHaveLength(4);
   });
@@ -120,8 +128,8 @@ describe('Import as Object persists like the metadata door (showcase, cold boot)
     expect.soft(renamed.status, JSON.stringify(renamed.json)).toBe(200);
     expect.soft(recordsOf(renamed.json)).toHaveLength(3);
 
-    const sameName = await call('GET', `/data/${SAME_NAME}`);
-    expect.soft(sameName.status, JSON.stringify(sameName.json)).toBe(200);
-    expect.soft(recordsOf(sameName.json)).toHaveLength(4);
+    const derived = await call('GET', `/data/${DERIVED}`);
+    expect.soft(derived.status, JSON.stringify(derived.json)).toBe(200);
+    expect.soft(recordsOf(derived.json)).toHaveLength(4);
   }, 180_000);
 });

packages/qa/dogfood/test/external-validate-sees-runtime-save.dogfood.test.ts

@@ -49,7 +49,8 @@ const SAVED = 'dogfood_ext_cust_21842';
 /** A column the saved object declares and the remote `customers` table does not have. */
 const MISSING = 'loyalty_tier';
 /** Imported at runtime from the remote `orders` table. */
-const IMPORTED = 'dogfood_ext_ord_21842';
+// [#21889] Prefixed: an import over `showcase_external` is held to the showcase package's namespace.
+const IMPORTED = 'showcase_dogfood_ext_ord_21842';
 
 const validatePath = `/datasources/${DATASOURCE}/external/validate`;