Skip to content

Commit c4d5713

Browse files
fix(cloud-connection): the install-local listing answers withSampleData from the caller's own organization's rows (#21820)
Fixes #21775 Clause-②: no ## What was wrong `GET /api/v1/marketplace/install-local` answered each entry's `withSampleData` from the install ledger's record (`withSampleData: e.withSampleData ?? false`), which holds one value per install. Under an organization wall, sample data is per organization: the install, the reseed and the purge each act in the caller's active organization. So after a purge in organization A, the listing read as organization B answered `false` while B held all 28 of its CRM seed rows, and a restart kept that answer. ## What changed This follows the triage ruling (`5984286272`): the listing derives `withSampleData` from the caller's own organization's rows, matched by the seed key the purge already uses (`externalId`). The ledger gains no field, and the purge and the reseed behave as before. - **`marketplace-install-local-purge.ts`.** The purge's identification pass is now a separate read-only export, `matchSeedRows`. `purgeSeedRows` deletes exactly what it returns, child before parent. Counts, order, write context and every log line are byte-for-byte what they were: the matcher reports problems as data (`SeedMatchProblem`), and the purge words them the same way as before. The existing purge pins pass unchanged. `firstOnly` stops at the first identified row. - **`marketplace-install-local-plugin.ts`.** `handleList` answers `withSampleData` from `sampleDataInScope`, which runs `matchSeedRows` (`firstOnly`) per seeded entry, under the purge's scope rule (`organizationWallActive` + `resolveActiveOrgId`) and over the loader's own dependency graph. The new `seedDatasetsOf` gives the purge and the listing one definition of what a seed dataset is. - **`local-manifest-source.ts`.** The docblock of the two ledger fields now says they are install-time records, one per install and not per organization, and that the listing does not read `withSampleData`. The docblock lives here rather than in the plugin file the claim named. ## Readings that decided the shape (all on `origin/main` `75ddcd1b41`) - **H1 holds.** `handleList` sits at `:1348`, and `:1367` is `withSampleData: e.withSampleData ?? false`. - **H2 holds, and the matcher is reused.** The purge matched seed rows by each dataset's `externalId` under `READ_CONTEXT = { isSystem: true }`, with reference key parts translated through the matched parents. That loop moved into `matchSeedRows` unchanged, and the listing calls it. Nothing is restated. - **H3: ADR-0123 D2 governs, and it is pinned.** `resolveActiveOrgId` answers the session's active organization or `null`. D2: "Tenant-scoped reads resolve to nothing ... zero rows, HTTP 200, no error." So on a walled session with no active organization, every entry answers `false` with `200`, and no seed row is read. The ADR answers this case, so there is no open question. - **H4: "present" means at least one seed row in scope.** The console reads the flag in one place: objectui `packages/app-shell/src/console/marketplace/MarketplacePackagePage.tsx` (read at objectui `9dfaca6`). It picks the reseed item's label (`reseedAgain` when true, `addSampleData` when false) and disables the purge item when the flag is false. "Any" is exactly when the purge has a row to delete. "All" would disable the purge while 27 of 28 rows remain. A count would change the response shape, and Clause-② is `no`. A seed record the purge cannot identify either (no key value, or a key that two rows carry) is not evidence of presence. ## Tests (head `c668f551`) - `pnpm --filter @objectstack/cloud-connection typecheck` (both tsconfigs): exit 0. - `pnpm --filter @objectstack/cloud-connection exec vitest run --maxWorkers=2`: **39 files, 485 passed**. That is 477 before, plus 8 new pins in `marketplace-install-local-purge.test.ts`, which reuse that file's existing engine double, so no new double was pinned: - matcher: identifies exactly what the purge deletes and deletes nothing; `firstOnly` reads one object when the first parent matches; with no seed row in scope it reads every seeded object; a failed read is reported as `read` data; - listing: walled, after a purge in A, read as B answers `true` (reads pinned to `org_b` only) and read as A answers `false` (A keeps a user-authored row); walled with no active organization, `false`, 200, zero seed-object reads; off-wall, the derived answer equals the ledger's before and after a purge; unreadable rows answer `false` with a `warn` naming the package and the cause. - Fixture triage, two cases. `marketplace-install-local-reseed.test.ts` read the reseed's ledger write back through the listing, and now reads the ledger itself, since its seed loader is a stub that writes no rows. In `marketplace-install-local-list-posture.test.ts`, the fixture's ledger says `true` but its manifest bundles no seed dataset, so the expected value is now `false`. - `pnpm --filter @objectstack/dogfood typecheck`: exit 0. - Dogfood, on real boots (`dist/`-resolved; `withSampleData.has(e.manifestId)` counted once in `dist/index.js` and once in `dist/index.cjs` before the run): **3 files, 20 passed**. - The new `install-local-listing-sample-data.dogfood.test.ts` is the ruling's pin. On a walled boot with A and B (28 seed rows each), after a purge in A the listing read as B answers `true` (28 rows) and read as A answers `false` (0 rows), while the ledger record says `false`. A second boot over the same `databaseFile` and ledger keeps both answers. - `install-local-purge-sample-data.dogfood.test.ts` gains the off-wall pin: listing beside the ledger record after install, purge and reseed, `true/true`, `false/false`, `true/true`. - `install-local-no-active-organization.dogfood.test.ts` passes unchanged. ## Reverse verification (committed fix first, `node scripts/ablation-replace.mjs`, wrap mode) - **Mutation.** `withSampleData: withSampleData.has(e.manifestId),` became `withSampleData: e.withSampleData ?? false,`, which is origin/main's read. The anchor went from 1 to 0, and the blob from `50a71c2f` to `eecd9640`. - **Build.** JS was emitted. DTS failed on TS6133 (the now-unused set), which does not affect the JS the suites load. `ablation-dist-preflight.mjs @objectstack/cloud-connection 'e.withSampleData ?? false'` exited 0, with the marker present in `dist/index.js` and `dist/index.cjs`. - **Expected direction: red. Observed: red.** - Unit: 4 failed, 25 passed. The four are the walled A/B pin, the no-active-organization pin, the unreadable-rows pin and the list-posture member pin. The off-wall pin stays green by design. - Dogfood: 2 failed, 11 passed. "after the purge in A, the listing read as B answers withSampleData: true" got `false`, which is the card's measured defect, and the restart pin also got `false`. - **Restore.** The blob equals the HEAD blob `50a71c2f`, and `git diff HEAD` is empty. The rebuild exited 0, and `--absent` preflight exited 0 with the marker gone from all 6 built files and the tree clean. ## Gates - **The derived set** came from `dispatch-gates --commands`, run without paths. Its 67 commands ran at `d46b1a3d`, and every one exited 0. - `check:dual-build-cjs-loads` first exited 3 (PREREQUISITE NOT MET: 8 packages had no `dist/`). It passed after those were built. - `dispatch-gates --ran` reported: 67 derived, 67 run, 0 NOT-MEASURED, 0 UNRUN. - **After the last commit** (`c668f551`, a type annotation in the new dogfood test), these ran again and exited 0: `check:type-check-debt`, `check:type-check-coverage`, `check:test-source-alias`, `check:cross-package-test-inputs`, `check:engine-double-contract`, `check:nul-bytes`, `check:doc-authoring` and `check:issue-citations`. - **Full `pnpm lint`** (`eslint . --no-inline-config`, not narrowed) exited 0 at `c668f551`. ## Acceptance notes - After this PR, nothing in the runtime reads the ledger's `withSampleData`. The install, the heal, the reseed and the purge still write it, as the ruling keeps it as an install-time record. `sampleDataPurged` keeps its one reader, the heal, which heals nothing under a wall. - **Cost.** Each listing request now builds one dependency graph per seeded package. It also runs one `find` per seeded object for the key columns in scope, and stops at the first seed row found. With no seed row in scope, it reads every seeded object. That is the same read the purge makes, once per request. - **Off-wall, the rows win over the record** where the two disagree. A purge the engine refused for some rows, rows deleted by hand, or a heal that landed nothing now answers from the rows. In the flows that keep them in step (install, purge, reseed), both answers are equal, as pinned. - A rows read that fails answers `false` with a `warn` once per entry per request, the way this door already reports a corrupt ledger entry on every GET. The wire shape is unchanged. ## Carried measurement (from the #21762 review, not acted on here) - **Setup.** An unwalled real boot with `databaseFile`. The CRM package was installed (28 rows), and then the ledger entry's `manifest.engines.protocol` was set to `^16` on a protocol-17 runtime. After a restart, the rehydrate logged `OS_PROTOCOL_INCOMPATIBLE ... is NOT loaded`, and `GET /api/v1/marketplace/install-local` ran once. - **This PR.** 200, `total: 1`, and the entry is listed with every field (`packageId`, `versionId`, `manifestId`, `version`, `installedAt`, `installedBy` for the operator) and `withSampleData: false`. Each such GET logs one `warn` naming the five seeded objects as `Object '...' not found`. - **origin/main's listing read**, measured through the same mutation as above: the same entry and fields, with `withSampleData: true` from the ledger record. On main itself no `warn` is logged, because the derivation does not exist there. - In both cases nothing on the entry says the runtime did not load it. --- _Generated by [Claude Code](https://claude.ai/code/session_01RWZbGvPFcRKvUqASZtunCU)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 3237b4a commit c4d5713

9 files changed

Lines changed: 635 additions & 43 deletions
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
"@objectstack/cloud-connection": patch
3+
---
4+
5+
`GET /api/v1/marketplace/install-local` now answers each entry's `withSampleData` for the caller's own organization. Before, after a purge in organization A, the listing read as organization B answered `withSampleData: false` while B still held every one of its seed rows.
6+
7+
Clause-②: no
8+
9+
- **What was wrong.** The listing served the install ledger's `withSampleData`, one value per install. Under an organization wall, sample data is per organization: the install, the reseed and the purge each act in the caller's active organization. A purge in A flipped the one value for every organization, and a restart kept it.
10+
- **What it does now.** The listing reads the rows. An entry answers `true` when at least one of the package's seed rows is in the caller's scope. Rows are matched the way the purge matches them, by each dataset's `externalId`. "At least one" is exactly when the purge has something to delete, and it decides whether the console labels its reseed action "Add sample data" or "Reseed again". The purge's matching is now a separate read-only step, and the purge deletes what it returns, with the same counts and log lines as before.
11+
- **Scope.** Under a wall, the scope is the caller's active organization. A session with no active organization reads nothing, so every entry answers `false` with `200`, and no row is read. Without a wall, the match covers the whole table, and the answer is the one the ledger records after an install, a purge and a reseed.
12+
- **When the rows cannot be read** (for example, a package the runtime did not load), the entry answers `false`, and the server log says why at `warn`, once per entry per request.
13+
- **The response keeps its shape.** The ledger keeps its shape too. Its `withSampleData` and `sampleDataPurged` stay as install-time records, and their docs now say they are not per organization.

‎packages/cloud-connection/src/local-manifest-source.ts‎

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -42,15 +42,26 @@ export interface InstalledManifestEntry {
4242
manifest: any;
4343
installedAt: string;
4444
installedBy: string | null;
45-
/** Whether the bundled seed datasets have been loaded into the kernel
46-
* database. True after install (seedNow=true) or an explicit reseed;
47-
* false after a purge. Persisted so the UI can show "Add" vs "Re-seed". */
45+
/** Install-time record: true after an install that seeded, a reseed or a
46+
* rehydrate heal that landed rows; false after a purge. ONE value per
47+
* install, NOT per organization — under an organization wall it records
48+
* whichever organization seeded or purged last.
49+
*
50+
* ⛔ Not the answer to "does this caller's organization have the sample
51+
* data?". The GET listing derives that per request from the caller's own
52+
* rows (MarketplaceInstallLocalPlugin.sampleDataInScope, #21775) and does
53+
* not read this field. */
4854
withSampleData?: boolean;
4955
/** True only after an explicit purge-sample-data call. The rehydrate-time
5056
* sample-data healer (see MarketplaceInstallLocalPlugin.maybeHealSampleData)
5157
* must not resurrect demo rows the user deliberately removed — an empty
5258
* table after a purge is desired state, not data loss. Cleared again by
53-
* install/reseed runs that land rows. */
59+
* install/reseed runs that land rows.
60+
*
61+
* Like `withSampleData`, ONE value per install, not per organization.
62+
* The healer is its one reader, and under an organization wall the healer
63+
* heals nothing at all, so this field decides something only without a
64+
* wall, where the install is one tenant. */
5465
sampleDataPurged?: boolean;
5566
/**
5667
* [ADR-0120 D5e] The installer's answer to the `isolated`-posture question

‎packages/cloud-connection/src/marketplace-install-local-list-posture.test.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -269,7 +269,11 @@ describe('#9011 — an authenticated non-operator gets the inventory WITHOUT the
269269
manifestId: 'app.test.crm',
270270
version: '1.4.0',
271271
installedAt: INSTALLED.installedAt,
272-
withSampleData: true,
272+
// The ledger records `true`, but this manifest bundles no seed
273+
// datasets, so no seed row can be in the caller's scope: the listing
274+
// answers from the rows, not the record (#21775, pinned in
275+
// `marketplace-install-local-purge.test.ts`).
276+
withSampleData: false,
273277
});
274278

275279
// ② The half it NARROWS — absent keys, not null values. `null` would be

‎packages/cloud-connection/src/marketplace-install-local-plugin.ts‎

Lines changed: 135 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,10 @@
4444
* authenticated principal (anonymous → 401); `installedBy` and
4545
* `storageDir` are served only to a `manage_metadata` holder
4646
* (commit 01074e551). The four routes above require `manage_metadata`
47-
* outright (#8976).
47+
* outright (#8976). Each entry's `withSampleData` says whether the
48+
* caller's own scope — its active organization under a wall —
49+
* holds any of the package's seed rows, read from the rows, never
50+
* from the install-wide ledger record (#21775).
4851
*
4952
* DELETE /api/v1/marketplace/install-local/:manifestId
5053
* → removes the cached manifest, withdraws the package from the
@@ -114,7 +117,7 @@ import {
114117
} from './local-manifest-source.js';
115118
import { ConnectionCredentialStore } from './connection-credential-store.js';
116119
import { MARKETPLACE_INSTALLED_UI_BUNDLE } from './marketplace-ui.js';
117-
import { purgeSeedRows } from './marketplace-install-local-purge.js';
120+
import { matchSeedRows, purgeSeedRows, type PurgeSeedDataset } from './marketplace-install-local-purge.js';
118121
import type { IHttpServer, IMetadataService, IObjectQLEngine } from '@objectstack/spec/contracts';
119122
import type { DeletePackageRequest, UninstallCleanupOutcome } from '@objectstack/metadata-protocol';
120123

@@ -299,6 +302,17 @@ function manifestIdOf(p: any): string | undefined {
299302
return p?.manifest?.id ?? p?.id ?? p?.manifest?.name ?? undefined;
300303
}
301304

305+
/**
306+
* The seed datasets a cached manifest bundles: its `data[]` entries that name
307+
* an object and carry records. One reading for the two doors that match seed
308+
* rows — the purge, and the listing's `withSampleData` (#21775).
309+
*/
310+
function seedDatasetsOf(manifest: any): PurgeSeedDataset[] {
311+
return Array.isArray(manifest?.data)
312+
? manifest.data.filter((d: any) => d && d.object && Array.isArray(d.records))
313+
: [];
314+
}
315+
302316
/**
303317
* [ADR-0093 D4/D5, ADR-0105 D1 / #5262] Is an organization wall actually IN
304318
* FORCE for this boot? Both seeding decisions in this plugin key off it.
@@ -1344,6 +1358,12 @@ export class MarketplaceInstallLocalPlugin implements Plugin {
13441358
* the four mutating doors use, not a second session read. Two auth
13451359
* mechanisms in one file is how the next gap gets created, and this file has
13461360
* already produced one.
1361+
*
1362+
* ## [#21775] `withSampleData` is the caller's own scope, read from its rows
1363+
*
1364+
* Each entry's `withSampleData` answers "does MY scope hold this package's
1365+
* sample data?" — derived per request by {@link sampleDataInScope}, never
1366+
* read from the ledger's install-wide record of the same name.
13471367
*/
13481368
private handleList = async (c: any, ctx: PluginContext): Promise<Response> => {
13491369
// Before the ledger is touched, exactly as the mutating doors refuse
@@ -1355,6 +1375,7 @@ export class MarketplaceInstallLocalPlugin implements Plugin {
13551375

13561376
const { entries, skipped } = this.readAll();
13571377
this.warnSkippedLedgerEntries(ctx, skipped, 'it is MISSING from the installed-apps list served to the console');
1378+
const withSampleData = await this.sampleDataInScope(c, ctx, entries);
13581379
return c.json({
13591380
success: true,
13601381
data: {
@@ -1364,7 +1385,7 @@ export class MarketplaceInstallLocalPlugin implements Plugin {
13641385
manifestId: e.manifestId,
13651386
version: e.version,
13661387
installedAt: e.installedAt,
1367-
withSampleData: e.withSampleData ?? false,
1388+
withSampleData: withSampleData.has(e.manifestId),
13681389
...(operator ? { installedBy: e.installedBy } : {}),
13691390
})),
13701391
total: entries.length,
@@ -1373,6 +1394,116 @@ export class MarketplaceInstallLocalPlugin implements Plugin {
13731394
}, 200);
13741395
};
13751396

1397+
/**
1398+
* [#21775] The listed entries whose sample data is in the CALLER'S scope —
1399+
* the set the listing answers each entry's `withSampleData` from.
1400+
*
1401+
* ## The rows are the authority, not the ledger
1402+
*
1403+
* The ledger's `withSampleData` / `sampleDataPurged` are install-time
1404+
* records: one value per install, flipped by whichever organization seeded
1405+
* or purged last. Under an organization wall sample data is per
1406+
* organization — the install, the reseed and the purge each act in the
1407+
* caller's active organization, and the per-organization replay seeds every
1408+
* new one — so the listing that read the record told organization B "no
1409+
* sample data" after a purge in organization A, while B held every one of
1410+
* its seed rows. The ledger shape stays as it is; the listing stops reading
1411+
* it.
1412+
*
1413+
* ## Present = at least one seed row, identified the purge's way
1414+
*
1415+
* An entry is in the set when {@link matchSeedRows} — the purge's own
1416+
* identification (each dataset's `externalId`), read-only — finds at least
1417+
* ONE of the package's seed rows in scope. Not "all", and not a count: the
1418+
* console reads the flag to enable its purge item and to word its reseed
1419+
* item ("Add sample data" / "Reseed again"), and the purge has work to do
1420+
* exactly when one seed row is there. A seed record the purge could not
1421+
* identify either (no key value, a key two rows carry) is not evidence of
1422+
* presence; the purge reports those when it runs.
1423+
*
1424+
* ## Scope — the purge's
1425+
*
1426+
* Under a wall, the caller's active organization, from the same
1427+
* `organizationWallActive` + `resolveActiveOrgId` pair the purge scopes
1428+
* with. Without a wall the deployment is one logical tenant and the match is
1429+
* table-wide, as the purge's is, so the derived answer is the one the
1430+
* ledger records after an install that seeded, a purge and a reseed. Where
1431+
* a door's write did not land whole — a purge the engine refused for some
1432+
* rows, rows deleted by hand — the rows answer and the record does not.
1433+
*
1434+
* A walled session with NO active organization reads nothing: every entry
1435+
* answers `false`, HTTP 200, and no row is read (ADR-0123 D2 — a
1436+
* tenant-scoped read resolves to nothing, not an error). The doors that
1437+
* would act on that answer, reseed and purge, refuse the same session with
1438+
* D2's write refusal.
1439+
*
1440+
* ## When the rows cannot be read
1441+
*
1442+
* No metadata service, a dependency graph that does not build, an object
1443+
* whose rows the engine will not return: the entry answers `false`, and why
1444+
* is logged at `warn` — once per entry per request, the way this door
1445+
* reports a corrupt ledger entry — to the operator, with the wire shape
1446+
* unchanged.
1447+
*/
1448+
private sampleDataInScope = async (c: any, ctx: PluginContext, entries: InstalledEntry[]): Promise<Set<string>> => {
1449+
const present = new Set<string>();
1450+
const seeded = entries
1451+
.map((entry) => ({ entry, datasets: seedDatasetsOf(entry.manifest) }))
1452+
.filter(({ datasets }) => datasets.length > 0);
1453+
if (seeded.length === 0) return present;
1454+
1455+
let organizationId: string | undefined;
1456+
if (organizationWallActive(ctx)) {
1457+
const resolved = await this.resolveActiveOrgId(c, ctx);
1458+
// [ADR-0123 D2] No organization, so no organization's rows to read.
1459+
if (!resolved) return present;
1460+
organizationId = resolved;
1461+
}
1462+
1463+
const unread = (manifestId: string, why: string): void => {
1464+
ctx.logger?.warn?.(
1465+
`[MarketplaceInstallLocal] ${manifestId}: the installed-apps listing could not read this package's `
1466+
+ `seed rows (${why}), so it answers withSampleData: false for it`,
1467+
);
1468+
};
1469+
let ql: IObjectQLEngine | undefined;
1470+
let metadata: IMetadataService | undefined;
1471+
try { ql = ctx.getService<IObjectQLEngine>('objectql'); } catch { /* no data engine */ }
1472+
try { metadata = ctx.getService<IMetadataService>('metadata'); } catch { /* no metadata service */ }
1473+
if (!ql || !metadata) {
1474+
for (const { entry } of seeded) {
1475+
unread(entry.manifestId, 'the data engine (objectql) or the metadata service is not available on this runtime');
1476+
}
1477+
return present;
1478+
}
1479+
1480+
for (const { entry, datasets } of seeded) {
1481+
try {
1482+
// The graph the purge reads its order and its reference keys
1483+
// from: the loader's own, over the package's seeded objects.
1484+
const { SeedLoaderService } = await import('@objectstack/runtime');
1485+
const loader = new (SeedLoaderService as any)(ql, metadata, ctx.logger);
1486+
const graph = await loader.buildDependencyGraph([...new Set(datasets.map((d) => String(d.object)))]);
1487+
const failedReads: string[] = [];
1488+
const match = await matchSeedRows({
1489+
engine: ql,
1490+
datasets,
1491+
graph,
1492+
organizationId,
1493+
firstOnly: true,
1494+
report: (problem) => {
1495+
if (problem.kind === 'read') failedReads.push(`${problem.object}: ${problem.detail}`);
1496+
},
1497+
});
1498+
if (match.rows.length > 0) present.add(entry.manifestId);
1499+
else if (failedReads.length > 0) unread(entry.manifestId, failedReads.join('; '));
1500+
} catch (err: any) {
1501+
unread(entry.manifestId, String(err?.message ?? err));
1502+
}
1503+
}
1504+
return present;
1505+
};
1506+
13761507
private handleUninstall = async (c: any, ctx: PluginContext): Promise<Response> => {
13771508
const admission = await this.requireInstallCapability(c, ctx, 'Uninstalling a package');
13781509
if (!admission.ok) return admission.response;
@@ -1844,9 +1975,7 @@ export class MarketplaceInstallLocalPlugin implements Plugin {
18441975
return this.unreadableLedgerEntry(c, ctx, manifestId, failure, 'purge sample data');
18451976
}
18461977

1847-
const datasets = Array.isArray(entry.manifest?.data)
1848-
? entry.manifest.data.filter((d: any) => d && d.object && Array.isArray(d.records))
1849-
: [];
1978+
const datasets = seedDatasetsOf(entry.manifest);
18501979

18511980
if (datasets.length === 0) {
18521981
return c.json({

0 commit comments

Comments
 (0)