diff --git a/.changeset/21391-one-shot-boot-read-only.md b/.changeset/21391-one-shot-boot-read-only.md new file mode 100644 index 00000000000..a201088b6f0 --- /dev/null +++ b/.changeset/21391-one-shot-boot-read-only.md @@ -0,0 +1,24 @@ +--- +'@objectstack/cli': minor +'@objectstack/runtime': minor +--- + +The CLI's one-shot commands no longer write to the database as a side effect of booting. No `os migrate *`, `os meta resync`, `os secret orphans` or `os storage orphans` run loads the app's inline seed data, apply and delete modes included, and every mode that writes nothing now boots read-only. + +Clause-②: yes (narrowing) + + + +**BREAKING** — a no-write run of `os migrate value-shapes`, `os migrate recorded-by`, `os migrate resume`, `os secret orphans` or `os storage orphans` at a database that lacks a table it reads now exits 1, where it used to exit 0. It ships as `minor` under the launch-window convention for accept-set narrowings. + +**What was wrong.** Eight commands booted the full data stack in a mode their documentation says writes nothing: `os migrate value-shapes` (scan), `summary-nulls`, `files-to-references` and `recorded-by` (dry run), `os migrate resume` (list), `os secret orphans` and `os storage orphans` (report), and `os meta resync` without `--yes`. That boot ran schema sync and the app's inline seed loader. The seed loader upserts every seeded row, so each run bumped `updated_at`, stamped `organization_id` on seeded rows that had none, and put an operator's edit to a seeded row back to the seed's value. On `examples/app-crm` that was all 28 seeded rows on every run. On a database behind the app's schema, the boot also added columns and created tables. The apply and delete modes ran the same seed loader alongside the write the operator confirmed. + +**What changes for an operator.** + +- Every mode that writes nothing boots the way `os migrate plan` does: the schema sync is held back, no seed rows are written, and a SQLite file that does not exist is not created. The database is left byte-identical, and the report is the same as before. +- No one-shot CLI boot loads the app's inline seed data. `--apply`, `--delete`, `os migrate resume --run` and `os meta resync --yes` write what they report and nothing else. Seeding stays with `os dev` and `os serve`. +- The deferred schema sync now covers every SQL datasource the boot connects, not only the default one. `os migrate plan` lists a second datasource's pending tables, and `os migrate apply` creates them after you confirm. +- One edge changes: a no-write run pointed at a database that lacks a table it reads (a SQLite file that does not exist, a database that was never booted, or the wrong `--database-url`) refuses and exits 1 instead of creating the table and reporting nothing. Point `--database-url` at the deployment's database, or boot the deployment once first. `os secret orphans --json` answers that refusal with `"error": "scan_failed"`. +- `os migrate value-shapes --json` prints one JSON document when the scan fails its gate. It used to print a second one, `{"error":"EEXIT: 1"}`. + +**For embedders of `@objectstack/runtime`.** `createStandaloneStack` accepts `armLifecycleSweep` (default `true`). With `false`, the ADR-0057 lifecycle sweep (rotation, retention reaping, archiving and the dangling-reference audit that rides its clock) is never armed on that boot, and an explicit `sweep()` call on it returns an empty report. The CLI passes `false` on every one-shot boot. diff --git a/content/docs/deployment/cli.mdx b/content/docs/deployment/cli.mdx index 3c3e1a4521a..c79083cff33 100644 --- a/content/docs/deployment/cli.mdx +++ b/content/docs/deployment/cli.mdx @@ -530,6 +530,11 @@ Reports the `sys_secret` rows no producer references any more, and — only behi **Report-only by default: without `--delete` it writes nothing and deletes nothing.** It is never run for you; nothing on any boot or upgrade path invokes it. +The report boots your app read-only: no schema change, no seed rows, and a SQLite file +that does not exist is not created. Pointed at a database that lacks `sys_secret` (one +that was never booted, or the wrong `--database-url`), it refuses and exits 1 (under +`--json`: `"error": "scan_failed"`) instead of creating the table and reporting nothing. + ```bash os secret orphans # report (writes nothing) os secret orphans --json # the same report, machine-readable @@ -994,6 +999,17 @@ where the data lives. | `os migrate meta --stored` | Replay the metadata conversion chain over this deployment's `sys_metadata` rows and rewrite the ones still carrying a pre-protocol shape. Hygiene, not a gate — nothing depends on it having run | | `os migrate duplicates` | Report business identifiers already minted twice across the organization partitions, and the rows blocking the boot-time NULL-safe index tightenings — a read-only inventory as JSON on stdout. Renumbers nothing and writes nothing at all; run it before the boot-time tenancy repair, which overwrites part of the evidence | +**The boot itself writes nothing you did not ask for.** Each of these commands boots +your app to read its metadata. Without `--apply`, that boot is read-only, the same boot +`os migrate plan` takes: the schema sync is held back, the app's inline seed data is not +loaded, and a SQLite file that does not exist is not created. With `--apply`, the boot +creates missing tables and columns so the migration has somewhere to write, but it +still loads no seed data: the only rows that change are the migration's own. +One edge follows from the read-only boot. A dry run pointed at a database that lacks a +table it reads (a never-booted database, or the wrong `--database-url`) can fail and exit +1, naming the table, where it used to create the table and report nothing to do. Point +`--database-url` at the deployment's database, or boot the deployment once first. + ```bash os migrate files-to-references # Dry run: full report, writes nothing os migrate files-to-references --apply # Convert, verify, record the flag (prompts) diff --git a/packages/cli/src/commands/meta/resync.ts b/packages/cli/src/commands/meta/resync.ts index a16ba219939..b1c92228da8 100644 --- a/packages/cli/src/commands/meta/resync.ts +++ b/packages/cli/src/commands/meta/resync.ts @@ -128,9 +128,21 @@ export default class MetaResync extends Command { printStep('Booting runtime stack…'); } + // [#21391] A run that can never reach the write — no `--yes`, and nobody + // at a terminal to confirm (`--json`, or stdin not a TTY) — answers + // `confirmation_required` and writes nothing, so it boots READ-ONLY, the + // boot `os migrate plan` takes. A run that may write keeps the plain boot: + // `sys_permission_set` must exist before the resync writes into it, and + // the interactive prompt comes after the boot. + const mayWrite = flags.yes || (!flags.json && process.stdin.isTTY === true); + let stack; try { - stack = await bootSchemaStack({ jsonOutput: flags.json, databaseUrl: flags['database-url'] }); + stack = await bootSchemaStack({ + jsonOutput: flags.json, + databaseUrl: flags['database-url'], + ...(mayWrite ? {} : { deferSchemaDdl: true, readOnlyProbe: true }), + }); } catch (error: any) { if (flags.json) await emitJson({ error: error.message, ...errorCodeFields(error) }, 0, { compact: true }); else printError(error.message || String(error)); diff --git a/packages/cli/src/commands/migrate/files-to-references.ts b/packages/cli/src/commands/migrate/files-to-references.ts index 553b486b0f4..6dcb1ed1c3c 100644 --- a/packages/cli/src/commands/migrate/files-to-references.ts +++ b/packages/cli/src/commands/migrate/files-to-references.ts @@ -211,10 +211,15 @@ export default class MigrateFilesToReferences extends Command { let stack; try { + // [#21391] The dry run boots READ-ONLY, the boot `os migrate plan` + // takes: `deferSchemaDdl` holds schema DDL back on every SQL datasource, + // and `readOnlyProbe` keeps a missing sqlite file from being created. + // `--apply` keeps the plain boot: the tables must exist before it writes. stack = await bootSchemaStack({ jsonOutput: flags.json, databaseUrl: flags['database-url'], extraPlugins: await buildDataMigrationPlugins({ storage: true }), + ...(apply ? {} : { deferSchemaDdl: true, readOnlyProbe: true }), }); } catch (error: any) { if (flags.json) { await emitJson({ error: error.message, ...errorCodeFields(error) }, 0, { compact: true }); this.exit(1); } diff --git a/packages/cli/src/commands/migrate/multi-value-columns.no-auto-run.test.ts b/packages/cli/src/commands/migrate/multi-value-columns.no-auto-run.test.ts index dccbe14d607..a465f972f43 100644 --- a/packages/cli/src/commands/migrate/multi-value-columns.no-auto-run.test.ts +++ b/packages/cli/src/commands/migrate/multi-value-columns.no-auto-run.test.ts @@ -90,8 +90,14 @@ describe('nothing on the boot / reconcile path invokes this command (#11733)', ( ]; // Only this command's own suites. `commands/migrate/index.ts` deliberately // does NOT default to it either — the bare `os migrate` is the plan. + // + // [#21391] Plus one reader, named: the one-shot boot family's enumeration + // pin, which RUNS every `bootSchemaStack` caller's modes against a + // database and asserts the run wrote nothing it was not asked to. A test, + // never a boot path; any other reader still has to say why. + const FAMILY_PIN = 'utils/schema-migrate.one-shot-family.integration.test.ts'; expect(importers.filter((p) => !isTestFile(p))).toEqual([]); - expect(importers.every((p) => p.startsWith('commands/migrate/multi-value-columns.'))).toBe(true); + expect(importers.every((p) => p.startsWith('commands/migrate/multi-value-columns.') || p === FAMILY_PIN)).toBe(true); }); it('the command never routes the remedy through the reconciler', () => { diff --git a/packages/cli/src/commands/migrate/preview-read-only.integration.test.ts b/packages/cli/src/commands/migrate/preview-read-only.integration.test.ts index 72c1f387fc4..637d8da37b4 100644 --- a/packages/cli/src/commands/migrate/preview-read-only.integration.test.ts +++ b/packages/cli/src/commands/migrate/preview-read-only.integration.test.ts @@ -23,7 +23,8 @@ * report still names the work `--apply` would do — so the identity is not * the vacuous one of a walk that never read anything; * 2. the control: `--apply` still applies that work; - * 3. (SQLite) a preview pointed at a file that does not exist creates no file. + * 3. (SQLite) a preview pointed at a file that does not exist creates no file, + * and exits 1 with the refusal its changeset declared (#21391). * * ## The driver axis * @@ -54,10 +55,14 @@ import { SqlDriver } from '@objectstack/driver-sql'; import type { IObjectQLEngine } from '@objectstack/spec/contracts'; import MigrateMeta from './meta.js'; import MigrateAuditMetadataBodies from './audit-metadata-bodies.js'; -import { bootSchemaStack } from '../../utils/schema-migrate.js'; import { buildDataMigrationPlugins } from '../../utils/data-migration-plugins.js'; import { isExitSignal } from '../../utils/format.js'; +// [#10126] Pay the first transform of this dist-resolved workspace dep at +// MODULE LOAD: the fixture's served boot reaches it through a dynamic +// `import()` inside a clocked hook (`scripts/check-test-source-alias.mjs`). +import '@objectstack/runtime'; + const HERE = dirname(fileURLToPath(import.meta.url)); const CLI_ROOT = resolve(HERE, '..', '..', '..'); @@ -236,16 +241,20 @@ async function createFixture(cell: DialectCell): Promise { // The deployment as a served boot left it: every table either command reads, // the artifact's seed written, then an operator's edit to the seeded row, a // legacy flow row, and an audit copy of a datasource body in cleartext. - const stack = await bootSchemaStack({ - jsonOutput: false, - databaseUrl, - projectRoot: dir, - extraPlugins: await buildDataMigrationPlugins({ automation: true, audit: true }), - }); + // + // [#21391] A SERVED boot, not `bootSchemaStack`: no one-shot boot runs the + // seed loader any more, so the funnel cannot stand in for `os dev` here. + const { createStandaloneStack, Runtime } = await import('@objectstack/runtime'); + const served = await createStandaloneStack({ projectRoot: dir, databaseUrl }); + const runtime = new Runtime({ cluster: false }); + const kernel = runtime.getKernel(); + for (const plugin of served.plugins) await kernel.use(plugin as any); + for (const plugin of await buildDataMigrationPlugins({ automation: true, audit: true })) await kernel.use(plugin as any); + await runtime.start(); try { - const ql = stack.kernel.getService('objectql') as IObjectQLEngine; + const ql = kernel.getService('objectql') as IObjectQLEngine; const [acme] = await ql.find('rp_lead', { where: { name: 'Acme' } }, SYSTEM); - expect(acme?.status, 'the plain boot did not write the artifact seed — nothing to protect').toBe('open'); + expect(acme?.status, 'the served boot did not write the artifact seed — nothing to protect').toBe('open'); await ql.update('rp_lead', { id: acme.id, status: 'won' }, SYSTEM); await ql.insert('sys_metadata', { type: 'flow', @@ -254,7 +263,7 @@ async function createFixture(cell: DialectCell): Promise { metadata: JSON.stringify(LEGACY_FLOW), }, SYSTEM); } finally { - await stack.shutdown(); + await kernel.shutdown(); } const raw = probe(); try { @@ -469,6 +478,32 @@ for (const cell of DIALECT_CELLS) { expect(existsSync(path), `${path} was created by a preview`).toBe(false); } }, cell.timeout); + + // [#21391] The edge #21349's changeset declared BREAKING: a preview whose + // database lacks the table it reads used to create the table and answer + // "nothing to examine" with exit 0. It now refuses with exit 1 and names + // what it could not read. Both halves are asserted: the exit code a + // script reads, and the refusal the payload carries. + it('meta --stored without --apply on a database that does not exist exits 1 with the driver\'s refusal for sys_metadata', async () => { + const absent = join(fixture!.dir, 'data', 'never-started.db'); + const { payload, exitCode } = await runJson(meta, ['--stored', '--database-url', `file:${absent}`]); + + expect(exitCode).toBe(1); + expect(payload.code).toBe('DATABASE_ERROR'); + expect(payload.error).toContain("'sys_metadata'"); + }, cell.timeout); + + it('audit-metadata-bodies without --apply on a database that does not exist exits 1 with both tables counted unread', async () => { + const absent = join(fixture!.dir, 'data', 'never-started.db'); + const { payload, exitCode } = await runJson(auditBodies, ['--database-url', `file:${absent}`]); + + expect(exitCode).toBe(1); + expect(payload.apply).toBe(false); + // `failures` counts the tables whose rows were NOT examined. + expect(payload.report.failures).toBe(2); + expect(payload.report.scanned).toBe(0); + expect(Object.keys(payload.report.byObject).sort()).toEqual(['sys_activity', 'sys_audit_log']); + }, cell.timeout); } }); } diff --git a/packages/cli/src/commands/migrate/recorded-by.ts b/packages/cli/src/commands/migrate/recorded-by.ts index 99ca0de2624..ca7e938f5ff 100644 --- a/packages/cli/src/commands/migrate/recorded-by.ts +++ b/packages/cli/src/commands/migrate/recorded-by.ts @@ -96,10 +96,15 @@ export default class MigrateRecordedBy extends Command { let stack; try { + // [#21391] The dry run boots READ-ONLY, the boot `os migrate plan` + // takes: `deferSchemaDdl` holds schema DDL back on every SQL datasource, + // and `readOnlyProbe` keeps a missing sqlite file from being created. + // `--apply` keeps the plain boot: the tables must exist before it writes. stack = await bootSchemaStack({ jsonOutput: flags.json, databaseUrl: flags['database-url'], extraPlugins: await buildDataMigrationPlugins(), + ...(flags.apply ? {} : { deferSchemaDdl: true, readOnlyProbe: true }), }); } catch (error: any) { if (flags.json) { await emitJson({ error: error.message, ...errorCodeFields(error) }, 0, { compact: true }); this.exit(1); } diff --git a/packages/cli/src/commands/migrate/resume.ts b/packages/cli/src/commands/migrate/resume.ts index 3e047fd8a93..1fb23ff771d 100644 --- a/packages/cli/src/commands/migrate/resume.ts +++ b/packages/cli/src/commands/migrate/resume.ts @@ -105,10 +105,15 @@ export default class MigrateResume extends Command { let stack; try { + // [#21391] The list mode boots READ-ONLY, the boot `os migrate plan` + // takes: `deferSchemaDdl` holds schema DDL back on every SQL datasource, + // and `readOnlyProbe` keeps a missing sqlite file from being created. + // `--run` keeps the plain boot: the tables must exist before it writes. stack = await bootSchemaStack({ jsonOutput: flags.json, databaseUrl: flags['database-url'], extraPlugins: await buildDataMigrationPlugins(), + ...(flags.run ? {} : { deferSchemaDdl: true, readOnlyProbe: true }), }); } catch (error: any) { if (flags.json) { await emitJson({ error: error.message, ...errorCodeFields(error) }, 0, { compact: true }); this.exit(1); } diff --git a/packages/cli/src/commands/migrate/summary-nulls.ts b/packages/cli/src/commands/migrate/summary-nulls.ts index d5d51bb3f28..60fe01f4b2a 100644 --- a/packages/cli/src/commands/migrate/summary-nulls.ts +++ b/packages/cli/src/commands/migrate/summary-nulls.ts @@ -190,10 +190,15 @@ export default class MigrateSummaryNulls extends Command { let stack; try { + // [#21391] The dry run boots READ-ONLY, the boot `os migrate plan` + // takes: `deferSchemaDdl` holds schema DDL back on every SQL datasource, + // and `readOnlyProbe` keeps a missing sqlite file from being created. + // `--apply` keeps the plain boot: the tables must exist before it writes. stack = await bootSchemaStack({ jsonOutput: flags.json, databaseUrl: flags['database-url'], extraPlugins: await buildDataMigrationPlugins(), + ...(apply ? {} : { deferSchemaDdl: true, readOnlyProbe: true }), }); } catch (error: any) { if (flags.json) { await emitJson({ error: error.message, ...errorCodeFields(error) }, 0, { compact: true }); this.exit(1); } diff --git a/packages/cli/src/commands/migrate/value-shapes.ts b/packages/cli/src/commands/migrate/value-shapes.ts index f3c679f18d3..70e62148148 100644 --- a/packages/cli/src/commands/migrate/value-shapes.ts +++ b/packages/cli/src/commands/migrate/value-shapes.ts @@ -13,6 +13,7 @@ import { createTimer, emitJson, errorCodeFields, + isExitSignal, } from '../../utils/format.js'; import { bootSchemaStack } from '../../utils/schema-migrate.js'; import { buildDataMigrationPlugins } from '../../utils/data-migration-plugins.js'; @@ -134,10 +135,15 @@ export default class MigrateValueShapes extends Command { let stack; try { + // [#21391] The scan boots READ-ONLY, the boot `os migrate plan` + // takes: `deferSchemaDdl` holds schema DDL back on every SQL datasource, + // and `readOnlyProbe` keeps a missing sqlite file from being created. + // `--apply` keeps the plain boot: the tables must exist before it writes. stack = await bootSchemaStack({ jsonOutput: flags.json, databaseUrl: flags['database-url'], extraPlugins: await buildDataMigrationPlugins(), + ...(apply ? {} : { deferSchemaDdl: true, readOnlyProbe: true }), }); } catch (error: any) { if (flags.json) { await emitJson({ error: error.message, ...errorCodeFields(error) }, 0, { compact: true }); this.exit(1); } @@ -240,6 +246,14 @@ export default class MigrateValueShapes extends Command { this.exit(1); } } catch (error: any) { + // [#21391] `this.exit(1)` above is how a failed gate leaves, and it + // throws oclif's ExitError: rethrown, never re-reported. Caught here, it + // printed a second `--json` document (`{"error":"EEXIT: 1"}`) after the + // scan's own — the shape `summary-nulls` and `files-to-references` + // already guard against. A scan with unreadable objects fails the gate, + // and the read-only scan of a database without the app's tables has + // nothing else. + if (isExitSignal(error)) throw error; if (flags.json) { await emitJson({ error: error.message, ...errorCodeFields(error) }, 0, { compact: true }); this.exit(1); } printError(error.message || String(error)); this.exit(1); diff --git a/packages/cli/src/commands/secret/orphans.ts b/packages/cli/src/commands/secret/orphans.ts index aca253f56f7..40d081a473a 100644 --- a/packages/cli/src/commands/secret/orphans.ts +++ b/packages/cli/src/commands/secret/orphans.ts @@ -14,6 +14,8 @@ import { printStep, createTimer, emitJson, + errorCodeFields, + isExitSignal, } from '../../utils/format.js'; import { bootSchemaStack } from '../../utils/schema-migrate.js'; import type { @@ -211,6 +213,11 @@ export default class SecretOrphans extends Command { // attribution set is theirs, and without it nothing is attributable and // nothing is deletable (the safe direction, reported as a note). extraPlugins: [new PlatformObjectsPlugin(), new SettingsServicePlugin({ registerRoutes: false })], + // [#21391] The report boots READ-ONLY, the boot `os migrate plan` + // takes: `deferSchemaDdl` holds schema DDL back on every SQL + // datasource, and `readOnlyProbe` keeps a missing sqlite file from + // being created. `--delete` keeps the plain boot: it deletes rows. + ...(flags.delete ? {} : { deferSchemaDdl: true, readOnlyProbe: true }), }); } catch (error) { const message = error instanceof Error ? error.message : String(error); @@ -427,6 +434,17 @@ export default class SecretOrphans extends Command { for (const f of failed) printError(` ${f.id}: ${f.message}`); printInfo(`Keep ${exportPath} until you are certain the sweep was correct — it is the only record.`); if (failed.length > 0) this.exit(1); + } catch (error) { + // [#21391] A read the scan could not make is a refusal, and under + // `--json` a refusal is still one JSON document. The report boots + // read-only, so a database that lacks a table it reads (`sys_secret` on + // one never booted with the platform objects) is refused here, where + // the plain boot used to create the table and report nothing. + if (isExitSignal(error)) throw error; + const message = error instanceof Error ? error.message : String(error); + if (json) { await emitJson({ error: 'scan_failed', message, ...errorCodeFields(error) }, 1, { compact: true }); return; } + printError(message); + this.exit(1); } finally { await stack.shutdown(); } diff --git a/packages/cli/src/commands/storage/orphans.ts b/packages/cli/src/commands/storage/orphans.ts index 3f1f3f4dd54..49e80550490 100644 --- a/packages/cli/src/commands/storage/orphans.ts +++ b/packages/cli/src/commands/storage/orphans.ts @@ -100,10 +100,16 @@ export default class StorageOrphans extends Command { let stack; try { + // [#21391] The report boots READ-ONLY, the boot `os migrate plan` takes: + // `deferSchemaDdl` holds schema DDL back on every SQL datasource, and + // `readOnlyProbe` keeps a missing sqlite file from being created. This + // command has no writing mode. stack = await bootSchemaStack({ jsonOutput: flags.json, databaseUrl: flags['database-url'], extraPlugins: await buildDataMigrationPlugins({ storage: true }), + deferSchemaDdl: true, + readOnlyProbe: true, }); } catch (error: any) { if (flags.json) { diff --git a/packages/cli/src/utils/platform-migrations-arming.integration.test.ts b/packages/cli/src/utils/platform-migrations-arming.integration.test.ts index a7ba89be87c..f161042b49a 100644 --- a/packages/cli/src/utils/platform-migrations-arming.integration.test.ts +++ b/packages/cli/src/utils/platform-migrations-arming.integration.test.ts @@ -318,13 +318,15 @@ describe('#9380 the kernel:ready platform migrations, on the boots that reach th it('[read-only contract] a NON-deferred one-shot CLI boot also leaves it byte-identical', async () => { const before = await readState(); - // The half that a `deferSchemaDdl`-keyed policy would have missed, and the - // more dangerous one: `os migrate summary-nulls`, `value-shapes`, - // `recorded-by`, `resume`, `files-to-references` and `os migrate meta` all - // boot WITHOUT `deferSchemaDdl` and are still dry-run-by-default ("a dry - // run writes NOTHING"). If the arming were gated on deferral instead of on - // being a one-shot boot, every one of them would silently repair rows - // behind a report. + // The half that a `deferSchemaDdl`-keyed policy would have missed: the + // WRITE modes boot without `deferSchemaDdl` — `--apply` of `os migrate + // meta --stored`, `summary-nulls`, `value-shapes`, `recorded-by`, + // `files-to-references` and `audit-metadata-bodies`, `os migrate resume + // --run`, `os secret orphans --delete`, `os meta resync --yes`. (Every + // no-write mode boots deferred since #21391.) Each writes only what the + // operator confirmed; if the arming were gated on deferral instead of on + // being a one-shot boot, every one of them would also repair rows the + // operator never saw in the preview. const stack = await bootSchemaStack({ jsonOutput: false, databaseUrl: `file:${dbFile}`, diff --git a/packages/cli/src/utils/schema-migrate.deferred-ddl.integration.test.ts b/packages/cli/src/utils/schema-migrate.deferred-ddl.integration.test.ts index 7148b0592b7..398454201a5 100644 --- a/packages/cli/src/utils/schema-migrate.deferred-ddl.integration.test.ts +++ b/packages/cli/src/utils/schema-migrate.deferred-ddl.integration.test.ts @@ -22,6 +22,11 @@ import type { Plugin, PluginContext } from '@objectstack/core'; import { bootSchemaStack } from './schema-migrate.js'; import { composeForDeclarations } from './schema-migration-plugins.js'; +// [#10126] Pay the first transform of this dist-resolved workspace dep at +// MODULE LOAD: the second-datasource cases reach it through a dynamic +// `import()` inside a clocked `it()` body (`scripts/check-test-source-alias.mjs`). +import '@objectstack/service-datasource'; + const ARTIFACT = { // #8687: manifest fields under `manifest:` — the flat spelling is refused. manifest: { id: 'com.example.defer-smoke', name: 'Defer Smoke', version: '0.0.0', type: 'app' }, @@ -227,3 +232,117 @@ describe('bootSchemaStack({ deferSchemaDdl }) — the boot writes nothing (#3917 } }, 60_000); }); + +/** + * [#21391] The deferral covers EVERY SQL datasource the boot connects, not only + * the default one. + * + * `DeferSchemaDdlPlugin` used to arm the first `driver.*` SQL service it found, + * which is the default datasource. A second SQL datasource reaches the engine + * through `engine.registerDriver` alone (`DatasourceConnectionService.connect()`, + * driven by `AppPlugin.start()` for the datasources an artifact declares), and + * the connect then calls `syncObjectSchema` for the objects bound to it. That + * is boot schema sync on a database the operator pointed nothing at, in a + * dry run. This is the declared-datasource path, booted for real: the shared + * connection service, the real driver factory, a second SQLite file. + */ +describe('bootSchemaStack({ deferSchemaDdl }) — every SQL datasource the boot connects (#21391)', () => { + let dir: string; + let dbFile: string; + let secondFile: string; + const savedEnv: Record = {}; + + beforeEach(async () => { + dir = mkdtempSync(join(tmpdir(), 'os-defer-ds-')); + mkdirSync(join(dir, 'dist'), { recursive: true }); + mkdirSync(join(dir, 'data'), { recursive: true }); + dbFile = join(dir, 'data', 'app.db'); + secondFile = join(dir, 'data', 'second.db'); + writeFileSync(join(dir, 'dist', 'objectstack.json'), JSON.stringify({ + manifest: { id: 'com.example.defer-second-ds', name: 'Defer Second Datasource', version: '0.0.0', type: 'app' }, + objects: [ + { name: 'defer_home', fields: { label: { type: 'text' } } }, + // Bound to the second datasource: its connect syncs this object. + { name: 'defer_remote', datasource: 'second', fields: { label: { type: 'text' } } }, + ], + datasources: [ + { name: 'second', driver: 'sqlite', schemaMode: 'managed', origin: 'code', config: { filename: secondFile }, active: true }, + ], + })); + + // The second database exists and holds one table of its own, so "the boot + // changed nothing there" is a comparison, not the absence of a file. + const seed = new SqlDriver({ client: 'better-sqlite3', connection: { filename: secondFile }, useNullAsDefault: true }); + await (seed as any).knex.schema.createTable('remote_marker', (t: any) => { t.string('id').primary(); }); + await (seed as any).knex.destroy(); + + savedEnv.OS_ARTIFACT_PATH = process.env.OS_ARTIFACT_PATH; + savedEnv.NODE_ENV = process.env.NODE_ENV; + process.env.OS_ARTIFACT_PATH = join(dir, 'dist', 'objectstack.json'); + process.env.NODE_ENV = 'production'; + }); + + afterEach(() => { + process.env.OS_ARTIFACT_PATH = savedEnv.OS_ARTIFACT_PATH; + process.env.NODE_ENV = savedEnv.NODE_ENV; + try { rmSync(dir, { recursive: true, force: true }); } catch { /* ignore */ } + }); + + async function tablesOf(file: string): Promise { + const d = new SqlDriver({ client: 'better-sqlite3', connection: { filename: file }, useNullAsDefault: true }); + const k = (d as any).knex; + try { + const rows = await k.raw("SELECT name FROM sqlite_master WHERE type = 'table'"); + return (rows as Array<{ name: string }>).map((r) => r.name).filter((n) => !n.startsWith('sqlite_')).sort(); + } finally { + await k.destroy(); + } + } + + /** The shared connection service, wired the way `os serve` wires it, with the real factory. */ + async function datasourceAdmin(): Promise { + const { DatasourceAdminServicePlugin, createDefaultDatasourceDriverFactory } = await import('@objectstack/service-datasource'); + return new DatasourceAdminServicePlugin({ driverFactory: createDefaultDatasourceDriverFactory() }); + } + + it('a deferred boot creates nothing on the second datasource, and reports its work with the default\'s', async () => { + const stack = await bootSchemaStack({ + jsonOutput: false, + databaseUrl: `file:${dbFile}`, + deferSchemaDdl: true, + projectRoot: dir, + extraPlugins: [await datasourceAdmin()], + }); + try { + // Non-vacuity: the second datasource really connected and owns the object. + const engine = stack.kernel.getService('objectql'); + expect(engine.getDriverForObject('defer_remote')?.name).toBe('second'); + + expect(await tablesOf(secondFile)).toEqual(['remote_marker']); + expect(await tablesOf(dbFile)).not.toContain('defer_home'); + const pending = stack.pendingSchemaWork.map((p) => `${p.table}:${p.kind}`); + expect(pending).toContain('defer_home:create_table'); + expect(pending).toContain('defer_remote:create_table'); + } finally { + await stack.shutdown(); + } + }, 60_000); + + it('flushSchemaDdl performs the second datasource\'s work too', async () => { + const stack = await bootSchemaStack({ + jsonOutput: false, + databaseUrl: `file:${dbFile}`, + deferSchemaDdl: true, + projectRoot: dir, + extraPlugins: [await datasourceAdmin()], + }); + try { + const performed = (await stack.flushSchemaDdl()).map((p) => `${p.table}:${p.kind}`); + expect(performed).toContain('defer_remote:create_table'); + expect(await tablesOf(secondFile)).toEqual(['defer_remote', 'remote_marker']); + expect(await tablesOf(dbFile)).toContain('defer_home'); + } finally { + await stack.shutdown(); + } + }, 60_000); +}); diff --git a/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts b/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts new file mode 100644 index 00000000000..773f22e4f7f --- /dev/null +++ b/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts @@ -0,0 +1,533 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#21391] Every command that boots through `bootSchemaStack` keeps the two + * promises a one-shot boot makes, pinned across the whole family. + * + * 1. **A no-write mode leaves the database byte-identical**: the schema and + * every row, read on a connection of our own. A dry run, a scan or a + * report never reaches schema sync, never runs the seed loader, and never + * brings a missing SQLite file into existence. + * 2. **A write mode runs no seed write**: `--apply`, `--delete`, `--run`, + * `--yes` write what the command was asked to write, and the artifact's + * inline seed loader does not ride along. The operator never saw a seed + * write in the preview. + * + * ## Why one table, derived from source + * + * The defect was not in one command. Seven of the family booted the plain + * stack in their no-write mode, one call site at a time, each one reasonable + * on its own. So the family here is not a list someone remembered: it is every + * module under `src/` that value-imports `bootSchemaStack`, read off the + * source. A new caller fails the first case below, by file name, until its + * modes are declared in {@link CALLERS} and so run through the other two. + * + * ## The fixture + * + * A database a SERVED boot left behind: the artifact's inline seed written, + * then an operator's edit to the seeded row (`status: 'won'`). A seed replay + * puts the edit back to the seed's value and bumps `updated_at`. The commands + * then run against an artifact one step AHEAD of that database (one new + * field, one new object), so a boot that schema-syncs leaves a different + * schema behind. The same detection the card measured on `examples/app-crm`. + * + * SQLite only. The read-only boot is one code path for every dialect, and the + * live PostgreSQL leg of its two original members is + * `../commands/migrate/preview-read-only.integration.test.ts`. + */ + +import { describe, it, expect, beforeAll, afterAll, beforeEach, afterEach, vi } from 'vitest'; +import { copyFileSync, existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { randomBytes } from 'node:crypto'; +import { tmpdir } from 'node:os'; +import { dirname, join, relative, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { SqlDriver } from '@objectstack/driver-sql'; +import type { IObjectQLEngine } from '@objectstack/spec/contracts'; +import { isExitSignal } from './format.js'; +import { bootSchemaStack } from './schema-migrate.js'; +import { buildDataMigrationPlugins } from './data-migration-plugins.js'; +import MetaResync from '../commands/meta/resync.js'; +import MigrateAccountIssuer from '../commands/migrate/account-issuer.js'; +import MigrateApply from '../commands/migrate/apply.js'; +import MigrateAuditMetadataBodies from '../commands/migrate/audit-metadata-bodies.js'; +import MigrateDuplicates from '../commands/migrate/duplicates.js'; +import MigrateFilesToReferences from '../commands/migrate/files-to-references.js'; +import MigrateMeta from '../commands/migrate/meta.js'; +import MigrateMultiValueColumns from '../commands/migrate/multi-value-columns.js'; +import MigratePlan from '../commands/migrate/plan.js'; +import MigrateRecordedBy from '../commands/migrate/recorded-by.js'; +import MigrateResume from '../commands/migrate/resume.js'; +import MigrateSummaryNulls from '../commands/migrate/summary-nulls.js'; +import MigrateValueShapes from '../commands/migrate/value-shapes.js'; +import SecretOrphans from '../commands/secret/orphans.js'; +import StorageOrphans from '../commands/storage/orphans.js'; + +// [#10126] Pay the first transform of these dist-resolved workspace deps at +// MODULE LOAD: the commands reach them through dynamic `import()`s inside +// `run()`, which vitest clocks (`scripts/check-test-source-alias.mjs`). +import '@objectstack/runtime'; +import '@objectstack/objectql'; +import '@objectstack/platform-objects/plugin'; +import '@objectstack/service-settings'; +import '@objectstack/service-storage'; +import '@objectstack/plugin-audit'; + +const HERE = dirname(fileURLToPath(import.meta.url)); +const SRC = resolve(HERE, '..'); +const CLI_ROOT = resolve(HERE, '..', '..'); + +/** A boot plus a command run — oclif builds its command table on the first run in a process. */ +const CASE_TIMEOUT_MS = 120_000; + +/** Elevated so the fixture's writes bypass RLS on system objects. */ +const SYSTEM = { context: { isSystem: true } }; + +/** The app as the served boot ran it: one object and the inline seed it upserts on every start. */ +const SERVED_ARTIFACT = { + manifest: { id: 'com.example.one-shot-family', name: 'One Shot Family', version: '0.0.0', type: 'app' }, + objects: [{ name: 'osf_lead', fields: { name: { type: 'text' }, status: { type: 'text' } } }], + data: [{ object: 'osf_lead', externalId: 'name', mode: 'upsert', records: [{ name: 'Acme', status: 'open' }] }], +}; + +/** + * The same app one release later: a new field on the seeded object and a new + * object. A boot that schema-syncs adds the column and creates the table. + */ +const NEXT_ARTIFACT = { + ...SERVED_ARTIFACT, + objects: [ + { name: 'osf_lead', fields: { name: { type: 'text' }, status: { type: 'text' }, phone: { type: 'text' } } }, + { name: 'osf_note', fields: { body: { type: 'text' } } }, + ], +}; + +/** Env that would point a command's boot somewhere other than the fixture. */ +const OVERRIDING_ENV = ['OS_DATABASE_URL', 'DATABASE_URL', 'TURSO_DATABASE_URL', 'OS_DATABASE_DRIVER', 'OS_HOME'] as const; + +// ── The family, declared ───────────────────────────────────────────────────── + +/** One command, run through oclif exactly as the binary runs it. */ +type Invoke = (argv: string[]) => Promise; + +interface Mode { + /** How the case is named in the report. */ + label: string; + /** The argv after the command id. `@DB@` is the case's database URL, `@DIR@` the project directory. */ + argv: string[]; +} + +interface Caller { + run: Invoke; + /** Every mode of this command that writes nothing. `[]` only when the command has none. */ + noWrite: Mode[]; + /** Every mode that writes. `[]` when the command is report-only. */ + write: Mode[]; + /** Why `noWrite` or `write` is empty, when it is. */ + note?: string; +} + +const invoke = (command: { run: (argv: string[], opts: { root: string }) => Promise }): Invoke => + (argv) => command.run(argv, { root: CLI_ROOT }); + +/** + * Every `bootSchemaStack` caller, keyed by its path under `src/`, and the + * modes it has. The first case below holds this table equal to the source. + */ +const CALLERS: Record = { + 'commands/meta/resync.ts': { + run: invoke(MetaResync), + noWrite: [{ label: 'meta resync (no --yes)', argv: ['--database-url', '@DB@', '--json'] }], + write: [{ label: 'meta resync --yes', argv: ['--yes', '--database-url', '@DB@', '--json'] }], + }, + 'commands/migrate/account-issuer.ts': { + run: invoke(MigrateAccountIssuer), + noWrite: [{ label: 'migrate account-issuer', argv: ['--database-url', '@DB@', '--json'] }], + write: [], + note: 'a pre-flight report: it has no writing mode', + }, + 'commands/migrate/apply.ts': { + run: invoke(MigrateApply), + noWrite: [], + write: [{ label: 'migrate apply --yes', argv: ['--yes', '--database-url', '@DB@', '--json'] }], + note: 'the apply step itself: its preview is `os migrate plan`', + }, + 'commands/migrate/audit-metadata-bodies.ts': { + run: invoke(MigrateAuditMetadataBodies), + noWrite: [{ label: 'migrate audit-metadata-bodies', argv: ['--database-url', '@DB@', '--json'] }], + write: [{ label: 'migrate audit-metadata-bodies --apply', argv: ['--apply', '--yes', '--database-url', '@DB@', '--json'] }], + }, + 'commands/migrate/duplicates.ts': { + run: invoke(MigrateDuplicates), + // Always JSON: the command declares no `--json` flag. + noWrite: [{ label: 'migrate duplicates', argv: ['--database-url', '@DB@'] }], + write: [], + note: 'a pre-flight report: it has no writing mode', + }, + 'commands/migrate/files-to-references.ts': { + run: invoke(MigrateFilesToReferences), + noWrite: [{ label: 'migrate files-to-references', argv: ['--database-url', '@DB@', '--json'] }], + write: [{ label: 'migrate files-to-references --apply', argv: ['--apply', '--yes', '--database-url', '@DB@', '--json'] }], + }, + 'commands/migrate/meta.ts': { + run: invoke(MigrateMeta), + // Without `--stored` the command converts source files and boots nothing. + noWrite: [{ label: 'migrate meta --stored', argv: ['--stored', '--database-url', '@DB@', '--json'] }], + write: [{ label: 'migrate meta --stored --apply', argv: ['--stored', '--apply', '--yes', '--database-url', '@DB@', '--json'] }], + }, + 'commands/migrate/multi-value-columns.ts': { + run: invoke(MigrateMultiValueColumns), + noWrite: [{ label: 'migrate multi-value-columns', argv: ['--database-url', '@DB@', '--json'] }], + write: [{ label: 'migrate multi-value-columns --apply', argv: ['--apply', '--yes', '--database-url', '@DB@', '--json'] }], + }, + 'commands/migrate/plan.ts': { + run: invoke(MigratePlan), + noWrite: [{ label: 'migrate plan', argv: ['--database-url', '@DB@', '--json'] }], + write: [], + note: 'the preview of `os migrate apply`: it has no writing mode', + }, + 'commands/migrate/recorded-by.ts': { + run: invoke(MigrateRecordedBy), + noWrite: [{ label: 'migrate recorded-by', argv: ['--database-url', '@DB@', '--json'] }], + write: [{ label: 'migrate recorded-by --apply', argv: ['--apply', '--yes', '--database-url', '@DB@', '--json'] }], + }, + 'commands/migrate/resume.ts': { + run: invoke(MigrateResume), + noWrite: [{ label: 'migrate resume', argv: ['--database-url', '@DB@', '--json'] }], + // No run to resume: the boot happens and the command then reports the id + // unknown. The boot is what this case is about. + write: [{ label: 'migrate resume --run', argv: ['--run', 'run_absent_21391', '--yes', '--database-url', '@DB@', '--json'] }], + }, + 'commands/migrate/summary-nulls.ts': { + run: invoke(MigrateSummaryNulls), + noWrite: [{ label: 'migrate summary-nulls', argv: ['--database-url', '@DB@', '--json'] }], + write: [{ label: 'migrate summary-nulls --apply', argv: ['--apply', '--yes', '--database-url', '@DB@', '--json'] }], + }, + 'commands/migrate/value-shapes.ts': { + run: invoke(MigrateValueShapes), + noWrite: [{ label: 'migrate value-shapes', argv: ['--database-url', '@DB@', '--json'] }], + write: [{ label: 'migrate value-shapes --apply', argv: ['--apply', '--yes', '--database-url', '@DB@', '--json'] }], + }, + 'commands/secret/orphans.ts': { + run: invoke(SecretOrphans), + noWrite: [{ label: 'secret orphans', argv: ['--database-url', '@DB@', '--json'] }], + write: [{ + label: 'secret orphans --delete', + argv: ['--delete', '--export', '@DIR@/secret-export.json', '--no-declared-datasources', '--yes', '--database-url', '@DB@', '--json'], + }], + }, + 'commands/storage/orphans.ts': { + run: invoke(StorageOrphans), + noWrite: [{ label: 'storage orphans', argv: ['--database-url', '@DB@', '--json'] }], + write: [], + note: 'report-only: it has no writing mode', + }, +}; + +// ── The family, read off the source ────────────────────────────────────────── + +/** Every non-test TypeScript module under `src/`, as a path relative to it. */ +function sourceModules(dir: string): string[] { + const out: string[] = []; + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const abs = join(dir, entry.name); + if (entry.isDirectory()) out.push(...sourceModules(abs)); + else if (/\.ts$/.test(entry.name) && !/\.(test|spec)\.ts$/.test(entry.name) && !entry.name.endsWith('.d.ts')) { + out.push(relative(SRC, abs).split('\\').join('/')); + } + } + return out; +} + +/** A VALUE import of `bootSchemaStack` from the funnel's module. `import type` boots nothing. */ +const VALUE_IMPORT = /import\s+(?!type\b)\{([^}]*)\}\s*from\s*['"][^'"]*schema-migrate\.js['"]/g; + +function callsTheFunnel(source: string): boolean { + for (const m of source.matchAll(VALUE_IMPORT)) { + const names = m[1].split(',').map((s) => s.trim()); + if (names.some((n) => n === 'bootSchemaStack' || n.startsWith('bootSchemaStack '))) return true; + } + return false; +} + +// ── Running the real command ───────────────────────────────────────────────── + +interface RunResult { + payload: any; + exitCode: number; +} + +/** + * Run one command and capture its JSON document. `process.exitCode` is + * process-global and `emitJson` sets it, so it is saved and restored. The + * stdout spy is installed before the boot, because a `--json` boot reserves + * stdout and keeps whatever `process.stdout.write` is at that moment. + * `os meta resync` ends in `process.exit()`, so that is trapped too. + */ +async function runJson(command: Invoke, argv: string[]): Promise { + const savedExit = process.exitCode; + const swallow = ((_chunk: unknown, ...rest: unknown[]) => { + const cb = rest.find((a) => typeof a === 'function') as (() => void) | undefined; + if (cb) cb(); + return true; + }) as typeof process.stdout.write; + const stdout = vi.spyOn(process.stdout, 'write').mockImplementation(swallow); + vi.spyOn(process.stderr, 'write').mockImplementation(swallow); + vi.spyOn(console, 'log').mockImplementation(() => {}); + vi.spyOn(console, 'warn').mockImplementation(() => {}); + vi.spyOn(console, 'error').mockImplementation(() => {}); + let processExit: number | undefined; + vi.spyOn(process, 'exit').mockImplementation(((code?: number) => { + processExit = code ?? 0; + throw new Error(`__PROCESS_EXIT__:${processExit}`); + }) as never); + let thrownExit: number | undefined; + try { + try { + await command(argv); + } catch (error) { + const trapped = error instanceof Error && error.message.startsWith('__PROCESS_EXIT__'); + if (!trapped && !isExitSignal(error)) throw error; + if (!trapped) thrownExit = (error as { oclif?: { exit?: number } }).oclif?.exit; + } + const out = stdout.mock.calls.map((c) => String(c[0])).join('').trim(); + let payload: any; + try { + payload = JSON.parse(out); + } catch { + payload = { unparsed: out.slice(-400) }; + } + return { payload, exitCode: processExit ?? thrownExit ?? (process.exitCode as number | undefined) ?? 0 }; + } finally { + process.exitCode = savedExit; + vi.restoreAllMocks(); + } +} + +// ── The fixture database ───────────────────────────────────────────────────── + +let dir: string; +let templateDb: string; +const savedEnv: Record = {}; +const savedCwd = process.cwd(); + +function probe(dbFile: string): SqlDriver { + return new SqlDriver({ client: 'better-sqlite3', connection: { filename: dbFile }, useNullAsDefault: true }); +} + +/** + * The SERVING boot, `os dev` / `os serve`: the standalone stack plus the + * platform plugins the family's commands read, started, so the seed loader + * runs. Not `bootSchemaStack`, which is what is under test. + */ +async function bootServedStack(dbFile: string): Promise { + const { createStandaloneStack, Runtime } = await import('@objectstack/runtime'); + const { SettingsServicePlugin } = await import('@objectstack/service-settings'); + const stack = await createStandaloneStack({ projectRoot: dir, databaseUrl: `file:${dbFile}` }); + const runtime = new Runtime({ cluster: false }); + const kernel = runtime.getKernel(); + for (const plugin of stack.plugins) await kernel.use(plugin as any); + for (const plugin of await buildDataMigrationPlugins({ storage: true, automation: true, audit: true })) { + await kernel.use(plugin as any); + } + await kernel.use(new SettingsServicePlugin({ registerRoutes: false }) as any); + await runtime.start(); + try { + const ql = kernel.getService('objectql') as IObjectQLEngine; + const [acme] = await ql.find('osf_lead', { where: { name: 'Acme' } }, SYSTEM); + expect(acme?.status, 'the served boot did not write the artifact seed: nothing to protect').toBe('open'); + // The operator's edit a seed replay would put back. + await ql.update('osf_lead', { id: acme.id, status: 'won' }, SYSTEM); + } finally { + await kernel.shutdown(); + } +} + +/** The schema plus every row of every table, ordered, as plain JSON. */ +async function readState(dbFile: string): Promise { + const driver = probe(dbFile); + const k = (driver as any).knex; + try { + const schema = await k.raw('SELECT type, name, sql FROM sqlite_master ORDER BY type, name'); + const rows: Record = {}; + for (const entry of schema as Array<{ type: string; name: string }>) { + if (entry.type !== 'table' || entry.name.startsWith('sqlite_')) continue; + rows[entry.name] = await k.raw(`SELECT * FROM "${entry.name}" ORDER BY rowid`); + } + return { schema, rows }; + } finally { + await driver.disconnect(); + } +} + +/** + * The seeded row, projected onto the columns the served boot created. A write + * mode is ALLOWED to schema-sync (it adds `phone`), so `SELECT *` would differ + * for a reason that is not this pin's. + */ +async function readSeededRows(dbFile: string): Promise { + const driver = probe(dbFile); + try { + return await (driver as any).knex('osf_lead').select('id', 'name', 'status', 'updated_at', 'organization_id').orderBy('id'); + } finally { + await driver.disconnect(); + } +} + +let caseSeq = 0; + +/** A fresh copy of the served database, and the argv with its placeholders filled. */ +function prepareCase(argv: string[], opts: { absent?: boolean } = {}): { dbFile: string; argv: string[] } { + caseSeq += 1; + const caseDir = join(dir, 'cases', String(caseSeq)); + mkdirSync(caseDir, { recursive: true }); + const dbFile = join(caseDir, 'app.db'); + if (!opts.absent) copyFileSync(templateDb, dbFile); + return { + dbFile, + argv: argv.map((a) => a.replace('@DB@', `file:${dbFile}`).replace('@DIR@', caseDir)), + }; +} + +beforeAll(async () => { + for (const key of [...OVERRIDING_ENV, 'OS_ARTIFACT_PATH', 'NODE_ENV', 'OS_LIFECYCLE_DISABLED', 'OS_SECRET_KEY'] as const) { + savedEnv[key] = process.env[key]; + } + for (const key of OVERRIDING_ENV) delete process.env[key]; + delete process.env.OS_LIFECYCLE_DISABLED; + process.env.NODE_ENV = 'production'; + // The key this production-posture file needs — the served boot and every + // command that composes `SettingsServicePlugin` (`secret orphans`, and the + // storage arm of `files-to-references` / `storage orphans`) construct a + // `LocalCryptoProvider`, which refuses to start in production without one. + // Declared here rather than inherited from a persisted + // `$HOME/.objectstack/dev-crypto-key` (#16491): one fresh value for the + // whole file, so every boot in it decrypts what the others wrote, and never + // written to disk. + process.env.OS_SECRET_KEY = randomBytes(32).toString('hex'); + + dir = mkdtempSync(join(tmpdir(), 'os-21391-')); + mkdirSync(join(dir, 'dist'), { recursive: true }); + mkdirSync(join(dir, 'data'), { recursive: true }); + // The standalone stack reads `OS_ARTIFACT_PATH`, else `/dist/objectstack.json`. + process.env.OS_ARTIFACT_PATH = join(dir, 'dist', 'objectstack.json'); + writeFileSync(process.env.OS_ARTIFACT_PATH, JSON.stringify(SERVED_ARTIFACT)); + templateDb = join(dir, 'data', 'served.db'); + await bootServedStack(templateDb); + // The commands run against the next release of the app. + writeFileSync(process.env.OS_ARTIFACT_PATH, JSON.stringify(NEXT_ARTIFACT)); +}, 180_000); + +afterAll(() => { + process.chdir(savedCwd); + for (const [key, value] of Object.entries(savedEnv)) { + if (value === undefined) delete process.env[key]; + else process.env[key] = value; + } + try { rmSync(dir, { recursive: true, force: true }); } catch { /* ignore */ } +}); + +// The commands take `process.cwd()` as the project root, as a real invocation +// from the project directory does. +beforeEach(() => { process.chdir(dir); }); +afterEach(() => { process.chdir(savedCwd); }); + +// ── The cases ──────────────────────────────────────────────────────────────── + +describe('[#21391] the bootSchemaStack family is the table above', () => { + it('every module that value-imports bootSchemaStack is declared, and nothing else is', () => { + const found = sourceModules(SRC) + .filter((rel) => rel !== 'utils/schema-migrate.ts') + .filter((rel) => callsTheFunnel(readFileSync(join(SRC, rel), 'utf8'))) + .sort(); + // Non-vacuity: the detector finds the family's oldest member. + expect(found).toContain('commands/migrate/plan.ts'); + expect(found, 'a bootSchemaStack caller is missing from CALLERS, or a declared one no longer boots') + .toEqual(Object.keys(CALLERS).sort()); + }); + + it('every caller declares at least one mode, and an empty side says why', () => { + for (const [file, caller] of Object.entries(CALLERS)) { + expect(caller.noWrite.length + caller.write.length, file).toBeGreaterThan(0); + if (caller.noWrite.length === 0 || caller.write.length === 0) expect(caller.note, file).toBeTruthy(); + } + }); +}); + +const NO_WRITE = Object.entries(CALLERS).flatMap(([file, c]) => c.noWrite.map((m) => ({ file, run: c.run, ...m }))); +const WRITE = Object.entries(CALLERS).flatMap(([file, c]) => c.write.map((m) => ({ file, run: c.run, ...m }))); + +describe('[#21391] a no-write mode leaves the database byte-identical, boot included', () => { + it.each(NO_WRITE)('$label', async ({ run, argv }) => { + const c = prepareCase(argv); + const before = await readState(c.dbFile); + const result = await runJson(run, c.argv); + + // The run reached its report or its refusal, never a failed boot. + expect(result.payload?.error, JSON.stringify(result.payload).slice(0, 400)).not.toBe('boot_failed'); + expect(await readState(c.dbFile)).toEqual(before); + }, CASE_TIMEOUT_MS); +}); + +describe('[#21391] a no-write mode does not bring a missing SQLite file into existence', () => { + it.each(NO_WRITE)('$label', async ({ run, argv }) => { + const c = prepareCase(argv, { absent: true }); + const result = await runJson(run, c.argv); + + for (const path of [c.dbFile, `${c.dbFile}-wal`, `${c.dbFile}-shm`, `${c.dbFile}-journal`]) { + expect(existsSync(path), `${path} was created by a no-write mode`).toBe(false); + } + // Nothing to read there: whether the mode reports or refuses, `--json` + // still answers with one JSON document (the declared narrowing's face). + expect(result.payload?.unparsed, 'no JSON document on stdout').toBeUndefined(); + }, CASE_TIMEOUT_MS); +}); + +describe('[#21391] a write mode runs no seed write', () => { + it.each(WRITE)('$label', async ({ run, argv }) => { + const c = prepareCase(argv); + const before = await readSeededRows(c.dbFile); + expect(before).toEqual([expect.objectContaining({ name: 'Acme', status: 'won' })]); + + await runJson(run, c.argv); + + // A seed replay puts `won` back to `open` and bumps `updated_at`. + expect(await readSeededRows(c.dbFile)).toEqual(before); + }, CASE_TIMEOUT_MS); +}); + +describe('[#21391] a one-shot boot arms no lifecycle sweep', () => { + /** The sweep's two timers, as `LifecycleService` holds them. Private, read on purpose: they ARE the arming. */ + const armed = (kernel: any): boolean => { + const lifecycle = kernel.getService('lifecycle') as { initialTimer?: unknown; timer?: unknown }; + return lifecycle.initialTimer !== undefined || lifecycle.timer !== undefined; + }; + + it.each([ + ['the read-only boot', { deferSchemaDdl: true, readOnlyProbe: true }], + ['the plain boot a write mode takes', {}], + ] as Array<[string, Record]>)('%s', async (_name, options) => { + const c = prepareCase([]); + const stack = await bootSchemaStack({ jsonOutput: false, databaseUrl: `file:${c.dbFile}`, projectRoot: dir, ...options }); + try { + expect(armed(stack.kernel)).toBe(false); + } finally { + await stack.shutdown(); + } + }, CASE_TIMEOUT_MS); + + it('the control: a served boot of the same stack arms it', async () => { + const c = prepareCase([]); + const { createStandaloneStack, Runtime } = await import('@objectstack/runtime'); + const stack = await createStandaloneStack({ projectRoot: dir, databaseUrl: `file:${c.dbFile}`, skipSeedData: true }); + const runtime = new Runtime({ cluster: false }); + const kernel = runtime.getKernel(); + for (const plugin of stack.plugins) await kernel.use(plugin as any); + await runtime.start(); + try { + expect(armed(kernel)).toBe(true); + } finally { + await kernel.shutdown(); + } + }, CASE_TIMEOUT_MS); +}); diff --git a/packages/cli/src/utils/schema-migrate.teardown.integration.test.ts b/packages/cli/src/utils/schema-migrate.teardown.integration.test.ts index d11b31bb8ac..0fd2bd11dfa 100644 --- a/packages/cli/src/utils/schema-migrate.teardown.integration.test.ts +++ b/packages/cli/src/utils/schema-migrate.teardown.integration.test.ts @@ -22,10 +22,21 @@ * * Two assertions matter here and they pull in opposite directions on purpose: * - * - while the engine is LIVE, a CLI-booted stack really does audit (this is - * not #4747's rejected option C — "one-shot commands skip the audit" would - * make it permanently blind exactly where an operator has no other tool); + * - while the engine is LIVE, the stack really does audit; * - once the stack is torn down, the sweep issues no reads at all. + * + * ## [#21391] A one-shot boot arms no sweep at all + * + * The family ruling on #21391 took the sweep off every one-shot boot: it used + * to be armed on an unref'd 60-second timer, so "a one-shot never sweeps" was a + * timing fact, and a run longer than a minute did reap, rotate and audit in the + * middle of a dry run. `bootSchemaStack` now passes `armLifecycleSweep: false`, + * and `lifecycle.enabled` is the service's master switch, so the one-shot stack + * neither arms the schedule nor answers an explicit `sweep()`. That is the + * first case below. The two directions above, which are #4747's fix, are + * pinned on the composition that still sweeps: the same standalone stack + * booted without the one-shot policy, torn down through the same + * `kernel.shutdown()` path. */ import { describe, it, expect, beforeAll, afterAll } from 'vitest'; @@ -34,8 +45,16 @@ import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { bootSchemaStack } from './schema-migrate.js'; +// [#10126] Pay the first transform of this dist-resolved workspace dep at +// MODULE LOAD: the served composition reaches it through a dynamic `import()` +// inside a clocked `it()` body (`scripts/check-test-source-alias.mjs`). +import '@objectstack/runtime'; + interface LifecycleServiceLike { stopped: boolean; + /** The schedule's two timers. TypeScript-private, read on purpose: they ARE the arming. */ + initialTimer?: unknown; + timer?: unknown; sweep(): Promise<{ danglingReferences?: { unreadableObjects: string[]; aborted?: boolean } }>; } @@ -77,17 +96,44 @@ describe('[#4747] bootSchemaStack teardown disarms the ADR-0057 sweep', () => { try { rmSync(dir, { recursive: true, force: true }); } catch { /* ignore */ } }); - it('audits while the engine is live, and reads nothing once the stack is down', async () => { + it('a one-shot stack arms no sweep, and its teardown still closes the kernel and the pool (#21391)', async () => { const stack = await bootSchemaStack({ jsonOutput: false, databaseUrl: `file:${dbFile}`, projectRoot: dir }); + const lifecycle = stack.kernel.getService('lifecycle') as LifecycleServiceLike; + expect(lifecycle).toBeTruthy(); + expect(lifecycle.initialTimer).toBeUndefined(); + expect(lifecycle.timer).toBeUndefined(); + // The master switch is off, so an explicit sweep is inert too. + expect((await lifecycle.sweep()).danglingReferences).toBeUndefined(); + + await stack.shutdown(); + expect(stack.kernel.isRunning()).toBe(false); + expect(lifecycle.stopped).toBe(true); + const engine = stack.kernel.getService('objectql') as { + find(object: string, options: Record): Promise; + }; + await expect(engine.find('td_note', { limit: 1, context: { isSystem: true } })).rejects.toThrow(); + }, 120_000); + + it('audits while the engine is live, and reads nothing once the stack is down', async () => { + // The standalone stack WITHOUT the one-shot policy — the composition that + // still sweeps — torn down through `kernel.shutdown()`, the path + // `bootSchemaStack().shutdown()` takes since #4747. + const { createStandaloneStack, Runtime } = await import('@objectstack/runtime'); + const served = await createStandaloneStack({ projectRoot: dir, databaseUrl: `file:${dbFile}`, skipSeedData: true }); + const runtime = new Runtime({ cluster: false }); + const kernel = runtime.getKernel(); + for (const plugin of served.plugins) await kernel.use(plugin as any); + await runtime.start(); + const stack = { kernel, shutdown: async () => { await kernel.shutdown(); } }; // Resolved BEFORE teardown — the point is what this same instance does // afterwards, and service resolution post-shutdown is not the subject. const lifecycle = stack.kernel.getService('lifecycle') as LifecycleServiceLike; expect(lifecycle).toBeTruthy(); // ── While the engine is live: the audit runs for real ───────────────── - // Not "the CLI skips the audit" — it reads, and reports a clean, COMPLETE - // run. An empty `unreadableObjects` here is a fact about the database, - // which is precisely what it stopped being before this fix. + // It reads, and reports a clean, COMPLETE run. An empty + // `unreadableObjects` here is a fact about the database, which is + // precisely what it stopped being before #4747. expect(lifecycle.stopped).toBe(false); const live = await lifecycle.sweep(); expect(live.danglingReferences).toBeDefined(); diff --git a/packages/cli/src/utils/schema-migrate.ts b/packages/cli/src/utils/schema-migrate.ts index 3ad23cf6278..5a6895a7b4b 100644 --- a/packages/cli/src/utils/schema-migrate.ts +++ b/packages/cli/src/utils/schema-migrate.ts @@ -132,8 +132,19 @@ export function findSqlDriverForKernel(kernel: unknown): SqlDriverLike | null { } /** - * Arms the SQL driver's deferred-DDL mode before boot schema-sync can run - * (#3917). + * The kernel services under which `ObjectQLPlugin.init()` publishes the engine. + * Both names point at one `ObjectQL` instance; it is shadowed once. + */ +const ENGINE_SERVICES = ['objectql', 'data'] as const; + +/** One order for deferred work, whichever drivers it came from: the driver's own. */ +function sortPendingSchemaWork(work: PendingSchemaWork[]): PendingSchemaWork[] { + return work.sort((a, b) => a.table.localeCompare(b.table) || a.kind.localeCompare(b.kind)); +} + +/** + * Arms deferred-DDL mode on EVERY SQL driver the boot connects, before any of + * them can schema-sync (#3917, #21391). * * Timing is the whole point, and it is why this is a plugin rather than a call * in `bootSchemaStack`. The kernel runs **every** plugin's `init()` (Phase 1) @@ -142,6 +153,31 @@ export function findSqlDriverForKernel(kernel: unknown): SqlDriverLike | null { * `syncRegisteredSchemas` — the create-table/add-column DDL this issue is about * — in its `start()`. An `init()` that depends on the datasource plugin * therefore lands in the one window where the driver exists and no DDL has run. + * + * ## Every SQL datasource, not the first one (#21391) + * + * This used to arm the first `driver.*` SQL service it found, which is the + * default datasource. Any other SQL datasource reaches the engine through + * `engine.registerDriver` alone: `DatasourceConnectionService.connect()` (how + * `AppPlugin.start()` connects the datasources an artifact declares, and it + * then calls `syncObjectSchema` for the objects bound to each), a host + * plugin's `drivers.register`, or `ObjectQLPlugin.start()` handing the engine + * a `driver.*` service some later `init()` published. Each of those schema-synced + * on a dry run. So the deferral is armed on three paths: + * + * - every `driver.*` service published so far (the default among them); + * - every driver the engine already holds, through its public accessors (the + * default by name, and the driver each registered object resolves to); + * - every driver registered from here on: `registerDriver` is shadowed on the + * engine instance the kernel publishes, and the shadow arms the driver + * BEFORE the engine holds it, then forwards the same instance. This is the + * seam `createDeclarationBootWriteGuard` uses for the same reason; that + * module's header states why an own property on the instance is what every + * caller reaches. + * + * {@link drivers} keeps what was armed, so the stack can report every + * datasource's held-back work and flush all of it on the operator's say-so. + * No driver changes: each one keeps its own deferred set and its own flush. */ class DeferSchemaDdlPlugin { name = 'com.objectstack.cli.defer-schema-ddl'; @@ -149,25 +185,105 @@ class DeferSchemaDdlPlugin { /** Ordering, not optionality: our init must follow the one that registers `driver.*`. */ dependencies = ['com.objectstack.runtime.default-datasource']; - driver: SqlDriverLike | null = null; + /** Every driver this boot deferred, in the order it was armed. */ + readonly drivers: SqlDriverLike[] = []; + + /** Engines whose `registerDriver` is shadowed, and what to restore. */ + private readonly shadows: Array<{ + engine: Record; + original: PropertyDescriptor | undefined; + shadow: (...args: unknown[]) => unknown; + }> = []; init = async (ctx: any) => { - this.driver = findSqlDriverVia((name) => ctx.getService(name)); - if (!this.driver) { - // No SQL driver (memory/mongo) — nothing issues DDL, nothing to defer. - ctx.logger?.debug?.('[defer-schema-ddl] no SQL driver — deferral not armed'); + for (const name of SQL_DRIVER_SERVICES) { + try { this.arm(ctx.getService(name)); } catch { /* not registered */ } + } + const services: Map | undefined = ctx.getServices?.(); + for (const [name, service] of services?.entries?.() ?? []) { + if (typeof name === 'string' && name.startsWith('driver.')) this.arm(service); + } + for (const name of ENGINE_SERVICES) { + let engine: unknown; + try { engine = ctx.getService(name); } catch { /* not registered */ } + if (engine && typeof engine === 'object') this.shadowEngine(engine as Record); + } + if (this.drivers.length === 0) { + // No SQL driver yet (memory/mongo). Nothing has DDL to defer; a SQL + // driver registered later is armed on arrival by the shadow above. + ctx.logger?.debug?.('[defer-schema-ddl] no SQL driver yet — deferral armed on arrival'); + } + }; + + /** Arm one driver. Idempotent; a non-SQL driver has nothing to defer. */ + arm(driver: unknown): void { + if (!driver || typeof driver !== 'object') return; + const d = driver as SqlDriverLike; + if (this.drivers.includes(d)) return; + if (typeof d.setDeferredDdl === 'function') { + d.setDeferredDdl(true); + this.drivers.push(d); return; } - if (typeof this.driver.setDeferredDdl !== 'function') { + if (typeof d.detectManagedDrift === 'function' && typeof d.applyMigrationEntries === 'function') { // Fail loudly rather than silently boot-syncing: the caller asked for a // dry run and this driver cannot give one. + const name = (driver as { name?: unknown }).name; throw new Error( - 'The active SQL driver does not support deferred schema DDL, so this command cannot ' + - 'guarantee a dry run. Upgrade @objectstack/driver-sql.', + `The SQL driver${typeof name === 'string' ? ` '${name}'` : ''} does not support deferred schema DDL, ` + + 'so this command cannot guarantee a dry run. Upgrade @objectstack/driver-sql.', ); } - this.driver.setDeferredDdl(true); - }; + } + + /** Arm what the engine holds, and every driver it is handed from now on. */ + private shadowEngine(engine: Record): void { + if (this.shadows.some((s) => s.engine === engine)) return; // `objectql` and `data` are one instance + const registerDriver = engine.registerDriver; + if (typeof registerDriver !== 'function') return; + this.armHeld(engine); + const original = Object.getOwnPropertyDescriptor(engine, 'registerDriver'); + const shadow = (...args: unknown[]): unknown => { + // Arm BEFORE forwarding: the connect that registers a driver calls + // `syncObjectSchema` on it in the very next statement. + this.arm(args[0]); + return Reflect.apply(registerDriver as (...a: unknown[]) => unknown, engine, args); + }; + Object.defineProperty(engine, 'registerDriver', { + value: shadow, + writable: true, + configurable: true, + enumerable: original?.enumerable ?? false, + }); + this.shadows.push({ engine, original, shadow }); + } + + /** The drivers the engine already holds, through the accessors it makes public. */ + private armHeld(engine: Record): void { + const e = engine as { + getDefaultDriverName?: () => unknown; + getDriverByName?: (name: string) => unknown; + getDriverForObject?: (name: string) => unknown; + registry?: { getAllObjects?: () => unknown }; + }; + const name = e.getDefaultDriverName?.(); + if (typeof name === 'string') this.arm(e.getDriverByName?.(name)); + if (typeof e.getDriverForObject !== 'function') return; + const objects = e.registry?.getAllObjects?.(); + for (const obj of Array.isArray(objects) ? objects : []) { + const objectName = (obj as { name?: unknown } | null)?.name; + if (typeof objectName === 'string') this.arm(e.getDriverForObject(objectName)); + } + } + + /** Put every shadowed `registerDriver` back, unless something else now sits on top of ours. */ + release(): void { + for (const { engine, original, shadow } of this.shadows.splice(0)) { + if (Object.getOwnPropertyDescriptor(engine, 'registerDriver')?.value !== shadow) continue; + if (original) Object.defineProperty(engine, 'registerDriver', original); + else delete engine.registerDriver; + } + } } /** @@ -239,20 +355,28 @@ export async function bootSchemaStack( */ composeHostStack?: boolean; /** - * Boot WITHOUT touching the target database (#3917). + * Boot WITHOUT touching the target database's schema (#3917). + * + * Boot schema-sync issues create-table / add-column DDL, which used to + * happen before `os migrate plan` rendered its "dry run" and before + * `os migrate apply` asked `[y/N]`. With this set, every SQL driver the + * boot connects registers metadata but records the physical work instead + * of performing it ({@link SchemaStack.pendingSchemaWork}), so the plan + * describes the database as it actually is. Call + * {@link SchemaStack.flushSchemaDdl} after confirmation to perform the work. + * (The artifact's inline seed is off on every boot through here, set or + * not: see the `skipSeedData` note in the body.) * - * Boot schema-sync issues create-table / add-column DDL, and the artifact's - * inline seed writes rows — both used to happen before `os migrate plan` - * rendered its "dry run" and before `os migrate apply` asked `[y/N]`. With - * this set, the driver registers metadata but records the physical work - * instead of performing it ({@link SchemaStack.pendingSchemaWork}), and the - * seed is suppressed, so the boot is read-only and the plan describes the - * database as it actually is. Call {@link SchemaStack.flushSchemaDdl} after - * confirmation to perform the work. + * [#21391] **Every no-write mode sets this, with {@link readOnlyProbe}**: + * the read-only boot. A command's writes happen only in its own apply step, + * and a dry run, a scan or a report never reaches schema sync. The + * enumeration pin `schema-migrate.one-shot-family.integration.test.ts` runs + * every caller's no-write modes against a database and fails on any byte + * that moves, and fails by file name on a caller it has not been told + * about. * - * Commands that boot in order to READ AND WRITE DATA (`os meta resync`, - * `os migrate files-to-references`) must leave this off — they need the - * tables to exist. + * A WRITE mode that needs the tables to exist before it writes (`--apply` + * of the data commands, `os meta resync --yes`) leaves this off. */ deferSchemaDdl?: boolean; /** @@ -303,8 +427,20 @@ export async function bootSchemaStack( const stack = await createStandaloneStack({ projectRoot: opts.projectRoot ?? process.cwd(), ...(opts.databaseUrl ? { databaseUrl: opts.databaseUrl } : {}), - ...(defer ? { skipSeedData: true } : {}), + // [#21391] No seed loader on a one-shot CLI boot — unconditional, and NOT + // keyed on `deferSchemaDdl`, for the reason `runPlatformMigrations` below + // is not. The artifact's inline seed UPSERTS every seeded row on every + // boot (`updated_at` bumped, `organization_id` stamped, an operator's edit + // put back to the seed's value). Keyed on `defer`, it ran under every + // no-write mode that booted plain, and it still runs under every `--apply` + // / `--delete`: a write the operator never saw in the preview, riding along + // with the one they confirmed. Seeding stays with the boots that serve. + skipSeedData: true, ...(opts.readOnlyProbe ? { sqliteAbsentFile: 'empty-in-memory' as const } : {}), + // [#21391] No lifecycle sweep either. A one-shot boot used to arm it on an + // unref'd timer whose first run is a minute out, so "it never sweeps" was + // a timing fact. Not armed, it is a structural one. + armLifecycleSweep: false, // [#9380] No boot repair migrations on a one-shot CLI boot — unconditional, // and NOT keyed on `deferSchemaDdl`. // @@ -316,16 +452,18 @@ export async function bootSchemaStack( // every one of them is a command that reports or applies exactly what the // operator asked for: // - // • `os migrate plan` / `os migrate duplicates` boot deferred + read-only - // and are declared dry runs; - // • `os migrate meta` / `value-shapes` / `recorded-by` / `resume` / - // `summary-nulls` / `files-to-references` boot NOT deferred and are - // STILL dry-run-by-default ("a dry run writes NOTHING"). Keying this off - // `defer` would have left that whole second group repairing rows behind - // a report — the more dangerous half, and the quieter one; - // • `os migrate apply` / `os meta resync` do write, but only the change - // the operator confirmed. A repair riding along is a change they never - // saw in the plan (which is #8725's separate complaint). + // • every no-write mode — `os migrate plan` / `duplicates`, and the + // default mode of every other command booted here — boots deferred + + // read-only (#21391) and is a declared dry run or report; + // • the write modes — `--apply` of `os migrate meta --stored` / + // `value-shapes` / `recorded-by` / `summary-nulls` / + // `files-to-references` / `audit-metadata-bodies`, `os migrate resume + // --run`, `os secret orphans --delete`, `os meta resync --yes` — boot + // NOT deferred, so a `defer`-keyed policy would have left all of them + // repairing rows. They write, but only the change the operator + // confirmed; `os migrate apply` likewise. A repair riding along is a + // change they never saw in the plan (which is #8725's separate + // complaint). // // The serving boots — `os dev`, `os serve`, `os start` — do not come // through here and take the default, which is where an install gets @@ -340,8 +478,9 @@ export async function bootSchemaStack( for (const plugin of stack.plugins) { await kernel.use(plugin); } - if (defer) { - await kernel.use(new DeferSchemaDdlPlugin() as any); + const deferral = defer ? new DeferSchemaDdlPlugin() : null; + if (deferral) { + await kernel.use(deferral as any); } // #12938 — the deployment's own object set, when this command asked for it. // Registered here, after the data stack, for the same reason `extraPlugins` @@ -351,7 +490,8 @@ export async function bootSchemaStack( ? await buildSchemaMigrationPlugins({ basePlugins: stack.plugins, cwd: opts.projectRoot ?? process.cwd(), - skipSeedData: defer, + // The same answer the standalone stack got above: never on this boot. + skipSeedData: true, }) : { plugins: [], hostConfigPath: null, hostConfigLoaded: false, hostConfigError: null, @@ -400,9 +540,12 @@ export async function bootSchemaStack( // Read AFTER the pass above — that is the step which fills both of them. const managedTableCount = driver ? (driver as any).managedObjectFields?.size ?? 0 : 0; - const pendingSchemaWork = defer && driver?.previewDeferredSchemaWork - ? await driver.previewDeferredSchemaWork() - : []; + // [#21391] Every datasource the deferral armed, not only the default's. + const pendingSchemaWork: PendingSchemaWork[] = []; + for (const d of deferral?.drivers ?? []) { + if (d.previewDeferredSchemaWork) pendingSchemaWork.push(...(await d.previewDeferredSchemaWork())); + } + sortPendingSchemaWork(pendingSchemaWork); return { driver, @@ -433,9 +576,16 @@ export async function bootSchemaStack( return []; } }, - flushSchemaDdl: async () => (defer && driver?.flushDeferredSchemaDdl - ? await driver.flushDeferredSchemaDdl() - : []), + flushSchemaDdl: async () => { + // [#21391] Every armed datasource, in the order it was armed; then the + // stack stops deferring drivers registered from here on. + const performed: PendingSchemaWork[] = []; + for (const d of deferral?.drivers ?? []) { + if (d.flushDeferredSchemaDdl) performed.push(...(await d.flushDeferredSchemaDdl())); + } + deferral?.release(); + return sortPendingSchemaWork(performed); + }, composition, /** * Tear the one-shot stack down through the kernel's own teardown — the @@ -458,6 +608,7 @@ export async function bootSchemaStack( * `destroy()` closes the ones it owns); a second disconnect is a no-op. */ shutdown: async () => { + deferral?.release(); try { await kernel.shutdown(); } catch { /* teardown is best-effort */ } try { await driver?.disconnect?.(); } catch { /* ignore */ } // Only now — `kernel.shutdown()` is itself two INFO lines ("Graceful diff --git a/packages/cli/src/utils/schema-migration-plugins.ts b/packages/cli/src/utils/schema-migration-plugins.ts index c6f8722f373..890494177f5 100644 --- a/packages/cli/src/utils/schema-migration-plugins.ts +++ b/packages/cli/src/utils/schema-migration-plugins.ts @@ -1255,7 +1255,8 @@ const NOTHING_COMPOSED: SchemaMigrationComposition = Object.freeze({ * host already brought). * @param skipSeedData mirrors the boot's own setting onto a config-derived * `AppPlugin`, so the config path and the artifact path suppress the inline - * seed identically. + * seed identically. `bootSchemaStack` passes `true` on every boot (#21391): + * no one-shot CLI boot runs the seed loader. */ export async function buildSchemaMigrationPlugins(opts: { basePlugins: readonly unknown[]; diff --git a/packages/runtime/src/standalone-stack-lifecycle-sweep.test.ts b/packages/runtime/src/standalone-stack-lifecycle-sweep.test.ts new file mode 100644 index 00000000000..65e1a31984d --- /dev/null +++ b/packages/runtime/src/standalone-stack-lifecycle-sweep.test.ts @@ -0,0 +1,98 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// [#21391] `armLifecycleSweep: false` keeps the ADR-0057 lifecycle sweep from +// being armed at all, so a one-shot boot never sweeps by construction rather +// than because it exits before the first run is due. +// +// Two halves, pinned where each is observable: +// +// 1. The DECLARATION, read off the plugin the stack hands the kernel: the +// `lifecycle` options `ObjectQLPlugin` holds. A serving boot (the key +// omitted, or `true`) hands it exactly what it always did, nothing. +// 2. The EFFECT, on a real kernel over a SQLite file: after `start()`, the +// `LifecycleService`'s timers are unset with the key `false` and set +// without it. The timers ARE the arming, so they are read directly. + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { Runtime } from './runtime.js'; +import { createStandaloneStack } from './standalone-stack.js'; + +// [#10126] Pay the first transform of these dist-resolved workspace deps at MODULE +// LOAD: the stack and the boot import them lazily, inside a clocked `it()` body +// (`scripts/check-test-source-alias.mjs`, the clocked-window rule). +import '@objectstack/metadata'; +import '@objectstack/objectql'; +import '@objectstack/service-datasource'; + +const OBJECTQL_PLUGIN = 'com.objectstack.engine.objectql'; +const BOOT_TIMEOUT = 120_000; + +/** The `lifecycle` options as `ObjectQLPlugin` holds them. TypeScript-private, read on purpose. */ +function lifecycleOptions(plugins: readonly unknown[]): unknown { + const objectql = plugins.find((p: any) => p?.name === OBJECTQL_PLUGIN) as any; + expect(objectql, `stack must carry ${OBJECTQL_PLUGIN}`).toBeDefined(); + return objectql.lifecycleOptions; +} + +describe('[#21391] createStandaloneStack declares whether the lifecycle sweep is armed', () => { + const dirs: string[] = []; + const kernels: any[] = []; + let savedDisabled: string | undefined; + + beforeEach(() => { + // The env switch would make every reading below `false` for a reason + // that is not the key under test. + savedDisabled = process.env.OS_LIFECYCLE_DISABLED; + delete process.env.OS_LIFECYCLE_DISABLED; + }); + + afterEach(async () => { + for (const k of kernels.splice(0)) { + try { await k.shutdown(); } catch { /* noop */ } + } + if (savedDisabled === undefined) delete process.env.OS_LIFECYCLE_DISABLED; + else process.env.OS_LIFECYCLE_DISABLED = savedDisabled; + for (const d of dirs.splice(0)) { + try { rmSync(d, { recursive: true, force: true }); } catch { /* noop */ } + } + }); + + async function armedAfterStart(armLifecycleSweep: boolean | undefined): Promise { + const dir = mkdtempSync(join(tmpdir(), 'os-21391-lifecycle-')); + dirs.push(dir); + const stack = await createStandaloneStack({ + projectRoot: dir, + databaseUrl: `file:${join(dir, 'app.db')}`, + skipSeedData: true, + runPlatformMigrations: false, + ...(armLifecycleSweep === undefined ? {} : { armLifecycleSweep }), + }); + const runtime = new Runtime({ cluster: false }); + const kernel = runtime.getKernel(); + kernels.push(kernel); + for (const p of stack.plugins) await kernel.use(p); + await kernel.bootstrap(); + const lifecycle = kernel.getService('lifecycle') as { initialTimer?: unknown; timer?: unknown }; + return lifecycle.initialTimer !== undefined || lifecycle.timer !== undefined; + } + + it('hands ObjectQLPlugin `lifecycle: { enabled: false }` for `false`, and nothing otherwise', async () => { + const dir = mkdtempSync(join(tmpdir(), 'os-21391-lifecycle-decl-')); + dirs.push(dir); + const databaseUrl = `file:${join(dir, 'app.db')}`; + expect(lifecycleOptions((await createStandaloneStack({ projectRoot: dir, databaseUrl, armLifecycleSweep: false })).plugins)) + .toEqual({ enabled: false }); + expect(lifecycleOptions((await createStandaloneStack({ projectRoot: dir, databaseUrl, armLifecycleSweep: true })).plugins)) + .toBeUndefined(); + expect(lifecycleOptions((await createStandaloneStack({ projectRoot: dir, databaseUrl })).plugins)) + .toBeUndefined(); + }, BOOT_TIMEOUT); + + it('a started kernel holds no sweep timer with `false`, and holds one without the key', async () => { + expect(await armedAfterStart(false)).toBe(false); + expect(await armedAfterStart(undefined)).toBe(true); + }, BOOT_TIMEOUT); +}); diff --git a/packages/runtime/src/standalone-stack.ts b/packages/runtime/src/standalone-stack.ts index 515c1ea5f15..894ad41afaf 100644 --- a/packages/runtime/src/standalone-stack.ts +++ b/packages/runtime/src/standalone-stack.ts @@ -194,10 +194,11 @@ export const StandaloneStackConfigSchema = z.object({ */ dev: z.boolean().optional(), /** - * Suppress the artifact's inline boot seed (#3917). Set by one-shot - * commands that boot the stack only to READ metadata — `os migrate plan` / - * `os migrate apply` — so the boot cannot write demo rows into the - * operator's live database before they have confirmed anything. + * Suppress the artifact's inline boot seed (#3917). Set by every one-shot + * CLI boot (`bootSchemaStack`, #21391), apply and delete modes included, + * so the boot cannot write demo rows into the operator's live database. + * The seed's upserts rewrite every seeded row on each boot, and no command + * the operator ran asked for that. Seeding stays with the serving boots. */ skipSeedData: z.boolean().optional(), /** @@ -230,7 +231,7 @@ export const StandaloneStackConfigSchema = z.object({ * * Set `false` for a boot that must not repair anything behind the * operator's back. `bootSchemaStack` (the CLI's ONE one-shot boot funnel) - * passes `false` for every `os migrate *` / `os meta *` command: those are + * passes `false` for every command it boots: those are mostly * dry-run-by-default report commands, and a repair that fires under them * destroys the very evidence they were run to collect * (`packages/cli/src/commands/migrate/duplicates.integration.test.ts` @@ -239,6 +240,26 @@ export const StandaloneStackConfigSchema = z.object({ * boot an operator starts in order to RUN the install. */ runPlatformMigrations: z.boolean().optional(), + /** + * [#21391] Does this boot ARM the ADR-0057 lifecycle sweep (rotation, + * retention reaping, archiving, and the #4551 reference audit that rides + * the same clock)? + * + * Defaults to `true`: a serving boot owns its data's lifecycle, and the + * sweep is how a declared retention is enforced at all. + * + * Set `false` for a boot that exits before the sweep's first run is due. + * `bootSchemaStack` (the CLI's one-shot boot funnel) passes `false` for + * every `os migrate *` / `os meta *` / `os secret *` / `os storage *` + * command. Before this key, the guarantee that such a boot never swept was + * a timing fact: the first sweep waits `DEFAULT_LIFECYCLE_INITIAL_DELAY_MS` + * on an unref'd timer, so the one-shot was expected to exit first. With + * `false` the sweep is never armed (`ObjectQLPlugin`'s `lifecycle.enabled`), + * which makes it a structural fact. `enabled` is the service's master + * switch, so an explicit `sweep()` on such a boot is inert too: it returns + * an empty report and reads nothing. + */ + armLifecycleSweep: z.boolean().optional(), }); export type StandaloneStackConfig = z.input; @@ -789,6 +810,9 @@ export async function createStandaloneStack(config?: StandaloneStackConfig): Pro environmentId, runPlatformMigrations: cfg.runPlatformMigrations ?? true, hydrateMetadataFromDb: true, + // [#21391] Only a `false` is passed through, so a serving boot + // hands the plugin exactly the options it always did. + ...(cfg.armLifecycleSweep === false ? { lifecycle: { enabled: false } } : {}), }), ]; if (artifactBundle) {