Skip to content

Commit be97cf3

Browse files
test(dogfood): every test file runs in its own temporary working directory (#21919)
Fixes #21914 Clause-②: no ## What changes Every dogfood test file now runs in its own temporary working directory. The suite fails when any file leaves `.objectstack/data` in `packages/qa/dogfood`. - **`test/per-file-cwd.setup.ts`** (new) is a `setupFiles` entry, wired explicitly in BOTH projects of `vitest.config.ts`, because inline projects inherit nothing from the root block. `shared-showcase` keeps `isolate: false`; the module still runs once per file there. - At module top level, before the test file's own imports, it creates a directory under the run's temporary root and `chdir`s into it. - In `afterAll` it restores the previous cwd. That `afterAll` is also **the guard**: it THROWS when `packages/qa/dogfood/.objectstack/data` exists. The message names the directory, its entries and the remedy. It also says the named file may be a concurrent one on another worker rather than the writer, and whether the directory was already present when the file started. - **`test/per-file-cwd.global-setup.ts`** (new) is a root-level `globalSetup`. It runs once per run, covering both projects and each `OS_TEST_SHARD` slice (measured). - At the START it clears a stale `packages/qa/dogfood/.objectstack`, so a developer's earlier run never reds the suite. - It creates one temporary root for the run and hands it to the workers with `provide` / `inject`. - At the END it removes that root, which is **where the per-file directories are removed**. The removal is run-level, not per file, because the memoized `shared-showcase` boot keeps its SQLite handles open in the directory of the file that booted it. - The teardown judges nothing (see Evidence: a throwing teardown is a false green). - **`vitest.config.ts`** wires the two modules. A header section explains why there are two halves and why the guard is not in the teardown. - **`test/enterprise-organizations.ts`**: the module-level `probeOrganizations()` now passes this package's root as `hostRoot`, resolved from the module's location (`new URL('..', import.meta.url)`), not the cwd. This was measured to be needed; see Evidence. No per-file edits. The five files the card names, and the other 87 measured writers, are covered by the module with no change of their own. Test isolation only: `@objectstack/dogfood` is `private: true`, so no published package moves and there is no changeset (`skip-changeset`). ## The invariant for every dogfood author - **Each test file runs in its own temporary cwd.** Anything it writes relative to the cwd is its own, no other file sees it, and it is removed at the end of the run. A file needs no `mkdtemp` / `chdir` of its own. - **A package-relative read must resolve from the module's location** (`new URL('..', import.meta.url)`, `import.meta.dirname`), never from `process.cwd()`. The cwd is a temporary directory. - **A file that writes into `packages/qa/dogfood/.objectstack/data` fails the run.** That happens through an absolute path built from the package root, or through a `process.chdir()` back to the package directory before a boot. The fix is to write relative to the file's own cwd. - Files that already `chdir` into a temp dir of their own still work, because they restore to the per-file directory. Their own `chdir` is now redundant and harmless. ## Why (measured) A per-file probe over the whole suite measured 92 test files leaving `.objectstack/data/showcase_external.db` in the package directory, not the five the card names: - 7 leave the populated federated fixture (24576 B, 2 tables): the card's five, plus `showcase-demo-personas-loginable` and `showcase-demo-personas-membership`, which pass `onEnable` in the bundle. - 85 leave an empty SQLite file (4096 B, 0 tables). The showcase's declared external datasource has a cwd-relative filename, and its auto-connect creates the file on every showcase boot, `onEnable` or not. A later boot on the same runner found or missed the federated tables depending on which files ran before it, and that ordering is how PR #21905 went red only on dogfood shard 3/3. The seat chose this route (one module) and this guard (comment `6004950414` on #21914), on the dev's measurement (comment `6004909676`). ## Evidence All runs are at head `967ce88a`, under the shared verify lock, from a clean package directory. - **Whole suite**: `pnpm --filter @objectstack/dogfood test` gave `Test Files 205 passed | 1 skipped (206)` and `Tests 1591 passed | 9 skipped (1600)`. Afterwards `packages/qa/dogfood/.objectstack` does not exist, and no `os-dogfood-run-*` root is left in the temp dir. - **CI's three-shard split**: CI's dogfood leg exports `OS_TEST_SHARD=k/3` and `vitest.config.ts` turns it into vitest's `shard`. Here each shard ran as `OS_TEST_SHARD=k/3 pnpm --filter @objectstack/dogfood test`: the same vitest selection, without turbo, so no cached replay. Each exited 0 and left no `.objectstack`: | shard | Test Files | Tests | |---|---|---| | 1/3 | 69 passed (69) | 507 passed (507) | | 2/3 | 69 passed (69) | 461 passed, 1 skipped (462) | | 3/3 | 67 passed, 1 skipped (68) | 623 passed, 8 skipped (631) | The three add up to the whole run: 206 files, 1600 tests. - **Ablation (H4)** through `scripts/ablation-replace.mjs`, wrap mode. The central `process.chdir(...)` was replaced by the bare `mkdtempSync(...)`: anchor count 1 to 0, blob `0991eb9c` to `ee5a65be`. - With the chdir dropped, `showcase-external-autoconnect` and `showcase-search` ran: `Test Files 2 failed (2)`, `Tests 8 passed (8)`, exit 1. Each failed in the guard: `.../packages/qa/dogfood/.objectstack/data exists after this test file ran. Entries: showcase_external.db` (plus `-shm` / `-wal` for the shared-showcase file). - Restore was proven by the tool: blob after restore equals HEAD (`0991eb9c`), and `git diff HEAD` is empty. - The same two files then gave `2 passed`, exit 0, and left nothing. - No build step is involved: vitest loads the mutated module from source. - **Stale directory**: `.objectstack/data/x.db` was planted, then 9 files were run. Result: `Test Files 9 passed (9)`, exit 0, nothing left (the globalSetup cleared it). - **Census**: those 9 files are the 7 populated-fixture writers plus `showcase-search` and `showcase-permission-zoo`, both `shared-showcase` files. - **`hostRoot` line, measured both ways**, running `rls-multitenant`, `org-create-default-team` and `enterprise-organizations.test`: - Without the line (commit `4d07dc29`), the skip text read `not resolvable from /tmp/os-dogfood-run-.../file-...` and told the reader to declare the package in that temp directory's `package.json`. - With it (`967ce88a`), the text names `packages/qa/dogfood/`. - The verdict is the same both ways (skipped), because no framework package declares `@objectstack/organizations`. - **Guard placement**: a throwing `globalSetup` teardown was measured on vitest 4.1.11 to print `error during close` and still exit 0, a false green. So the guard is the per-file `afterAll`. (A teardown that sets `process.exitCode = 1` does exit 1, but the summary still reads all-passed.) - **Typecheck and lint**: `pnpm --filter @objectstack/dogfood typecheck` is green, and `tsc --listFiles` includes both new modules and `enterprise-organizations.ts`. `pnpm lint` exits 0. - **Gates**: 130 commands at `967ce88a`, the dispatch list plus `pnpm check:dispatcher-error-vocabulary` from `dispatch-gates --commands`. `dispatch-gates --ran`: `48 derived famil(ies) accounted for — 48 run, 0 NOT-MEASURED`. - `check:dual-build-cjs-loads` and `check:published-readme-exports` first exited 3 (dist prerequisite: 7 packages unbuilt). After building those 7, both exit 0. - The three PR-context scripts (`check-closing-target-claim`, `check-partof-closing-keyword`, `check-single-claim-paths`) are re-run with this PR's context; the results are in the report on the card. ## Open PRs that add dogfood files | PR | new file | boots the showcase | own `chdir` | under this PR | |---|---|---|---|---| | #21864 | `showcase-public-form-withdrawal-layers.dogfood.test.ts` | yes | no | Covered with no author action. Without this PR it would leave `.objectstack/data` in the package directory. | | #21917 | `organization-delete-federated-fixture.dogfood.test.ts` | yes, with `onEnable` | yes | Unaffected; its own `chdir` is redundant. | | #21906 | `external-import-code-datasource-namespace.dogfood.test.ts` (also edits three `external-*` files) | yes, with `onEnable` | yes | Unaffected. None of its files is edited here. | | #21877 | `datasource-contractless-credentials.dogfood.test.ts` | yes | yes | Unaffected. | | #21897 | `flow-node-config-values-at-registration.dogfood.test.ts` | no (fixture stack) | no | Runs in its own temp cwd; it reads nothing relative to the cwd. | None of these files reads a package-relative path through `process.cwd()`. Only their own `prevCwd` captures do. ## Acceptance notes - **Observation, not filed.** The showcase's external datasource is declared read-only (`schemaMode: 'external'`, `allowWrites: false`). Its auto-connect CREATES a missing `.objectstack/data/showcase_external.db`, plus `-wal` / `-shm` (measured on 85 harness boots). - The declaration's own comment in `showcase-external.datasource.ts` says that if the fixture file cannot be opened, "the boot stops with that as the reason rather than serving a showcase whose federation pages are quietly dead". - It was measured only through the verify harness's `bootStack`, never at a public door (`os start` / `os dev`), so it stays here. - **Latent, unreachable today.** `bootStack(..., { multiTenant: true })` also defaults its `hostRoot` to the cwd: `rls-multitenant.dogfood.test.ts:79`, and `attachments-permission-matrix.dogfood.test.ts:766` through `bootFixture`. Both are gated on `organizationsAvailable`, which is false in this repository because no framework package may declare `@objectstack/organizations` (ADR-0132). A run that declares it in this package would need those boots to pass the package root too. Carrier: whoever declares it. - The own `chdir` in `external-validate-sees-runtime-save`, `external-import-destructive-remedy`, PR #21906's file and PR #21917's file is now redundant. It is left untouched and can be removed once #21906 lands. Carrier: the `domain:cli` seat. - Attribution limit: under parallel workers, the guard can name a file that ran at the same time as the writer. The message says so, and says whether the directory was already present when the named file started. --- _Generated by [Claude Code](https://claude.ai/code/session_01RWZbGvPFcRKvUqASZtunCU)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent f5b8e29 commit be97cf3

4 files changed

Lines changed: 161 additions & 1 deletion

File tree

‎packages/qa/dogfood/test/enterprise-organizations.ts‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@
9494
*/
9595

9696
import { createHostImporter, hostImportFailureKind } from '@objectstack/types/node';
97+
import { fileURLToPath } from 'node:url';
9798

9899
/**
99100
* The multi-org package (ADR-0105 D12 as amended by ADR-0132) these gates need.
@@ -208,7 +209,9 @@ export async function probeOrganizations(
208209
* time when {@link MULTI_ORG_ENV} is declared but the package is missing — the
209210
* loud half of the contract.
210211
*/
211-
const probe = await probeOrganizations();
212+
// #21914: every dogfood file runs in its own temporary cwd (test/per-file-cwd.setup.ts),
213+
// so the host app is this package, resolved from this module's location, not the cwd.
214+
const probe = await probeOrganizations(fileURLToPath(new URL('..', import.meta.url)));
212215

213216
export const organizationsAvailable: boolean = probe.available;
214217

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
//
3+
// Run-level half of the dogfood suite's per-file working directory (#21914).
4+
// The per-file half, and the guard, live in `./per-file-cwd.setup.ts`. Both are
5+
// wired in `../vitest.config.ts`.
6+
//
7+
// At the START of a run it does two things:
8+
//
9+
// 1. It clears a stale `packages/qa/dogfood/.objectstack`. Such a directory is
10+
// left by a developer's earlier run on a tree without this isolation, or by
11+
// a crashed run. The guard judges only what THIS run leaves, so an old
12+
// leftover never reds a run that wrote nothing.
13+
// 2. It creates ONE temporary root for the run and hands it to every worker
14+
// through `provide` / `inject`. Each test file makes its own working
15+
// directory under that root.
16+
//
17+
// At the END of the run it removes that root, and with it every per-file
18+
// directory. The removal is run-level, not per-file: on the `shared-showcase`
19+
// project (`isolate: false`) one memoized boot serves every file on a worker,
20+
// and its SQLite handles stay open in the directory of the file that booted it.
21+
//
22+
// ⛔ This teardown never JUDGES anything. On vitest 4.1.11 an error thrown from
23+
// a globalSetup teardown is printed as `error during close` and the run still
24+
// exits 0 (measured), so a guard placed here would be a false green. The guard
25+
// is a throwing `afterAll` in the per-file module, which fails a test file.
26+
import { mkdtempSync, rmSync } from 'node:fs';
27+
import { tmpdir } from 'node:os';
28+
import { join } from 'node:path';
29+
import { fileURLToPath } from 'node:url';
30+
import type { TestProject } from 'vitest/node';
31+
32+
/** `packages/qa/dogfood`, resolved from this module's own location. */
33+
const PACKAGE_ROOT = fileURLToPath(new URL('..', import.meta.url));
34+
35+
declare module 'vitest' {
36+
export interface ProvidedContext {
37+
/** The run's temporary root; each test file makes its working directory under it. */
38+
dogfoodCwdRoot: string;
39+
}
40+
}
41+
42+
let runRoot: string | undefined;
43+
44+
export function setup(project: TestProject): void {
45+
rmSync(join(PACKAGE_ROOT, '.objectstack'), { recursive: true, force: true });
46+
runRoot = mkdtempSync(join(tmpdir(), 'os-dogfood-run-'));
47+
project.provide('dogfoodCwdRoot', runRoot);
48+
}
49+
50+
export function teardown(): void {
51+
if (runRoot) rmSync(runRoot, { recursive: true, force: true });
52+
}
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
//
3+
// Every dogfood test file runs in its OWN temporary working directory (#21914).
4+
// Wired as `setupFiles` in BOTH projects of `../vitest.config.ts`; the run-level
5+
// half is `./per-file-cwd.global-setup.ts`.
6+
//
7+
// ## Why
8+
//
9+
// The showcase declares its external datasource with a cwd-relative file,
10+
// `.objectstack/data/showcase_external.db`. Every showcase boot auto-connects
11+
// it, and SQLite CREATES the file when it is missing. The showcase's `onEnable`
12+
// also provisions its federated tables there. Before this module, every file
13+
// that booted the showcase from `packages/qa/dogfood` left that database in the
14+
// package directory: 92 files (measured), 7 of them with the tables populated.
15+
// A later boot on the same runner then found or missed the federated tables
16+
// depending on which files ran before it, so its outcome depended on shard
17+
// composition and file order.
18+
//
19+
// ## What it does
20+
//
21+
// At module top level, which runs before the test file's own imports, it makes
22+
// a directory under the run's temporary root and `chdir`s into it. `afterAll`
23+
// restores the previous working directory. The directories are removed at the
24+
// end of the run by the globalSetup, not here: the memoized `shared-showcase`
25+
// boot keeps its SQLite handles open in the first file's directory.
26+
//
27+
// The invariant for every dogfood author: a file runs in its own temporary
28+
// cwd, so anything cwd-relative it writes is its own and disappears with the
29+
// run. A file that READS a package-relative path must resolve it from its
30+
// module's location (`new URL('..', import.meta.url)`), never from
31+
// `process.cwd()`.
32+
//
33+
// ## The guard
34+
//
35+
// `afterAll` THROWS when `packages/qa/dogfood/.objectstack/data` exists, so a
36+
// file that writes outside its temporary cwd fails the run. The globalSetup
37+
// clears a stale directory at the start of the run, so the guard judges only
38+
// what this run leaves.
39+
import { afterAll, inject } from 'vitest';
40+
import { existsSync, mkdtempSync, readdirSync } from 'node:fs';
41+
import { join } from 'node:path';
42+
import { fileURLToPath } from 'node:url';
43+
44+
/** `packages/qa/dogfood`, resolved from this module's own location. */
45+
const PACKAGE_ROOT = fileURLToPath(new URL('..', import.meta.url));
46+
/** What a file must never leave in the package directory. */
47+
const LEFTOVER = join(PACKAGE_ROOT, '.objectstack', 'data');
48+
49+
const runRoot = inject('dogfoodCwdRoot');
50+
if (!runRoot) {
51+
throw new Error(
52+
'per-file-cwd.setup.ts: no run root was provided. The globalSetup ' +
53+
'`test/per-file-cwd.global-setup.ts` must be wired in packages/qa/dogfood/vitest.config.ts; ' +
54+
'without it this file would run in the package directory.',
55+
);
56+
}
57+
58+
const previousCwd = process.cwd();
59+
const presentAtStart = existsSync(LEFTOVER);
60+
process.chdir(mkdtempSync(join(runRoot, 'file-')));
61+
62+
afterAll(() => {
63+
process.chdir(previousCwd);
64+
if (!existsSync(LEFTOVER)) return;
65+
let entries: string;
66+
try {
67+
entries = readdirSync(LEFTOVER).join(', ') || '(empty)';
68+
} catch (e) {
69+
entries = `(unreadable: ${(e as Error).message})`;
70+
}
71+
throw new Error(
72+
`${LEFTOVER} exists after this test file ran. Entries: ${entries}. ` +
73+
'Every dogfood file runs in its own temporary working directory ' +
74+
'(test/per-file-cwd.setup.ts), so something wrote into the package directory instead: ' +
75+
'an absolute path built from the package root, or a process.chdir() back to it before a boot. ' +
76+
'Fix the writer so it writes only relative to its own working directory. ' +
77+
'The file named here may not be the writer: a file running at the same time on another ' +
78+
`worker can leave it too. It was ${presentAtStart ? 'ALREADY present' : 'absent'} ` +
79+
'when this file started.',
80+
);
81+
});

‎packages/qa/dogfood/vitest.config.ts‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,11 +131,30 @@ runProjectCliOverridePreflight({
131131
// task. `shard` is typed through `TestUserConfig` because vitest declares it on
132132
// its CLI options and not on `InlineConfig`; it sits on the ROOT `test` block
133133
// (the projects below do not carry it), where vitest resolves it for all of them.
134+
//
135+
// #21914 -- EVERY TEST FILE RUNS IN ITS OWN TEMPORARY WORKING DIRECTORY.
136+
// Showcase boots write `.objectstack/data/showcase_external.db` relative to the
137+
// cwd. From the package directory that file outlived the run, and a later boot's
138+
// federated state depended on which files ran before it. Two halves:
139+
// - `PER_FILE_CWD` is a `setupFiles` entry named in EACH project below, because
140+
// inline projects inherit nothing from this root block (the same measured gap
141+
// as `disableConsoleIntercept`). It `chdir`s into a fresh directory before the
142+
// test file's imports, restores the cwd in `afterAll`, and THROWS there when
143+
// `.objectstack/data` exists in the package directory: that throw is the guard.
144+
// - The `globalSetup` below is ROOT-level: one run, one call, covering both
145+
// projects and each `OS_TEST_SHARD` slice (measured). It clears a stale
146+
// `.objectstack` at the start and removes the run's temporary root at the end.
147+
// Its teardown judges nothing, because a throw there exits 0 on vitest 4.1.11.
148+
// Both modules' headers carry the rest, including what a dogfood author owes.
149+
const PER_FILE_CWD = './test/per-file-cwd.setup.ts';
150+
134151
export default defineConfig({
135152
test: {
136153
// The file-level slice, when the dogfood gate runs one (#20820) -- see the
137154
// section above `export default` for why it is spread and typed this way.
138155
...({ shard: process.env.OS_TEST_SHARD } satisfies Pick<TestUserConfig, 'shard'>),
156+
// #21914: the run-level half of the per-file working directory (section above).
157+
globalSetup: ['./test/per-file-cwd.global-setup.ts'],
139158
projects: [
140159
{
141160
test: {
@@ -152,6 +171,9 @@ export default defineConfig({
152171
name: 'shared-showcase',
153172
include: SHARED_SHOWCASE,
154173
isolate: false,
174+
// #21914: PER PROJECT, like the two settings above. With `isolate: false`
175+
// the module still runs once per file, so each file gets its own cwd.
176+
setupFiles: [PER_FILE_CWD],
155177
},
156178
},
157179
{
@@ -289,6 +311,8 @@ export default defineConfig({
289311
name: 'isolated',
290312
include: ['test/**/*.test.ts'],
291313
exclude: SHARED_SHOWCASE,
314+
// #21914: PER PROJECT, as in `shared-showcase` above.
315+
setupFiles: [PER_FILE_CWD],
292316
},
293317
},
294318
],

0 commit comments

Comments
 (0)