Repository navigation
Commit be97cf3
test(dogfood): every test file runs in its own temporary working directory (#21919)
Fixes #21914
Clause-②: no
## What changes
Every dogfood test file now runs in its own temporary working directory.
The suite fails when any file leaves `.objectstack/data` in
`packages/qa/dogfood`.
- **`test/per-file-cwd.setup.ts`** (new) is a `setupFiles` entry, wired
explicitly in BOTH projects of `vitest.config.ts`, because inline
projects inherit nothing from the root block. `shared-showcase` keeps
`isolate: false`; the module still runs once per file there.
- At module top level, before the test file's own imports, it creates a
directory under the run's temporary root and `chdir`s into it.
- In `afterAll` it restores the previous cwd. That `afterAll` is also
**the guard**: it THROWS when `packages/qa/dogfood/.objectstack/data`
exists. The message names the directory, its entries and the remedy. It
also says the named file may be a concurrent one on another worker
rather than the writer, and whether the directory was already present
when the file started.
- **`test/per-file-cwd.global-setup.ts`** (new) is a root-level
`globalSetup`. It runs once per run, covering both projects and each
`OS_TEST_SHARD` slice (measured).
- At the START it clears a stale `packages/qa/dogfood/.objectstack`, so
a developer's earlier run never reds the suite.
- It creates one temporary root for the run and hands it to the workers
with `provide` / `inject`.
- At the END it removes that root, which is **where the per-file
directories are removed**. The removal is run-level, not per file,
because the memoized `shared-showcase` boot keeps its SQLite handles
open in the directory of the file that booted it.
- The teardown judges nothing (see Evidence: a throwing teardown is a
false green).
- **`vitest.config.ts`** wires the two modules. A header section
explains why there are two halves and why the guard is not in the
teardown.
- **`test/enterprise-organizations.ts`**: the module-level
`probeOrganizations()` now passes this package's root as `hostRoot`,
resolved from the module's location (`new URL('..', import.meta.url)`),
not the cwd. This was measured to be needed; see Evidence.
No per-file edits. The five files the card names, and the other 87
measured writers, are covered by the module with no change of their own.
Test isolation only: `@objectstack/dogfood` is `private: true`, so no
published package moves and there is no changeset (`skip-changeset`).
## The invariant for every dogfood author
- **Each test file runs in its own temporary cwd.** Anything it writes
relative to the cwd is its own, no other file sees it, and it is removed
at the end of the run. A file needs no `mkdtemp` / `chdir` of its own.
- **A package-relative read must resolve from the module's location**
(`new URL('..', import.meta.url)`, `import.meta.dirname`), never from
`process.cwd()`. The cwd is a temporary directory.
- **A file that writes into `packages/qa/dogfood/.objectstack/data`
fails the run.** That happens through an absolute path built from the
package root, or through a `process.chdir()` back to the package
directory before a boot. The fix is to write relative to the file's own
cwd.
- Files that already `chdir` into a temp dir of their own still work,
because they restore to the per-file directory. Their own `chdir` is now
redundant and harmless.
## Why (measured)
A per-file probe over the whole suite measured 92 test files leaving
`.objectstack/data/showcase_external.db` in the package directory, not
the five the card names:
- 7 leave the populated federated fixture (24576 B, 2 tables): the
card's five, plus `showcase-demo-personas-loginable` and
`showcase-demo-personas-membership`, which pass `onEnable` in the
bundle.
- 85 leave an empty SQLite file (4096 B, 0 tables). The showcase's
declared external datasource has a cwd-relative filename, and its
auto-connect creates the file on every showcase boot, `onEnable` or not.
A later boot on the same runner found or missed the federated tables
depending on which files ran before it, and that ordering is how PR
#21905 went red only on dogfood shard 3/3. The seat chose this route
(one module) and this guard (comment `6004950414` on #21914), on the
dev's measurement (comment `6004909676`).
## Evidence
All runs are at head `967ce88a`, under the shared verify lock, from a
clean package directory.
- **Whole suite**: `pnpm --filter @objectstack/dogfood test` gave `Test
Files 205 passed | 1 skipped (206)` and `Tests 1591 passed | 9 skipped
(1600)`. Afterwards `packages/qa/dogfood/.objectstack` does not exist,
and no `os-dogfood-run-*` root is left in the temp dir.
- **CI's three-shard split**: CI's dogfood leg exports
`OS_TEST_SHARD=k/3` and `vitest.config.ts` turns it into vitest's
`shard`. Here each shard ran as `OS_TEST_SHARD=k/3 pnpm --filter
@objectstack/dogfood test`: the same vitest selection, without turbo, so
no cached replay. Each exited 0 and left no `.objectstack`:
| shard | Test Files | Tests |
|---|---|---|
| 1/3 | 69 passed (69) | 507 passed (507) |
| 2/3 | 69 passed (69) | 461 passed, 1 skipped (462) |
| 3/3 | 67 passed, 1 skipped (68) | 623 passed, 8 skipped (631) |
The three add up to the whole run: 206 files, 1600 tests.
- **Ablation (H4)** through `scripts/ablation-replace.mjs`, wrap mode.
The central `process.chdir(...)` was replaced by the bare
`mkdtempSync(...)`: anchor count 1 to 0, blob `0991eb9c` to `ee5a65be`.
- With the chdir dropped, `showcase-external-autoconnect` and
`showcase-search` ran: `Test Files 2 failed (2)`, `Tests 8 passed (8)`,
exit 1. Each failed in the guard:
`.../packages/qa/dogfood/.objectstack/data exists after this test file
ran. Entries: showcase_external.db` (plus `-shm` / `-wal` for the
shared-showcase file).
- Restore was proven by the tool: blob after restore equals HEAD
(`0991eb9c`), and `git diff HEAD` is empty.
- The same two files then gave `2 passed`, exit 0, and left nothing.
- No build step is involved: vitest loads the mutated module from
source.
- **Stale directory**: `.objectstack/data/x.db` was planted, then 9
files were run. Result: `Test Files 9 passed (9)`, exit 0, nothing left
(the globalSetup cleared it).
- **Census**: those 9 files are the 7 populated-fixture writers plus
`showcase-search` and `showcase-permission-zoo`, both `shared-showcase`
files.
- **`hostRoot` line, measured both ways**, running `rls-multitenant`,
`org-create-default-team` and `enterprise-organizations.test`:
- Without the line (commit `4d07dc29`), the skip text read `not
resolvable from /tmp/os-dogfood-run-.../file-...` and told the reader to
declare the package in that temp directory's `package.json`.
- With it (`967ce88a`), the text names `packages/qa/dogfood/`.
- The verdict is the same both ways (skipped), because no framework
package declares `@objectstack/organizations`.
- **Guard placement**: a throwing `globalSetup` teardown was measured on
vitest 4.1.11 to print `error during close` and still exit 0, a false
green. So the guard is the per-file `afterAll`. (A teardown that sets
`process.exitCode = 1` does exit 1, but the summary still reads
all-passed.)
- **Typecheck and lint**: `pnpm --filter @objectstack/dogfood typecheck`
is green, and `tsc --listFiles` includes both new modules and
`enterprise-organizations.ts`. `pnpm lint` exits 0.
- **Gates**: 130 commands at `967ce88a`, the dispatch list plus `pnpm
check:dispatcher-error-vocabulary` from `dispatch-gates --commands`.
`dispatch-gates --ran`: `48 derived famil(ies) accounted for — 48 run, 0
NOT-MEASURED`.
- `check:dual-build-cjs-loads` and `check:published-readme-exports`
first exited 3 (dist prerequisite: 7 packages unbuilt). After building
those 7, both exit 0.
- The three PR-context scripts (`check-closing-target-claim`,
`check-partof-closing-keyword`, `check-single-claim-paths`) are re-run
with this PR's context; the results are in the report on the card.
## Open PRs that add dogfood files
| PR | new file | boots the showcase | own `chdir` | under this PR |
|---|---|---|---|---|
| #21864 | `showcase-public-form-withdrawal-layers.dogfood.test.ts` |
yes | no | Covered with no author action. Without this PR it would leave
`.objectstack/data` in the package directory. |
| #21917 | `organization-delete-federated-fixture.dogfood.test.ts` |
yes, with `onEnable` | yes | Unaffected; its own `chdir` is redundant. |
| #21906 | `external-import-code-datasource-namespace.dogfood.test.ts`
(also edits three `external-*` files) | yes, with `onEnable` | yes |
Unaffected. None of its files is edited here. |
| #21877 | `datasource-contractless-credentials.dogfood.test.ts` | yes |
yes | Unaffected. |
| #21897 | `flow-node-config-values-at-registration.dogfood.test.ts` |
no (fixture stack) | no | Runs in its own temp cwd; it reads nothing
relative to the cwd. |
None of these files reads a package-relative path through
`process.cwd()`. Only their own `prevCwd` captures do.
## Acceptance notes
- **Observation, not filed.** The showcase's external datasource is
declared read-only (`schemaMode: 'external'`, `allowWrites: false`). Its
auto-connect CREATES a missing `.objectstack/data/showcase_external.db`,
plus `-wal` / `-shm` (measured on 85 harness boots).
- The declaration's own comment in `showcase-external.datasource.ts`
says that if the fixture file cannot be opened, "the boot stops with
that as the reason rather than serving a showcase whose federation pages
are quietly dead".
- It was measured only through the verify harness's `bootStack`, never
at a public door (`os start` / `os dev`), so it stays here.
- **Latent, unreachable today.** `bootStack(..., { multiTenant: true })`
also defaults its `hostRoot` to the cwd:
`rls-multitenant.dogfood.test.ts:79`, and
`attachments-permission-matrix.dogfood.test.ts:766` through
`bootFixture`. Both are gated on `organizationsAvailable`, which is
false in this repository because no framework package may declare
`@objectstack/organizations` (ADR-0132). A run that declares it in this
package would need those boots to pass the package root too. Carrier:
whoever declares it.
- The own `chdir` in `external-validate-sees-runtime-save`,
`external-import-destructive-remedy`, PR #21906's file and PR #21917's
file is now redundant. It is left untouched and can be removed once
#21906 lands. Carrier: the `domain:cli` seat.
- Attribution limit: under parallel workers, the guard can name a file
that ran at the same time as the writer. The message says so, and says
whether the directory was already present when the named file started.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01RWZbGvPFcRKvUqASZtunCU)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent f5b8e29 commit be97cf3
4 files changed
Lines changed: 161 additions & 1 deletion
File tree
- packages/qa/dogfood
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
94 | 94 | | |
95 | 95 | | |
96 | 96 | | |
| 97 | + | |
97 | 98 | | |
98 | 99 | | |
99 | 100 | | |
| |||
208 | 209 | | |
209 | 210 | | |
210 | 211 | | |
211 | | - | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
212 | 215 | | |
213 | 216 | | |
214 | 217 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
131 | 131 | | |
132 | 132 | | |
133 | 133 | | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
134 | 151 | | |
135 | 152 | | |
136 | 153 | | |
137 | 154 | | |
138 | 155 | | |
| 156 | + | |
| 157 | + | |
139 | 158 | | |
140 | 159 | | |
141 | 160 | | |
| |||
152 | 171 | | |
153 | 172 | | |
154 | 173 | | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
155 | 177 | | |
156 | 178 | | |
157 | 179 | | |
| |||
289 | 311 | | |
290 | 312 | | |
291 | 313 | | |
| 314 | + | |
| 315 | + | |
292 | 316 | | |
293 | 317 | | |
294 | 318 | | |
| |||
0 commit comments