Skip to content

test(spec): pin the OpenAPI WRITE-refusal test on the generator's diagnostic - #18790

Merged
os-bill merged 1 commit into
mainfrom
claude/issue-18591-openapi-refuses-write-output-assert
Sep 17, 2026
Merged

os-bill merged 1 commit into
mainfrom
claude/issue-18591-openapi-refuses-write-output-assert

Conversation

@os-bill

@os-bill os-bill commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

Fixes #18591

Clause-②: no

refuses to WRITE the artifact when the document is inconsistent
(packages/spec/scripts/openapi-self-consistency.test.ts) asserted only a non-zero exit
status and a missing artifact. A subprocess that never starts satisfies both, so the
case stayed green in exactly the scenario where the gate it exists to pin never ran once.

The fix is the shape its three sibling spawn tests already carry: assert the generator's
own diagnostic text as well. Test file only, 8 lines added, nothing else touched.

Which text to assert — measured, not assumed

The claim comment could not say what the generator actually prints for the injected
ApiErrorTypo ref. The mutated generator was run once and read:

Error: OpenAPI document is not self-consistent: 1 unresolvable $ref(s).
  - #/components/schemas/ApiErrorTypo  (referenced at components.schemas.Broken)
  defined components.schemas: [CreateRequest, UpdateRequest, SingleRecordResponse,
  ListRecordResponse, DeleteResponse, ApiError, BulkRequest, BulkResponse, BaseResponse, Broken]

So the sibling's pair does apply verbatim, and both halves are now asserted:
toMatch(/unresolvable \$ref/) and toContain('#/components/schemas/ApiErrorTypo').

Acceptance controls

Probe: park node_modules/.pnpm/tsx@4.23.12 aside, so npx tsx resolves the pnpm shim
and the shim's target is gone — the generator subprocess never starts. Proven in effect
before any result was read (npx tsx --version exit 1, Cannot find module …/tsx/dist/cli.mjs),
restored under a trap … EXIT INT TERM with absolute paths and proven byte-identical after
each leg (same inode 689295, same tar-stream sha256 e8d303f8…67e6).

LIT — the red leg

Same command both times: pnpm --filter @objectstack/spec exec vitest run --project local --maxWorkers=2 scripts/openapi-self-consistency.test.ts

tsx parked file counts refuses to WRITE …
before (2085be2b2d) Tests 8 failed | 13 passed (21) PASSED — the vacuous pass
after (cd4904d361) Tests 9 failed | 12 passed (21) FAILED

The new failure, in full:

FAIL  |local| scripts/openapi-self-consistency.test.ts > build-openapi.ts end to end >
  refuses to WRITE the artifact when the document is inconsistent
AssertionError: expected 'node:internal/modules/cjs/loader:1386…' to match /unresolvable \$ref/

All nine spawn tests in the file now go red under the probe; before this change, eight did.

DARK — normal tree

tsx in place result
before (2085be2b2d) Test Files 1 passed (1) · Tests 21 passed (21)
after (cd4904d361) Test Files 1 passed (1) · Tests 21 passed (21)

Nothing reddened. Whole package at cd4904d361: pnpm --filter @objectstack/spec test →
Test Files 485 passed | 1 skipped (486) · Tests 14015 passed | 1 skipped (14016), exit 0.
pnpm --filter @objectstack/spec typecheck → exit 0 (tsc --noEmit, check:scripts-typecheck,
check:test-typecheck; the test-typecheck debt ledger did not move).

Changeset — measured

npm pack --dry-run --json in packages/spec at cd4904d361: 275 published entries,
0 of them under scripts/, so the changed file is not in this package's files[].
Positive control on the same reading: src/**/*.zod.ts is present (src/ai/agent.zod.ts, …),
so the instrument is live. Nothing published moves ⇒ no changeset is owed; skip-changeset
applies and is declared, not omitted.

Gates

node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --ran … at cd4904d361:
56 derived families — 51 run exit 0, 5 NOT MEASURED (recorded exit 3, PREREQUISITE NOT MET:
check:dts-closure, check:dual-build-cjs-loads, check:lean-entry-closure,
check:sourcemap-no-sources-content, check:type-check-debt). All five read a built dist/
across the workspace and refuse without one; this diff emits nothing to any dist/, so they are
CI's closure-build run, not a red. Dependency-closure build (--filter '@objectstack/spec^...')
is an empty run: packages/spec has no workspace dependencies.

eslint --no-inline-config --format json on the changed file: 1 file, 0 errors, 0 warnings.
The repo-wide pnpm lint sweep is CI's run. Invariance leg, quoted from eslint.config.mjs's
own header: "this repo runs one eslint.config.mjs, which never enables type-aware linting
(no parserOptions.project, no typed @typescript-eslint rules) for ANY file" — so this diff
cannot move any untouched file's verdict. The universe count was not measured, so this is a
targeted run, not a proven narrowing.

Acceptance notes

  • Swept the repo for the same vacuous shape: git grep "status).not.toBe(0)" -- '*.test.ts'
    returns 9 hits across 4 files; the 5 outside this file each carry an output assertion on the
    adjacent line, so no second instance of this defect class exists in that spelling. Observation,
    not a card.
  • runGenerator() could immunise the whole spawn family structurally by refusing a launch
    failure once, instead of each case re-deriving it from diagnostic text. Out of scope here —
    the card prescribes the siblings' shape and forbids inventing one — and moot for this file
    now that all nine spawn cases assert output. Noted, not filed.

Generated by Claude Code

`refuses to WRITE the artifact when the document is inconsistent` asserted only
a non-zero exit status and a missing artifact. A subprocess that never starts
satisfies both, so the case stayed green in exactly the scenario where the gate
it exists to pin never ran at all — measured by parking tsx aside, where 8 of
the 9 spawn tests go red and this one is among the 13 that still pass.

Assert the generator's own dangling-ref diagnostic as well, which is the shape
its three sibling spawn tests already carry.

Claude-Session: https://claude.ai/code/session_01JbZnqu8bt6YqfJsr9vaFb3
Co-authored-by: Claude <noreply@anthropic.com>
@os-bill os-bill added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 17, 2026 — with Claude
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json d9ba33df4c54c6093b330ce97dbec6be443e9528 → packageMentionDocs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xs skip-changeset PR has no user-facing published change; bypasses the changeset gate tests tooling

Projects

None yet

2 participants