feat(spec): a refinement that never reaches the published JSON Schema now makes a noise

[os-litant]

Refs #18670 (item 1)

Clause-②: no

Triage on #18670 wrote the handoff order and reserved the second half for the maintainer. This PR is item 1 only: census the refinements whose rule never reaches the published JSON Schema, and make that fact make a noise. It narrows no published shape, removes no refinement, and adds no CI job — the ratchet lives inside packages/spec/scripts/build-schemas.ts, which check:authorable-surface already runs.

The premises, re-measured — two reproduce, one does not

The card was filed by the dispatching seat, so every claim in it was re-measured here with its own control.

① z.toJSONSchema drops a refinement — REPRODUCES. On zod 4.4.3 (the version packages/spec resolves), a plain record, the same record with a .refine(), and the same record with an aborting .refine() project byte-identically. Lit control in the same run: z.string().min(1) does move the bytes ("minLength":1), so the instrument can see a projected constraint. Pinned in packages/spec/scripts/dropped-refinements.test.ts.

② TraceSamplingConfig.condition.anyOf[0] — REPRODUCES, byte for byte. The generated file reads {"type":"object","propertyNames":{"type":"string"},"additionalProperties":{}}.

③ The consequence, as written — DOES NOT REPRODUCE at that slot. Measured against the live schema:

claim	measured
the published anyOf[0] accepts {dialect:'cel'}, "which the runtime refuses"	the runtime accepts it too. condition is z.union([z.record(z.string(), z.unknown()), ExpressionInputSchema]); the permissive record branch absorbs the object, so published and runtime agree here
"the string branch carries no non-empty constraint"	it carries minLength: 1
"the runtime requires non-blank after trim"	it does not — " " is accepted; only "" is refused, which minLength: 1 refuses as well

So the tracing slot is not a specimen of the gap. The gap itself is real and the card's own dedupe words point straight at the right file: ExpressionInputSchema carries the refinement "Expression requires at least one of source or ast", and the published packages/spec/json-schema/shared/ExpressionInput.json states only "required": ["dialect"]. {"dialect":"cel"} is accepted by the published file and refused by the runtime — the card's sentence, one file up from where it was written.

The census

Measured by the generator itself, on this branch, at zod 4.4.3:

reading	value
refinement call sites in packages/spec/src/** (excluding *.test.ts)	126 — .refine( 48 · .superRefine( 75 · .check( 3
published schemas carrying at least one dropped refinement	240
dropped refinement sites on published schemas	688
sites that did reach the published file	0
sites with no JSON form on either side to compare	3

The fan-out between 126 and 688 is the point: one refinement on a shared schema lands on every published file that embeds it. Per namespace: ui 123 · api 251 · data 102 · system 81 · automation 46 · kernel 32 · ai 14 · shared 13 · security 10 · identity 9 · integration 1.

⚠️ The dispatch's dark probe counted .refine( in 21 files. That count included *.test.ts and, more importantly, did not look for .superRefine( — which is 75 of the 126 sites. The population is larger than the card implies, in the card's own direction.

The whole census is committed as packages/spec/dropped-refinements.baseline.json, keyed by published file, each entry naming the paths at which a rule is dropped.

The noise, in three places

1. On the artifact. Each affected file under packages/spec/json-schema/** now carries x-dropped-refinements, naming its own sites — the sibling of the x-unprojectable-branches annotation [finding] Five filter operators ($gt/$gte/$lt/$lte/$between) reach NO published reference page — build-schemas.ts skips their whole schema over an unrepresentable z.date(), and the skip is silent #16431 already writes for the opposite direction. x- keywords are ignored by every validator, so the set of documents each schema accepts is byte-for-byte what it was.
2. In the build log. Every gen:schema / check:authorable-surface run reports the accepted population in full, the same discipline as the never-published ledger above it.
3. As a ratchet. A published schema that drops a refinement and is not in the ledger fails the generator; so does a ledger entry whose site list the build no longer observes, in either direction. The failure prints the corrected entry in full.

The detector is MEASURED, not asserted

collectDroppedRefinements does not trust the sentence above. For every node carrying a custom check it builds the same node without those checks — clone() recomputes the constraint bag from the check list — and compares the two projections byte for byte. A zod release that learns to project refinements therefore turns those sites projected and the ledger goes red asking to be emptied, instead of reading as current forever.

Two false readings were measured and closed while building it, both pinned:

• clone() does not carry .describe() text (it lives in z.globalRegistry, keyed by instance), so without a meta copy every described node read as projected.
• Walking a lazy node's _cachedInner memo reaches a second instance of the same graph, whose recursive $ref layout differs. All 80 sites the first build called projected were that, and none of them were about a refinement.

Proof it can fire, and proof it stays silent

A live ablation on this branch, both legs proven on disk rather than by exit code.

• Mutated — one real .refine() added to TestAssertionSchema in packages/spec/src/qa/testing.zod.ts (a namespace with zero recorded drops). Marker grep on disk: 0 -> 1; anchor grep 1 -> 0. pnpm --filter @objectstack/spec gen:schema exits 1, naming four published schemas and the exact path under each:

 4 published schema(s) drop a refinement and are not declared in dropped-refinements.baseline.json:
     + qa/TestAssertion  (1 site(s))
         ROOT  (object)      [the generator prints this position in angle brackets; respelled here]
     + qa/TestScenario  (1 site(s))
         steps.element.assertions.element  (object)
     + qa/TestStep  (1 site(s))
         assertions.element  (object)
     + qa/TestSuite  (1 site(s))
         scenarios.element.steps.element.assertions.element  (object)

• Restored — git checkout HEAD -- PATH; git hash-object equals the HEAD blob (198a4236e9c3e8f7a1c58c55dd4db7cbabc28c98) and git diff HEAD is empty for the target. The same command then exits 0 and prints the accepted population. The ablation script carries trap ... EXIT INT TERM with absolute paths, and treats an empty hash as a failure.
• Silent — the whole qa/ namespace had zero entries before that mutation, and the unit suite asserts silence on a live refinement-free schema (AggregationFunction) and on a synthetic schema whose only constraints project.

What this PR deliberately does NOT do

⛔ It does not narrow any published shape. Teaching the projection to emit what a refinement constrains — propertyNames, not, minLength and friends cover a lot of them — or declaring the artifact a floor, both change a public contract. That is triage's item 2, and it stops here with this report.

⛔ It does not delete or weaken a refinement. The runtime rule is correct; it is the projection that is silent.

One boundary worth naming: the annotation reaches the JSON file and not content/docs/references/** — a full gen:docs on this branch produces a zero-line diff. Rendering it on the reference pages is a docs-surface change and was left out.

Verification

Every reading below is from commit 74b5af39e1, the final commit on this branch, with each exit code captured after a redirect and never through a pipe.

command	exit
pnpm lint (eslint . --no-inline-config — the whole repo, no narrowing)	0
pnpm --filter @objectstack/spec exec vitest run --project local	0 — 487 files, 13900 tests
pnpm --filter @objectstack/spec typecheck (tsc --noEmit + scripts + test layers)	0
pnpm --filter @objectstack/spec check:generated	0 — all 15 generated artifacts up to date
pnpm --filter @objectstack/spec check:authorable-surface (the mode that runs this ratchet)	0
pnpm check:published-files · check:type-check-coverage · check:nul-bytes · check:merge-driver	0
check-adr-0087-registration · check-empty-changeset · check-changeset-no-major, each --base origin/main	0

The 66 families derived by node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack were all run; 62 exit 0 and the four that could not be answered locally are named in the acceptance notes above and in the dev report on the card.

Two readings worth recording because they were NOT about this diff: check:published-files first exited 1 with "origin/main and HEAD have no merge base in this checkout" — a shallow clone whose graft floor had moved past the branch point. git fetch --deepen 300 restored the merge base to the recorded branch point and the gate then exits 0. And a control for every zero reading above: the control-character sweep grep -naP over the five changed files exits 1 (no match) while the same expression over a file carrying one ESC byte exits 0 and prints it.

Acceptance notes

• Noticed, not filed — the annotation stops at the JSON tree. content/docs/references/** renders from packages/spec/json-schema/** but drops unknown x- keys, so a full gen:docs on this branch produces a zero-line diff. An author reading the reference page still sees nothing. Carrying it onto the page is a docs-surface change; nobody is mid-flight in that file today.
• Noticed, not filed — aborting reads the check-level flag only. .refine(fn, { abort: true }) is visible; a ctx.addIssue({ fatal: true }) written inside a superRefine body is a property of the issue raised at parse time, not of the check the graph carries, so it reads false. It is a report detail and never the verdict — such a site is still a drop. Pinned both ways in the test file.
• Noticed, not filed — the ledger's diff amplifies. One refinement on a shared schema lands on every published file that embeds it, so adding one .refine() can move several ledger entries at once (the ablation moved four). That is the true fan-out, and the gate prints every corrected entry in full, but a reviewer should expect the shape.
• Measured, not a finding — three derived families cannot be answered from one worktree. check:dual-build-cjs-loads, check:lean-entry-closure and check:type-check-debt answer PREREQUISITE NOT MET (exit 3) because they sweep the BUILT output of all 81 workspace packages; a spec-closure build does not satisfy them and a full workspace build is CI's Build Core job. check:pm-dispatch-gates is still running at 540s and is reported NOT MEASURED, not red. All four are reported as absences, never as passes.

Patch round 2 — Test Core (1/6) was red, and it was this PR's

CI on 05cdeb2611 read 29 success, 3 expected skips, one failure: @objectstack/spec#test:repo, 36 tests down in scripts/build-schemas-check-mode.test.ts, every one of them an expect(status).toBe(0) that got a 1. The shard is green on main, so it was ours.

Why neither earlier round saw it. packages/spec declares two vitest projects and two turbo tasks: test is vitest run --project local, test:repo is vitest run --project repo. Both rounds verified with test alone. test:repo owns the 31 files that read outside the package — including every fixture that spawns the real generator — and CI runs both. The ratchet this PR adds lives in the generator, so repo was exactly the project that could see it and exactly the one never run.

Defect 1 — the census was one reading per schema-evaluation mode, not one reading

lazySchema() (src/shared/lazy-schema.ts) returns the real schema under OS_EAGER_SCHEMAS=1 and a Proxy over it otherwise. gen:schema and check:authorable-surface both export that flag; the check-mode fixtures deliberately spawn the generator without it, and say so in their header.

The walk keyed its visited set on the schema instance. So a sub-schema reached both directly and through a lazySchema() edge was one node to the eager walk and two to the lazy one. Measured on ui/View, whose list / listViews.valueType and form / formViews.valueType pairs each reach one schema by both routes: 11 dropped sites eager, 13 lazy. Six ledger entries then disagreed with the build in one mode and agreed in the other — the same generator, the same tree, two censuses.

The key is now the node's _zod internals, with the Proxy resolved to the internals its facade prototype-delegates to. Two keys were rejected on measurement, and both rejections are recorded in the code:

• the instance — not mode-invariant, the defect above;
• the def — over-collapses in the other direction. clone() with no argument hands a second instance the first's def object, so keying on it dropped …options[3].object.fields.valueType from system/ChangeSet and system/MigrationOperation, a reading the accepted ledger does not make. That attempt was measured and reverted, not shipped.

_zod is per instance where the def is not, so it moves nothing in eager mode and makes the lazy walk agree with it. The committed ledger is untouched — no entry added, none removed, no count flattened.

Defect 2 — the sandboxes never mounted the new ledger

Each fixture builds a temp package tree that copies scripts/ and symlinks src/, node_modules/, package.json, then seeds the committed artefacts the generator reads. dropped-refinements.baseline.json was never added to the five builders, so the generator refused on a missing ledger before reaching whatever each fixture was about. The mount is now a list of committed package-root ledgers, so the next one is one entry rather than a sixth call to remember.

The pin, and its ablation

scripts/dropped-refinements.test.ts gains three cases: a shared sub-schema reached by two routes is one site, the same holds across a lazySchema() edge, and — the lit control — two genuinely distinct nodes carrying the same rule are still two sites.

⚠️ The first draft of that pin asserted nothing, and the ablation is what caught it: with the refinement one level down, the property schema is the same instance by either route, so the walk deduped on that alone and the pin passed against the very defect it was written for. Moving the rule onto the shared node makes identity the thing under test. Ablating the fix now reads [ 'direct', 'viaLazy' ] where it expects [ 'direct' ]; restore is git checkout HEAD -- PATH proven by an empty git diff HEAD, under a trap ... EXIT INT TERM with absolute paths.

Verification — this round, at 6eeebd726d

Exit codes captured into a file and read from $? after the redirect, never through a pipe.

command	before	after
pnpm --filter @objectstack/spec test:repo (project repo)	1 — 36 failed / 500 passed	0 — 536 passed, 31 files
pnpm --filter @objectstack/spec test (project local)	not re-measured this round	0 — 487 files, 13987 tests
pnpm --filter @objectstack/spec check:generated	not re-measured this round	0 — all 15 artefacts current
pnpm --filter @objectstack/spec check:authorable-surface	not re-measured this round	0 — 737 sites / 243 schemas
the same generator without OS_EAGER_SCHEMAS, tree-wide	not measured before	0 — 737 / 243, identical
pnpm --filter @objectstack/spec typecheck	—	0
pnpm lint (eslint . --no-inline-config, whole repo, no narrowing)	—	0
check:nul-bytes · check:cross-package-test-inputs · check:test-source-alias · check:type-check-coverage	—	0

The two mode readings are the load-bearing pair: 737 sites across 243 published schemas in both modes, where before the fix the lazy mode disagreed with the ledger on six entries.

⚠️ Declared narrowing. test:repo was not run as a single process: this container kills a foreground command at roughly ten minutes and that project takes about sixteen in CI. It was run as seven foreground chunks of the same project and config — one for the 30 other files (451 tests), six covering all 14 describe blocks of build-schemas-check-mode.test.ts (85 tests). 451 + 85 = 536, which is the count CI reports for the project. No chunk was skipped and none reported a failure.

⚠️ NOT MEASURED, reported as an absence and not as a pass: check:type-check-debt answers PREREQUISITE NOT MET (exit 3, its own code for "nothing was measured") because its --re-measure half needs the built dist of 30 workspace dependencies; its coverage half, check:type-check-coverage, exits 0. A full workspace build is CI's job, and the gate's own text says so.

⚠️ The round-1 verification table above this section is a reading from 74b5af39e1 and is left as written. Its census numbers — 240 schemas / 688 sites — are that commit's; the 243 / 737 here is the same measurement after a0127cfe26 recorded the 49 sites a sibling landing added.

Acceptance notes — patch round 2

• Noticed, not filed — a new package-root ledger is still a manual mount. The list makes it one line instead of five, but nothing mechanically holds the list equal to the set of ledgers the generator actually reads; the next one is caught by the same 36 red fixtures rather than by a gate that names it. Naming it would need the generator's read set derived statically. Nobody is mid-flight in these builders today.
• Noticed, not filed — the census's mode-invariance is pinned on a synthetic graph, not on the live one. The live discriminator would be spawning the generator twice per run, once per mode, which doubles the slowest gate in the package to pin a property the unit pin already fails on.

---

Generated by Claude Code

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

⚠️ 1 changed file(s) yielded no anchor (packages/spec/dropped-refinements.baseline.json), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files. Nothing else in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 1 changed package(s)).

What this run could not see

• 1 changed file(s) yielded no anchor (packages/spec/dropped-refinements.baseline.json) — pages documenting those are invisible to this run
• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 136 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 2085be2b2d8769c6227167bb3525f4fa72b9a486 → packageMentionDocs.