Skip to content

Commit 0d8bac2

Browse files
os-steveclaude
andauthored
fix(docs-audit): withhold the check-glyph when no page names the derived anchors (#11828)
The docs-drift headline ended in a green tick whenever anchors were derived, nothing went unanchored, and no hand-written page named any of them. That sentence reports the NAMING RELATION, and the relation only ever lists a page that ALREADY names a changed token -- so a PR that widens an enumerable vocabulary is invisible to it by construction: the new members' absence from the page is precisely the defect, and an absence names nothing. Measured on a real PR: six new flow-expression functions derived 9 anchors, matched 0 pages, rendered the tick -- while the flows page carried a binding table the same PR had just made incomplete. The verdict now states only what the run measured and leaves the clean-bill glyph to a run that earned one. It does NOT catch that class; nothing in the anchor model can. It stops the run from claiming it did. check-drift-comment.mjs pins the new verdict byte-exact on that state and pins it ABSENT on all four neighbouring states, so a notice that started rendering everywhere would fail there. Claude-Session: https://claude.ai/code/session_015ahemw8RcTgqtxrj15PEZx Co-authored-by: Claude <noreply@anthropic.com>
1 parent 6757eb2 commit 0d8bac2

3 files changed

Lines changed: 96 additions & 4 deletions

File tree

‎.github/workflows/docs-drift-check.yml‎

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,13 @@ name: Docs Drift Check
2525
# on one day (#11180, #11262). The fold was right and stays; what changed is that a run
2626
# with an unanchored file now says so where a reviewer reads it, and withholds its ✅.
2727
#
28+
# Since #11356 the ✅ is withheld from the last arm that carried it: "N anchor(s) derived,
29+
# no hand-written page names any of them". That sentence reports the NAMING RELATION, and
30+
# a page is listed only when it already names a changed token — so a PR that widens an
31+
# enumerable vocabulary (a function table, an allowed-value set) is structurally invisible
32+
# to it, because the new members' absence from the page is precisely the defect. ⚠️ The
33+
# narrowing does NOT catch that class; it stops the run from claiming it did.
34+
#
2835
# And since #9519 it states WHICH TREE it read. On `pull_request`, actions/checkout gives
2936
# the MERGE of base and head, so the row set is a fact about a commit that exists on no
3037
# branch a reader can name — while the comment's own re-derivation command sent them to
@@ -303,11 +310,31 @@ jobs:
303310
// ⛔ The ✅ is DROPPED whenever a file went unanchored, deliberately: a green
304311
// tick is the clean-bill glyph, and "no page names the anchors I derived" is
305312
// not a clean bill when some of the diff derived no anchor to name.
313+
//
314+
// ⛔ And since #11356 the ✅ is gone from this arm ENTIRELY — including the run
315+
// where nothing went unanchored, which is where it used to live. "No page names
316+
// the anchors I derived" is a fact about the NAMING RELATION, not about the
317+
// docs: a page is listed because it ALREADY names a changed token, so a change
318+
// that WIDENS an enumerable vocabulary can never be caught here — the new
319+
// members' absence from the page IS the defect, and an absence names nothing.
320+
// Measured on #11347 (six new flow-expression functions): 9 anchors derived, 0
321+
// pages named them, ✅ — while `content/docs/automation/flows.mdx` carried a
322+
// table enumerating the available bindings that the PR had just made
323+
// incomplete, and a reviewing seat almost passed the PR on that tick.
324+
//
325+
// ⚠️ This does NOT catch that class, and must not be read as doing so. Nothing
326+
// in the anchor model can (three in-model routes were measured to fail on the
327+
// one real instance; the authoring-mark route that would catch it is escalated
328+
// as #11817). What it stops is the run CLAIMING it did: the verdict now says
329+
// only what it measured — anchors were derived, no page named them — and leaves
330+
// the clean-bill glyph to a run that earned one.
306331
const headline = anchorList.length === 0
307332
? (anchorless.length
308333
? `${notCovered} Nothing else in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from **${pkgs.length}** changed package(s)).`
309334
: `Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from **${pkgs.length}** changed package(s)), so **this run has no opinion** about the docs.`)
310-
: `**${anchorList.length}** anchor(s) derived from **${pkgs.length}** changed package(s); no hand-written page names any of them.${anchorless.length ? ` ${notCovered}` : ' ✅'}`;
335+
: (anchorless.length
336+
? `**${anchorList.length}** anchor(s) derived from **${pkgs.length}** changed package(s); no hand-written page names any of them. ${notCovered}`
337+
: `**${anchorList.length}** anchor(s) derived from **${pkgs.length}** changed package(s); no hand-written page names any of them, so this run has **nothing to list** — **not a clean bill of health**. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.`);
311338
// The tree identity is rendered here only when anchors WERE derived: that
312339
// is the run with an opinion ("no page names any of them"), and a reader
313340
// whose own tree does yield a row is owed the reason. With no anchors the

‎scripts/docs-audit/README.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -209,6 +209,9 @@ reader has to expand a fold to find is one a reader does not find. The failure #
209209
records was never the tool lying; it was the tool never signalling its own limits at the
210210
point of use.
211211

212+
Since #11356 the same posture reaches the ✅ itself — see **The verdict when nothing names
213+
the anchors** below.
214+
212215
### Measured, before and after
213216

214217
Ten real PRs, each re-derived at its own merge base with its own docs corpus. `docs` rows:
@@ -241,6 +244,33 @@ run says so and points at the coarse set — where the old tool's 22 rows were e
241244
mentioning `@objectstack/cli`. A CLI **command name** (`os meta resync`) is exactly the
242245
recall class the shape guard costs us: it is a lowercase word, so it cannot anchor.
243246

247+
### The verdict when nothing names the anchors
248+
249+
That state — anchors derived, nothing left unanchored, no page naming any of them — used
250+
to end the headline in a ✅. It no longer does (#11356). The sentence reports the **naming
251+
relation**, and the relation only ever lists a page that ALREADY names a changed token, so
252+
a PR that **widens an enumerable vocabulary** is invisible to it by construction: the new
253+
members' absence from the page is precisely the defect, and an absence names nothing.
254+
255+
Measured on #11347 — six new flow-expression functions (`round`/`floor`/`ceil`/`abs`/
256+
`min`/`max`) — the run derived 9 anchors, matched 0 pages, and rendered the ✅, while
257+
`content/docs/automation/flows.mdx` carried a table enumerating the available bindings
258+
that the same PR had just made incomplete. A reviewing seat almost passed the PR on that
259+
tick.
260+
261+
⚠️ **The narrowing does not catch that class.** Nothing in the anchor model can: anchoring
262+
the new members is impossible by construction, the siblings live in carriers the diff
263+
never touches, and the container symbol is named by no page. The authoring-mark route
264+
that would catch it is escalated as #11817. What the narrowing changes is that the run no
265+
longer **claims** it did — the verdict states only what it measured, and the clean-bill
266+
glyph is left to a run that earned one. The rendering is pinned in both directions by
267+
`check-drift-comment.mjs`, which asserts the verdict byte-exact on that state and asserts
268+
it ABSENT on all four neighbouring states.
269+
270+
How often it renders, re-derived over the 40 first-parent commits ending at `e43b18fd9`:
271+
3 of the 17 package-touching runs (18%) — `20a452e664`, `f213793ddb`, `dd4113ec0b` — so it
272+
is a rare notice rather than a per-PR banner, which is what keeps it readable.
273+
244274
**Cost** (the card's open question): the anchor derivation reads the same 178-page corpus
245275
the old one did, plus the 18 route-registrar/ledger sources (~875 KB) and one `git show`
246276
per changed file per side. Measured end-to-end on the ten PRs above, `node affected-docs.mjs`

‎scripts/docs-audit/check-drift-comment.mjs‎

Lines changed: 38 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,16 @@
3535
*
3636
* diff → the real affected-docs.mjs → its real --json → the real comment script → bytes
3737
*
38+
* Since #11356 it pins one more verdict the same way: the `anchors-derived-none-named`
39+
* state — anchors derived, nothing unanchored, no page naming any of them — used to end
40+
* in a ✅. That glyph read as "docs verified" on a change whose own page had just gone
41+
* incomplete, because a page is listed only when it ALREADY names a changed token, so a
42+
* PR that widens an enumerable vocabulary can never be caught by the relation (the new
43+
* members' absence from the page IS the defect). The narrowing does not catch that class;
44+
* it stops the run from claiming it did. Both readings are on the record below: the
45+
* verdict is pinned byte-exact on that state and pinned ABSENT on all four neighbours,
46+
* because a notice that renders everywhere reports nothing.
47+
*
3848
* Each case declares the mapper facts it depends on (`anchors` / `anchorless` / `docs`)
3949
* and those are asserted BEFORE the rendered text is. Without that, a change in how the
4050
* mapper classifies a `.md` file would quietly move every case onto the same branch and
@@ -182,6 +192,15 @@ const NAMES_THE_ANCHOR = '# Guide\n\nThe cap is `MAX_WIDGETS`, and it bounds the
182192
const ADDS_ANCHOR = 'export function renderWidget(x: number) {\n return x + 1;\n}\n\nexport const MAX_WIDGETS = 5;\n';
183193
const README_EDITED = '# @objectstack/demo\n\nRun `os demo` to start. Then run `os demo studio` for the UI.\n';
184194

195+
/** The #11356 verdict, byte-exact — rendered by exactly one of the five cases below. */
196+
const ANCHORS_DERIVED_NONE_NAMED =
197+
'**1** anchor(s) derived from **1** changed package(s); no hand-written page names any of them,'
198+
+ ' so this run has **nothing to list** — **not a clean bill of health**. This check sees only'
199+
+ ' pages that NAME a derived anchor: one that documents this change in prose, or enumerates it'
200+
+ ' in an authoring dialect, names none and stays invisible to it on every run.';
201+
/** The phrase that tells that verdict apart from every neighbouring one. */
202+
const NARROWED_PHRASE = 'nothing to list';
203+
185204
/**
186205
* `want` is the mapper contract each case rides on — asserted before any text is, so a
187206
* case that silently stopped exercising its branch fails here instead of passing there.
@@ -193,6 +212,7 @@ const CASES = [
193212
base: { 'content/docs/guide.mdx': NAMES_NOTHING },
194213
change: { 'packages/demo/README.md': README_EDITED },
195214
want: { anchors: 0, anchorless: ['packages/demo/README.md'], docs: 0 },
215+
narrowed: false,
196216
expect: (headline, body) => {
197217
check('readme-only', 'the headline names the blind spot', true, headline.includes('NOT COVERED by this run'));
198218
check('readme-only', 'the headline names the file that went unanchored', true, headline.includes('packages/demo/README.md'));
@@ -203,15 +223,22 @@ const CASES = [
203223
},
204224
{
205225
id: 'anchored-source',
206-
what: 'an anchorable source file, no page naming it — the ✅ arm',
226+
what: 'an anchorable source file, no page naming it — the anchors-derived-none-named arm (#11356)',
207227
base: { 'content/docs/guide.mdx': NAMES_NOTHING },
208228
change: { 'packages/demo/src/widget.ts': ADDS_ANCHOR },
209229
want: { anchors: 1, anchorless: [], docs: 0 },
230+
narrowed: true,
210231
expect: (headline) => {
211232
check('anchored-source', 'the blind-spot sentence does NOT render — it is a report, not boilerplate',
212233
false, headline.includes('NOT COVERED'));
213-
check('anchored-source', 'the headline is the untouched ✅ text', true,
214-
headline === '**1** anchor(s) derived from **1** changed package(s); no hand-written page names any of them. ✅');
234+
check('anchored-source', 'the headline is the narrowed #11356 verdict, byte-exact', true,
235+
headline === ANCHORS_DERIVED_NONE_NAMED);
236+
// The point of the narrowing, asserted as its own reading: this state no longer
237+
// carries the clean-bill glyph. Byte equality above would catch a ✅ appended to
238+
// the end, but not one moved elsewhere in the line.
239+
check('anchored-source', 'the clean-bill glyph is withheld', false, headline.includes('✅'));
240+
check('anchored-source', 'and the verdict says what it could not see, not just what it found', true,
241+
headline.includes('not a clean bill of health'));
215242
},
216243
},
217244
{
@@ -220,6 +247,7 @@ const CASES = [
220247
base: { 'content/docs/guide.mdx': NAMES_NOTHING },
221248
change: { 'packages/demo/src/widget.ts': ADDS_ANCHOR, 'packages/demo/README.md': README_EDITED },
222249
want: { anchors: 1, anchorless: ['packages/demo/README.md'], docs: 0 },
250+
narrowed: false,
223251
expect: (headline) => {
224252
check('anchored-and-anchorless', 'the derived anchors are still reported', true, headline.startsWith('**1** anchor(s) derived'));
225253
check('anchored-and-anchorless', 'and the unanchored file is reported beside them', true, headline.includes('NOT COVERED by this run'));
@@ -232,6 +260,7 @@ const CASES = [
232260
base: { 'content/docs/guide.mdx': NAMES_THE_ANCHOR },
233261
change: { 'packages/demo/src/widget.ts': ADDS_ANCHOR },
234262
want: { anchors: 1, anchorless: [], docs: 1 },
263+
narrowed: false,
235264
expect: (headline, body) => {
236265
check('docs-listed', 'the work-list headline is unchanged', true,
237266
headline === 'This PR changes **1** package(s): `@objectstack/demo`, touching **1** documentable anchor(s).');
@@ -244,6 +273,7 @@ const CASES = [
244273
base: { 'content/docs/guide.mdx': NAMES_NOTHING },
245274
change: { 'content/docs/guide.mdx': `${NAMES_NOTHING}\nOne more line.\n` },
246275
want: { anchors: 0, anchorless: [], docs: 0 },
276+
narrowed: false,
247277
expect: (headline) => {
248278
check('no-package-change', 'the original headline is preserved byte-for-byte', true,
249279
headline === 'Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from **0** changed package(s)), so **this run has no opinion** about the docs.');
@@ -275,6 +305,11 @@ try {
275305
check(c.id, 'the comment keeps its heading', '### 📓 Docs Drift Check', lines[1]);
276306
const headline = lines[2];
277307
if (PRINT) console.log(`\n── ${c.id} — ${c.what}\n${headline}\n`);
308+
// Non-vacuity for the #11356 verdict, asserted in the LOOP rather than per case, so a
309+
// sixth case cannot be added without declaring which side of the line it sits on.
310+
if (typeof c.narrowed !== 'boolean') throw new Error(`case ${c.id} does not declare \`narrowed\``);
311+
check(c.id, `the #11356 narrowed verdict ${c.narrowed ? 'renders here' : 'does NOT render here'}`,
312+
c.narrowed, headline.includes(NARROWED_PHRASE));
278313
c.expect(headline, body);
279314
}
280315
} finally {

0 commit comments

Comments
 (0)