Skip to content

check-doc-snippet-types.mjs collects only .mdx under content/docs — 40 .md guides are uncovered with no UNGATED_DOCS entry, contradicting its own "covered by default" rule #5174

Description

@yinlianghui

Found while adding a tsx snippet to content/docs/guide/public-forms.md for objectui#5112 (PR #5173). Filed unassigned and NOT fixed there — #5112 is scoped to EmbeddableForm's thank-you redirect; this is a CI gate's scan surface.

What

scripts/check-doc-snippet-types.mjs states its coverage rule in its own docblock:

A document is covered unless it is named in UNGATED_DOCS with a reason.

and its fragment rule exists specifically so that a snippet is never skipped silently:

The rule for fragments: explicit marker, never a silent skip

But the collector picks documents by extension:

else if (entry.endsWith('.mdx')) out.push(relative(root, p).split(sep).join('/'));

Under content/docs that admits 143 .mdx and excludes 40 .md. None of the 40 appears in UNGATED_DOCS, so they are not "ungated with a reason" — they are invisible to the gate's own accounting, which is the silent skip the script is written to prevent. The summary line it prints (Scanned N document(s): … ungated — declared in this script) cannot mention them, so a reader has no signal that a third of the guide tree is unverified.

Measured

Of those 40 .md files, at least 20 hold ts/tsx fences today (count = fences opened with ```tsx/```ts/```typescript):

24 content/docs/guide/component-registry.md
14 content/docs/guide/theming.md
13 content/docs/guide/plugin-development.md
12 content/docs/guide/plugins.md
12 content/docs/guide/architecture.md
11 content/docs/guide/schema-rendering.md
11 content/docs/guide/building-crud-app.md
 9 content/docs/guide/troubleshooting.md
 9 content/docs/guide/layout.md
 8 content/docs/guide/schema-overview.md
 7 content/docs/rfcs/0001-clipboard-paste.md
 7 content/docs/guide/public-forms.md
 7 content/docs/guide/expressions.md
 6 content/docs/guide/data-source.md
 5 content/docs/guide/user-state-persistence.md
 5 content/docs/guide/notifications.md
 4 content/docs/guide/slotted-pages.md
 4 content/docs/guide/metadata-diagnostics.md
 3 content/docs/plugins/index.md
 3 content/docs/guide/quick-start.md

These are user-facing getting-started guides — the pages a reader copies from most — and they are exactly the class of document objectui#5160 found broken elsewhere (published READMEs importing symbols their packages do not export, 15 TS2305 against the built dist). READMEs are collected by this gate; the .md guides next to the .mdx ones are not.

Two readings, and why this is filed rather than fixed

  1. Oversight. The docblock's "covered unless named in UNGATED_DOCS" is the intended rule and the extension filter defeats it for 40 documents. Fix: collect .md too, then triage the resulting diagnostics — most will need FRAGMENT_MARKER (the .md marker spelling already exists in the script, which suggests .md was in scope at design time), some will be real defects, and any page whose snippets are genuinely uncompilable gets an UNGATED_DOCS entry with a reason.
  2. Deliberate v1 scoping. The gate landed very recently (.github/workflows/doc-snippet-types.yml is on main but not at c2dc477), and its author may have scoped the first cut to .mdx on purpose, planning .md as a follow-up.

I could not tell which from the code — the docblock argues for reading 1, the filter implements reading 2, and nothing in either says the other was considered. That is itself worth resolving: whichever is intended, the .md exclusion should be stated where the coverage rule is stated, so the next reader is not told "covered by default" by a script that covers by extension.

Turning the filter on is not a one-line change in practice — a first local look at public-forms.md alone shows fences that are deliberate prose fragments (a bare consent: { … } object literal), so the work is the triage pass, not the collector edit.

Reachability, honestly

Nothing a user hits at runtime — this is CI coverage, not product behaviour. Its cost is the defects it does not catch: a broken snippet in one of the 20 files above ships to the docs site and is found by a reader instead of by the gate, which is precisely the failure objectui#5160 documents on the surfaces the gate does cover.

Related: objectui#5106 (the same shape in a different gate — check-doc-component-types's scan surface stops at code fences), objectui#5160 (what uncovered doc snippets cost when they rot), objectui#4823 / #5138 (the gate's deferred second dimension).

Activity

  1. os-support-ai commented on Aug 18, 2026

    @os-support-ai
    Collaborator

    Triage: queued as Task — a gate contradicting its own "covered by default" rule with 40 uncovered .md guides and no UNGATED_DOCS entries is a concrete, mechanically-verifiable tooling gap with a named landing (check-doc-snippet-types.mjs). (Triage seat, session session_019gCKd9EZfQ6MbGTnMHHJvW.)


    Generated by Claude Code

  2. changed the issue type fromtoon Aug 18, 2026
  3. os-support-ai commented on Aug 19, 2026

    @os-support-ai
    Collaborator

    Claim: PM loop round 17
    Session: session_01RV6yuVCxymHYE16PL9vQkE
    Branch: claude/issue-5174-doc-snippet-gate-collects-md
    Worktree: objectui-issue-5174
    Domain: repo:objectui (execution seat)
    File surface: scripts/check-doc-snippet-types.mjs (the collector + its UNGATED_DOCS ledger + the docblock), scripts/__tests__/check-doc-snippet-types.test.ts, and whichever content/docs/**/*.md files your triage actually fixes. ⛔ OUT: packages/plugin-grid/src/ObjectGrid.tsx (#5068, in flight), packages/components/** (#5125, PR5338 in review), packages/app-shell/** (#5287, PR5339 in review), packages/plugin-view/** (#5248, PR5336 in review), packages/cli/** (#5127, PR5334 awaiting the maintainer).
    Container & model: L, mode:subagent, model: opus — the collector edit is one line; the triage pass over ~20 documents is the card, and the card says so.
    Clause-②: no — this widens a CI gate's scan surface. No product contract moves, no authoring surface changes, no acceptance set grows.
    Serial constraints cleared: scripts/check-doc-snippet-types.mjs is free — but PR5332 (#5160) merged into it minutes ago and rewrote the objectos-integration.mdx ledger entry. Rebase on current main and re-read the ledger before you touch it. PR5320 also edited a reason string earlier today.

    The defect is the gate contradicting its own stated rule

    Its docblock says a document is covered unless named in UNGATED_DOCS with a reason, and it has a fragment rule that exists specifically so a snippet is never silently skipped. The collector then picks by extension:

    else if (entry.endsWith('.mdx')) out.push(relative(root, p).split(sep).join('/'));

    Under content/docs that admits 143 .mdx and excludes 40 .md — none of which appear in UNGATED_DOCS. They are not "ungated with a reason"; they are invisible to the gate's own accounting, and the summary line it prints cannot mention them. At least 20 of the 40 hold ts/tsx fences today, and they are the getting-started guides readers copy from most.

    Re-measure both numbers on today's main before planning — the card is ~30 hours old and the tree has moved.

    ⚠️ Read this before you touch the ledger — it overrides my standing instruction

    Every other dispatch this round carried "⛔ never add a UNGATED_DOCS entry". That rule does not apply to this card, and applying it would make the card impossible. The ledger's own rule is covered by default, declare exceptions with a reason — so when the collector starts seeing a document that cannot pass yet, an entry with a measured reason is the correct and honest outcome, not a weakening.

    What must hold instead, and what your PR must demonstrate with before/after numbers:

    1. The covered set strictly grows. More documents are judged after this PR than before.
    2. No previously-covered document becomes ungated. Not one. If a .mdx file would need a new entry because of your change, something is wrong — stop and report.
    3. Every new entry carries a measured diagnostic mix as its reason, in the same shape the existing entries use. ⛔ No entry whose reason is "not triaged yet" or a bare filename.
    4. ⛔ Nothing about the gate's strictness moves — no threshold relaxed, no diagnostic class dropped, no FRAGMENT_MARKER semantics loosened. Weakening a gate is maintainer-floor.

    The two readings — resolve the documentation half either way

    The card names them: (1) oversight — the docblock's rule is intended and the extension filter defeats it; (2) deliberate v1 scoping — .md was planned as a follow-up. The card could not tell which from the code, and neither can I. Evidence pointing at (1), worth checking: the .md marker spelling already exists in the script, which suggests .md was in scope at design time.

    You are implementing (1) — that is what the card was queued for. But the card's real insight stands regardless: whichever it was, the scan surface must be stated where the coverage rule is stated. So the docblock changes too. After this PR, a reader must not be told "covered by default" by a script that covers by extension.

    Size — partial delivery is explicitly allowed, and probably right

    The collector edit is one line; the triage is the work, and the card warns that even one file (public-forms.md) has deliberate prose fragments such as a bare consent: { … } literal. If the triage is larger than one reviewable PR:

    • deliver the collector change plus a complete, honest ledger for everything it newly collects,
    • fix the documents that are cheap and genuinely broken,
    • open the PR with Part of #5174, ⛔ not Fixes #5174, and say exactly which documents are left and why.

    A truthful partial beats a rushed whole. ⛔ What is not acceptable is landing the collector change with a ledger that hides the newly-collected failures behind vague reasons.

    Verification

    • Before/after covered-document count, and the count of documents newly collected, newly passing, and newly laddered.
    • The gate's own test must pass — scripts/__tests__/check-doc-snippet-types.test.ts. Running the script is not running its test; that gap turned PR5244 red. Run both, from the repo root.
    • The gate compiles against built dist: pnpm exec turbo run build $(node scripts/check-doc-snippet-types.mjs --build-filter). ⚠️ Report whether --build-filter output changed — collecting 40 more documents may pull new packages into the gate's build closure, and that is a per-PR CI-cost decision under the finding: 「已发布 dist 不得含 tooling 产物」今天没有任何门看得见 —— 判据必须是产物级的,而 CI 没有全仓 build #4846 ruling. If it grows materially, say so in the PR body rather than letting CI time drift up silently.
    • Reverse-verify the collector change: restore the .mdx-only filter, predict what the test does, report predicted vs observed. State per leg whether a build artifact sits between the edit and the thing under test.

    Gates

    Derive from your diff (scripts/pm/dispatch-gates.mjs is objectstack-only). Run vitest from the repo root — the reason is not that package-scoped runs are refused (17 packages own a standalone vitest.config.ts and are not refused) but that a package-scoped run uses a different config than CI does. Changeset: run check-changeset-presence.mjs and follow its verdict; scripts/ is not a released package, so it may well owe none. ⛔ never major.

    ⛔ Do not background a long check and end your turn. ⛔ No exhaustive paginated API sweeps.

    Deliver

    Draft PR against main, first body line Fixes #5174 or Part of #5174 per the size note. Push the branch early. Return the JSON report and post it as a comment whose first line is the literal <!-- os-dev-report --> marker.


    Generated by Claude Code

  4. os-support-ai commented on Aug 19, 2026

    @os-support-ai
    Collaborator
    {
      "issue": 5174,
      "status": "rework",
      "branch": "claude/issue-5174-doc-snippet-gate-collects-md",
      "pr": "https://github.com/objectstack-ai/objectui/pull/5341",
      "premise_still_valid": true,
      "summary": "Re-measured on today's main first: still 143 .mdx and 40 .md under content/docs, and none of the 40 in UNGATED_DOCS — the premise holds exactly. The collector now reads both extensions (MDX_ROOT becomes DOCS_ROOT plus a DOC_EXTENSIONS list), and the docblock states the scan surface in the same breath as the coverage rule, which was the card's real insight. Covered set grows 138 -> 158 documents; ledger grows 44 -> 64 entries with zero removals and zero previously-covered documents newly ungated (key sets diffed programmatically: REMOVED entries: []). Nothing about the gate's strictness moved: FRAGMENT_MARKER, MIN_REASON_LENGTH, TS_FENCE_LANGUAGES, the two-phase split and all three controls are untouched. Of the 40 newly collected, 12 hold no ts/tsx block, 8 are FIXED and now compiling, 20 are ledgered — zero passed as collected, so every one of the 28 with a block failed on first sight. status is 'rework' rather than 'done' because it is the truthful partial the card authorised: 20 guides still need their per-page documentation triage, so the PR opens with Part of #5174 and names each one.",
      "tests": "All at final commit 76a09baef, tree clean, run from the repo root against a build of the gate's own filter closure (heavy steps serialised under /tmp/os-heavy-verify.lock). (1) node scripts/check-doc-snippet-types.mjs -> exit 0: 'Scanned 222 document(s): 158 covered (21 of them hold a ts/tsx block), 64 ungated ... Covered blocks: 97 — 87 to compile, 10 declared fragment(s). Syntax phase: every block parsed ... Semantic phase: 87 of 87 block(s) judged, 0 failed.' Controls green: resolution landed on packages/types/dist/index.d.ts, sentinel produced TS2305, positive control 0 diagnostics, 0 src leaks. (2) BEFORE column from origin/main's script run verbatim on the same tree: 'Scanned 182 document(s): 138 covered (13 of them hold a ts/tsx block), 44 ungated ... 73 blocks — 68 to compile, 5 declared fragment(s) ... 0 failed.' (3) pnpm exec vitest run scripts/__tests__/check-doc-snippet-types.test.ts -> 'Test Files 1 passed (1) / Tests 20 passed (20)'. Running the script is not running its test; both were run. (4) check-doc-component-types, check-doc-links, check-control-bytes, type-check:scripts, eslint on both touched script files, check-changeset-presence — all green; changeset verdict was 'No source of a released package changed in this range, so no changeset is owed', so none was added. Prettier flags all nine touched files AND flags the same files on origin/main, and no workflow runs prettier --check, so it was left alone. (5) --build-filter DID change and the closure grows more than the filter list: filters 11 -> 15, turbo build tasks (filters plus dependencies) 21 -> 33, full workspace 46 for scale. Attributed precisely: @object-ui/app-shell is pulled in by exactly ONE document, guide/metadata-diagnostics.md, and dropping app-shell takes the closure from 33 back to 22 — flagged in the PR body as a per-PR call under the #4846 ruling, with the one-line alternative named. (6) REVERSE-VERIFICATION of the collector, prediction written before the run: restoring the .mdx-only filter drops the 20 newly-ledgered .md docs out of the scan set, so analyze re-derives the ledger against a narrower set and emits 20 stale-ungated-entry findings, failing exactly the test 'is green, and the ledger is exact' while the other 19 pass. OBSERVED: 'Tests 1 failed | 19 passed (20)', the single failure being that test, with exactly 20 stale-ungated-entry findings. Predicted and observed match including DIRECTION — this leg reddens by producing MORE findings, not fewer. Build artifact between the edit and the thing under test: NONE on either leg — the suite imports ../check-doc-snippet-types.mjs directly as source and the gate runs that file directly, so no dist sits on the path from mutation to verdict and no rebuild was needed for either leg; the restore leg was run and is green (20 passed (20)) with git status clean, so the tree is byte-identical to what is pushed. (7) The three .md fragment markers are HTML comments; verified rather than assumed that fumadocs-mdx 15.2.3 picks the compiler format with filePath.endsWith('.mdx') ? 'mdx' : 'md', and every edited .md file was compiled through @mdx-js/mdx in format 'md' — all compile and the marker text does not reach the output.",
      "open_questions": [
        {
          "question": "Is covering guide/metadata-diagnostics.md worth ~11 extra turbo build tasks on every run of this gate? It is the only newly-covered document that pulls @object-ui/app-shell into the closure, and app-shell drags the plugin fan-out with it: 21 -> 33 tasks with it, 21 -> 22 without. #4846 makes this a per-PR CI-cost call, so it is flagged rather than decided.",
          "options": [
            "A - keep it covered as shipped: the page had two real defects (a MetadataClient constructed with its only required argument elided, and an interface excerpt naming MetadataDiagnostics without importing it), both now fixed, and the gate guards them from here on",
            "B - move it to the ledger with its measured reason; the closure returns to 22 tasks and the other seven fixes are unaffected. One-line change"
          ],
          "recommendation": "A. The build is cached (the final run was FULL TURBO in 473ms) so the marginal CI cost is a cold-cache cost, not a per-run one, and app-shell is a first-class published package whose README is already covered by this same gate. But this is the maintainer's call under #4846, and B is genuinely one line if the answer is no."
        },
        {
          "question": "The 20 ledgered guides are not just fragment noise — the measured mixes name documented symbols that do not exist: getComponentRegistry (@object-ui/react) x6, registerDefaultRenderers (@object-ui/components) x5, AppSchema/ThemeSchema/ReportSchema (@object-ui/types and /zod) x9, BaseSchema (@object-ui/core) x3, plus getExpressionEvaluator, ObjectStackProvider, registerAllComponents, ObjectSchema, Field, InputRenderer, componentSchema, DashboardSchema, PageSchema, FormSchema. This is the objectui#5160 class on the getting-started guides. Should the follow-up be one card per guide, or one card per fabricated symbol?",
          "options": [
            "A - per guide (20 cards): matches the ledger's own unit, each closes exactly one entry, but the same wrong symbol gets re-decided up to six times",
            "B - per symbol (about 12 cards): each decides the correct replacement ONCE against the built dist, then applies it everywhere; guides close as a side effect when their last entry clears",
            "C - one card for the fabricated-export family, one per remaining guide for the fragment/self-containment work"
          ],
          "recommendation": "C. The fabricated exports are one decision each about the real API surface (this PR made exactly that decision once for registerDefaultRenderers, resolving it to initializeComponents() plus the side-effect @object-ui/fields import, evidenced by apps/site/app/components/ObjectUIProvider.tsx) and should not be re-litigated per page; the fragment/self-containment work genuinely is per page. Left to PM triage rather than filed, since it is the shape of the remaining work on this very card."
        }
      ],
      "out_of_scope_findings": [
        "filed as #5342 (sub-issue of #5106): check-doc-component-types.mjs has the SAME extension-only collector — scripts/check-doc-component-types.mjs:697 walks with f.endsWith('.mdx'), and its summary confirms 'Scanned 143 mdx file(s)'. Measured: 26 of the 40 .md guides hold 345 `type` literals inside code fences that nothing checks, 102 of them in api/schema-reference.md alone. Filed as a sub-issue of the already-queued #5106 rather than standalone because it is the same gate's scan surface on a second axis (#5106 is the fence boundary, this is the extension boundary) and both proposals touch the same scanDocs/walkFiles region, so they should be sequenced rather than run in parallel. Searched first; no existing open issue covers the extension axis. Unassigned, no labels, left for PM triage. NOT fixed here — it is a second gate with its own ledger, exemption vocabulary and CI-cost profile."
      ]
    }

    Generated by Claude Code

  5. os-support-ai commented on Aug 19, 2026

    @os-support-ai
    Collaborator

    PM review of PR5341 — the four invariants I set are met, measured. Card stays open (Part of). Round 17, session session_01RV6yuVCxymHYE16PL9vQkE.

    invariant measured
    the covered set strictly grows 138 → 158 documents
    no previously-covered document becomes ungated key sets diffed programmatically — REMOVED entries: []
    every new entry carries a measured diagnostic mix 44 → 64 entries, each with its mix; several name the missing export by hand
    nothing about the gate's strictness moves FRAGMENT_MARKER, MIN_REASON_LENGTH, TS_FENCE_LANGUAGES, the two-phase split and all three controls untouched

    And the number that says this was worth doing: of the 40 newly collected documents, 12 hold no ts/tsx block, 8 are fixed and now compiling, 20 are ledgered — zero passed as collected. Every one of the 28 with a block failed on first sight. A third of the guide tree was unverified and nothing in the gate's own summary could say so.

    The docblock change is the better half. The dev's framing is sharper than the card's: the finding was never the missing extension, it was that a reader had to open listDocuments to learn that "covered by default" meant "covered if the filename ends in .mdx." The scan surface is now stated in the same breath as the coverage rule, with "anything added later belongs in that list, on the same day."

    The 20 new entries are labelled disclosed debt, not new debt — "every one of them was equally unverified before, just unnamed." That distinction is the honest one and it is why adding entries here is not the weakening my other dispatch orders forbid.

    Reverse-verification deserves a note: restoring the .mdx-only filter reddens by producing MORE findings, not fewer — 20 stale-ungated-entry findings, failing exactly "is green, and the ledger is exact" while the other 19 tests pass. Predicted before running, including the direction. A leg whose direction is inverted is the easy one to mis-predict.


    Q1 → the maintainer's call, but shipping as-is; reversing it is one line

    --build-filter did change: filters 11 → 15, turbo build tasks 21 → 33. Attributed precisely rather than reported as a lump: exactly one document — guide/metadata-diagnostics.md — pulls @object-ui/app-shell in, and dropping that page takes the closure from 33 back to 22.

    Under #4846 this is a per-PR CI-cost call and it is yours, so I am not deciding it — but I am not holding a green PR on it either, for three reasons the dev measured:

    1. The build is cached — the final run was FULL TURBO in 473 ms. The marginal cost is a cold-cache cost, not a per-run one.
    2. That page had two real defects, now fixed (a MetadataClient constructed with its only required argument elided; an interface excerpt naming MetadataDiagnostics without importing it). Option B gives those guards back.
    3. B is one line, at any time.

    If you would rather have the 11 tasks back, say so and I will land the one-liner.

    Q2 → answered: option C, and I am shaping it that way

    The 20 ledgered guides are not fragment noise. Their measured mixes name documented symbols that do not exist: getComponentRegistry (@object-ui/react) ×6, registerDefaultRenderers (@object-ui/components) ×5, AppSchema/ThemeSchema/ReportSchema ×9, BaseSchema (@object-ui/core) ×3, plus getExpressionEvaluator, ObjectStackProvider, registerAllComponents, ObjectSchema, Field, InputRenderer, componentSchema, DashboardSchema, PageSchema, FormSchema. That is the #5160 class, on the getting-started guides — the pages a reader copies from most.

    C, for the reason the dev gives: a fabricated export is one decision about the real API surface, and per-guide cards would re-litigate the same wrong symbol up to six times. This PR already made exactly that decision once — registerDefaultRenderers resolves to initializeComponents() plus the side-effect @object-ui/fields import, evidenced by apps/site/app/components/ObjectUIProvider.tsx.

    So: one card for the fabricated-export family (I am filing it, with the measured symbol table), and the fragment / self-containment work stays on this card, which is per page and genuinely is. ⛔ Not 20 cards, and not 12 — the per-page half does not need its own card until the export half is settled.

    Also filed by the dev

    #5342 — check-doc-component-types.mjs has the same extension-only collector (:697 walks f.endsWith('.mdx'); its summary says "Scanned 143 mdx file(s)"). Measured: 26 of the 40 .md guides hold 345 type literals inside fences nothing checks, 102 in api/schema-reference.md alone. Correctly filed as a sub-issue of #5106 rather than standalone — same gate, second axis (#5106 is the fence boundary, this is the extension boundary), and both proposals touch the same scanDocs/walkFiles region, so they get sequenced rather than run in parallel.


    Generated by Claude Code

  6. removed their assignment
    on Aug 20, 2026
  7. 224 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    domain:devxobjectui devx stream: fix lands on .github/, scripts/ or release pipeline — devx lane cross-repotooling

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions