ci: pin the generated sitemap's locale composition against the content tree

[hotlong]

Fixes #175

Nothing in this repo could tell you that a generated artifact's locale composition was wrong. This adds the gate that can.

Why

The same missing-language-argument defect has now been found and repaired in four generated artifacts — sitemap.ts (#169), llms.txt (#170), llms-full.txt (#177), the OG card (#185) — and every one of them was green on every gate for its entire lifetime. On the pre-#169 tree sitemap.xml advertised 3892 entries over 574 distinct URLs, 3318 of them exact duplicates, against 335 real MDX source files. Type-check passed, build passed, test passed. On #177 the same shape meant 4.6 MB of seven-locale text served under a robots.txt that had just been opened to crawlers.

The common root cause is an API that takes an optional language argument and quietly returns every locale's page set when you omit it. Nothing about that is a type error and nothing about it is a build error. It is only visible as a count.

What the gate asserts

.github/scripts/check-locale-surface.mjs derives the expected URL set — every run — from the two authorities the repo already has, then compares it to the built sitemap.xml:

• content/docs/** for the pages and which locales really have each one (a translation is a locale-suffixed sibling; its absence means the page falls back to English, and a fallback URL must not be advertised as a translation);
• apps/docs/lib/i18n.ts for the locale list and default locale — AGENTS.md names it the authority, check-translation-output.mjs already reads it the same way, and turbo.json already names it an input to the test task.

Nothing is hardcoded that moves on a content PR. A gate that pins 346 by hand needs hand-editing on every translation and gets deleted within a month — and worse, the edit that updates the number is the same edit that would hide a regression.

Both directions fail, per the card:

rule	what it catches
unexpected-url	advertised, no source file — an untranslated page advertised as a translation
missing-url	source file, not advertised — a shipped translation invisible to crawlers
duplicate-url	the same URL emitted more than once
artifact-missing	no build output to read — a failure, never a skip
artifact-empty	artifact found but the reader matched nothing (format drift)
translation-orphan	a locale file with no English source, which would inflate the oracle to agree with an artifact that is also wrong

Duplicates are counted rather than collapsed on read, because de-duplicating would have hidden all 3318 of the pre-#169 tree's behind a correct distinct count.

Composition, not formatting — as adjudicated on the card. The gate reads the loc values and compares URL sets. #178's relative-to-absolute llms.txt link change does not touch it, and #173 changes where sitemap.ts reads the privacy/terms locale set from, not what it is.

Shape and wiring

It reads the build output because importing sitemap.ts pulls in the whole MDX collection and the @/ alias — the constraint the card priced, inherited rather than rediscovered. So:

• ci.yml invokes the gate as its own step after pnpm turbo run build, with shell: bash so the tee pipe cannot swallow a non-zero exit (the same trap the node-floor job documents).
• Deliberately not a turbo task. Turbo would hash it against the ci-scripts package's inputs, which do not include apps/docs/.next/ — and a locale gate replaying a cached green is the exact failure this card exists to end.
• run-self-tests.mjs gains it in SELF_TESTED, so pnpm turbo run test keeps running its --self-test. That mode needs no build: it drives the same collect/evaluate pipeline over fixture trees in a temp directory, so the readers are exercised too.
• turbo.json needed no change. Its test inputs already name $TURBO_ROOT$/.github/scripts/**, so the self-test re-runs when this script changes. Adding content/docs/** there would declare an input the self-test does not read.
• Root package.json gains check:locale-surface for running it locally after a build.

ARTIFACTS is the whole extension point. #184 adds llms.txt and llms-full.txt there with their own reader and expectation; the harness needs no other change. Per the card's scope note, this PR ships only the sitemap and #184 stays open.

Verification

The gate shown red on a real defect, not just green on a good tree.

Leg 1 — apps/docs/app/sitemap.ts checked out verbatim at ee74379 (blob 079b27b9, proven on disk by git hash-object). It does not compile against today's tree:

app/sitemap.ts(25,23): error TS2554: Expected 2 arguments, but got 1.
Exit status 2

That is #169's own signature change working as designed — languageAlternates(path, locales) made the argument required precisely so a call site could not silently get all seven again. Restoring lib/seo.ts to ee74379 too was not an option: translatedLocales did not exist there, and the OG route and the docs page both import it now.

Leg 2 — the same defect reconstructed against today's signature (source.getPages() with no argument, languageAlternates(path, i18n.languages), iterate i18n.languages), which is behaviourally identical to ee74379. Mutation proven on disk by blob hash and by grep of the injected and removed text, then rebuilt:

loc count: 3892
distinct : 574

— the issue body's numbers exactly (556 logical paths x 7 locales = 3892; 79 docs + 3 static, x 7 = 574). The gate on that tree, exit 1:

| artifact    | advertised | distinct | expected | unexpected | missing | duplicated |
| sitemap.xml |       3892 |      574 |      346 |        228 |       0 |        553 |

X 781 locale-surface finding(s)

unexpected-url - 228
  - sitemap.xml: https://docs.objectos.ai/de/docs/architecture is advertised but the
    content tree has no source file for it - an untranslated page is being advertised
    as a translation

duplicate-url - 553
  - sitemap.xml: https://docs.objectos.ai/docs/architecture advertised 7 times

Restore proven by blob hash back to 197084a2 and by git diff HEAD returning 0 lines, then rebuilt — loc count: 346, gate exit 0.

Green run, at final commit fe6abaf:

Oracle: 79 logical docs page(s) over 7 locale(s) = 335 docs entries

| locale  | docs entries |     | artifact    | advertised | distinct | expected | unexpected | missing | duplicated |
| en      |           79 |     | sitemap.xml |        346 |      346 |      346 |          0 |       0 |          0 |
| zh-Hans |           62 |
| ja      |           39 |     OK every advertised URL has a source file,
| de      |           38 |        and every source file is advertised
| es      |           39 |
| fr      |           39 |
| ko      |           39 |

Per-locale counts match the card's measured values exactly, derived rather than declared.

pnpm turbo run type-check --continue and pnpm turbo run test at fe6abaf: both 1 successful, 1 total; the test task reports 3 self-test(s) passed. The new self-test: 7 cases over 6 rules, every rule demonstrated able to fail, plus a per-locale tally assertion and three cases proving an unreadable i18n.ts is rejected rather than guessed around.

Scope

Four files, all within the declared surface. No apps/docs source file — this card asserts behaviour, it does not change it. The working tree is clean and the sitemap source is byte-identical to origin/main.

---

Generated by Claude Code