From 6014b90e255464922d606d5e8daa7575ce0390b7 Mon Sep 17 00:00:00 2001 From: Jesus Armando Anaya Date: Sun, 2 Aug 2026 07:52:01 -0700 Subject: [PATCH] =?UTF-8?q?docs:=20close=20out=20the=20documentation=20wor?= =?UTF-8?q?kstream=20=E2=80=94=20an=20authoring=20guide,=20link=20hardenin?= =?UTF-8?q?g,=20and=20the=20npm=20policy=20(p-docs-s4)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit docs/contributing/documentation.md answers "I want to document a thing — where does it go, and what may I write?" without reading a generator. The syntax budget is stated from what the in-app renderer actually does rather than from convention: no footnotes, no front matter, no admonitions, no code-fence transforms, and no heading anchors — so `page.md#section` lands at the top of the page in the app while working correctly on the site. That is a constraint to design for, not to route around: link to a page, and if you need to point at one section of a long reference page, the page wants splitting. The inconsistency is settled in favour of backticks. QTextBrowser renders unstyled, so in the app it looks exactly like the surrounding prose and the distinction it was reaching for is lost; backticks are pure CommonMark and render identically in the app, on the site and on GitHub. Applying it to the seven pages that use stays a separate change, as scoped. check:links verifies every internal link and image in a BUILT tree — each version directory, dev/, and latest/ — each against the base it was built with. That is a different check from the adapter's, which reads source and cannot see a reference that only breaks once the pages are emitted; deleting a page or an image from a build is caught here and nowhere else. Outbound links get a non-blocking report instead: link rot in a third party's URL is not a reason a contributor's merge cannot happen, and a gate that fails for reasons nobody here can fix is one people learn to ignore. Each build now stamps the base it used into its output. The checker runs in its own shell — in CI and in a contributor's terminal — where the build's environment is long gone, and a checker that assumed the wrong base reported every link in the build as broken. This was found by following the new guide literally, as its acceptance criterion asks, and it would have failed CI. The stamp also makes latest/ self-describing, which matters because latest/ is a byte copy of the highest version and therefore carries THAT version's base. Its links all resolve; the consequence, now written down rather than left to be rediscovered, is that a reader who opens /latest/ is moved to the pinned version's URL on their first click. It is an entry point, not a browsable mirror. npm dependency policy was absent rather than thin: 374 installed packages, a committed lockfile and a CI licence gate, none of it written down. Both the policy and THIRD_PARTY_LICENSES.md now carry it — scoped to docs-site/, npm ci, the licence gate, a monthly human cadence, and an explicit rule that no npm automation may open pull requests or fail a job for the C++ side. The deliberately stubbed sharp is recorded as the standing example of the LGPL rule biting. Drift fixed: docs/README.md no longer says the tree is arranged so a static-site generator "could be adopted later", and the roadmap no longer describes an _order manifest for guides that was sketched during planning and never built. Closes #348 --- .github/workflows/docs-publish.yml | 7 + .github/workflows/docs-site.yml | 18 ++ CHANGELOG.md | 34 +++ THIRD_PARTY_LICENSES.md | 28 +++ docs-site/README.md | 8 + docs-site/package.json | 4 +- docs-site/scripts/build-local.mjs | 10 + docs-site/scripts/build-web.mjs | 13 ++ docs-site/scripts/check-links.mjs | 221 ++++++++++++++++++ docs-site/scripts/report-external-links.mjs | 125 ++++++++++ docs-site/test/check-links.test.mjs | 174 ++++++++++++++ docs/README.md | 14 +- docs/contributing/documentation.md | 241 ++++++++++++++++++++ docs/roadmap/README.md | 5 +- docs/standards/dependencies.md | 54 +++++ 15 files changed, 949 insertions(+), 7 deletions(-) create mode 100644 docs-site/scripts/check-links.mjs create mode 100644 docs-site/scripts/report-external-links.mjs create mode 100644 docs-site/test/check-links.test.mjs create mode 100644 docs/contributing/documentation.md diff --git a/.github/workflows/docs-publish.yml b/.github/workflows/docs-publish.yml index 916f7431..3332166f 100644 --- a/.github/workflows/docs-publish.yml +++ b/.github/workflows/docs-publish.yml @@ -159,6 +159,13 @@ jobs: echo '```' } >> "$GITHUB_STEP_SUMMARY" + # The whole assembled tree, not just the version just built: dev/, every + # version directory still being served, and latest/. A version published + # months ago is still a page a reader can open, and nothing else re-checks + # it. + - name: Internal links across the assembled tree + run: node docs-site/scripts/check-links.mjs published${{ steps.plan.outputs.scratch && '/_dryrun' || '' }} + - name: Verify the dry run stayed inside its scratch prefix if: github.event_name == 'workflow_dispatch' working-directory: published diff --git a/.github/workflows/docs-site.yml b/.github/workflows/docs-site.yml index d95be814..878db713 100644 --- a/.github/workflows/docs-site.yml +++ b/.github/workflows/docs-site.yml @@ -60,6 +60,19 @@ jobs: - name: Build (under a version segment, as published) run: npm run build:web -- --base=/dev/ + # Every internal link and image in the built output, checked against the + # base it was built with. The adapter's own check runs on SOURCE, so it + # cannot see a reference that only breaks once the pages are emitted. + - name: Internal links + run: npm run check:links + + # Report only, on purpose: link rot in a third party's URL is not a reason + # a contributor's merge cannot happen. `continue-on-error` as well as the + # script's own exit 0, so a crash in the reporter cannot block either. + - name: Outbound link report (never blocks) + continue-on-error: true + run: npm run report:external + - uses: actions/upload-artifact@v4 with: name: docs-site @@ -92,6 +105,11 @@ jobs: - name: Build the local reader run: npm run build:local + # The offline reader's links are relative rather than base-prefixed, so + # this is a genuinely different resolution path from the web build's. + - name: Internal links + run: npm run check:links + - uses: actions/upload-artifact@v4 with: name: docs-manual-local diff --git a/CHANGELOG.md b/CHANGELOG.md index 3810cb86..17e6ce90 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,40 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 Current version on `main`: **0.0.1**. ### Added +- **The documentation workstream is closed out: an authoring guide, link + hardening, and dependency hygiene** + ([#348](https://github.com/Robomous/RoadMaker/issues/348), docs-s4 — + [ADR-0009](docs/decisions/0009-documentation-site-tiered-docs.md)). + [Writing user documentation](docs/contributing/documentation.md) answers + "I want to document a thing — where does it go, and what may I write?" without + reading a generator: the two tiers and how to choose, the exact syntax budget + the in-app renderer supports, the bridge convention, image placement, and the + `index.md` manifest rule. + + The syntax budget is stated from what the renderer actually does rather than + from convention — no footnotes, no front matter, no admonitions, no code-fence + transforms, and **no heading anchors**, so `page.md#section` lands at the top + of the page in the app while working on the site. The `` inconsistency is + settled in favour of backticks, with the reasoning; applying it to the seven + pages that use `` stays a separate change. + + `check:links` verifies every internal link and image in a **built** tree — + each version directory, `dev/`, and `latest/`, each against the base it was + built with. That is a different check from the adapter's: the adapter reads + source and cannot see a reference that only breaks once pages are emitted. + Outbound links get a **non-blocking report** instead; link rot in a third + party's URL is not a reason a contributor's merge cannot happen. + + npm dependency policy, absent until now, is written down: scoped to + `docs-site/`, `npm ci` against a committed lockfile, the licence gate, a + documented **monthly** cadence, and an explicit rule that no npm automation may + open pull requests or fail a job for the C++ side. The site's build tools and + the deliberately stubbed `sharp` are recorded in `THIRD_PARTY_LICENSES.md`. + + Drift fixed: `docs/README.md` no longer says the tree is arranged so a + static-site generator "could be adopted later", and the roadmap no longer + describes an `_order` manifest for guides that was sketched during planning + and never built. - **The documentation site publishes itself, versioned** ([#347](https://github.com/Robomous/RoadMaker/issues/347), docs-s3 — [ADR-0009](docs/decisions/0009-documentation-site-tiered-docs.md)). GitHub diff --git a/THIRD_PARTY_LICENSES.md b/THIRD_PARTY_LICENSES.md index 06fd0a6a..f6b52246 100644 --- a/THIRD_PARTY_LICENSES.md +++ b/THIRD_PARTY_LICENSES.md @@ -35,6 +35,34 @@ BSL-1.0, Unlicense/CC0. License files verified in each upstream archive. | {fmt} | 12.2.0 | MIT | https://github.com/fmtlib/fmt | Formatting (kernel-wide, no iostream) | | spdlog | 1.17.0 | MIT | https://github.com/gabime/spdlog | Logging (built against external fmt) | +## Documentation-site build tools (npm, `docs-site/` only) + +Build-time only, on Linux CI and on a contributor's machine. **Nothing here is +compiled into, linked by, or shipped inside any RoadMaker artifact** — the +kernel, the Python wheels, and the editor never see them, and CMake never +invokes npm. What ships from this tree is the generated HTML, which is +RoadMaker's own content. + +Direct dependencies only; the full installed tree (374 packages at the time of +writing, all MIT/ISC/Apache-2.0/BSD/BlueOak/Python-2.0) is verified on every CI +run by `npm run licenses`, which fails on anything outside the permitted set. +Policy and the monthly update cadence: +[dependencies — npm](docs/standards/dependencies.md). + +| What | Version | License | Where | Notes | +|---|---|---|---|---| +| Astro | 5.14.1 | MIT | https://github.com/withastro/astro | Static site generator for the documentation site | +| Starlight | 0.36.0 | MIT | https://github.com/withastro/starlight | Documentation theme built on Astro | +| Node.js | 24.x (pinned by `.nvmrc` + `engines`) | MIT | https://nodejs.org | Runtime for the adapter, build and check scripts | + +**`sharp` is deliberately stubbed out.** Astro lists it as an optional +dependency for its default image service, and its prebuilt libvips binaries are +**LGPL-3.0-or-later**. Qt is this project's only sanctioned LGPL dependency, so +`docs-site/package.json` overrides `sharp` to a local no-op that throws if +anything imports it, and `astro.config.mjs` uses the passthrough image service +instead. `--omit=optional` was not usable — it would also drop a required +native binary. No libvips binary is ever downloaded, built, or shipped. + ## Bundled documentation (not dependencies) Third-party copyrighted material that lives in the repository but is **not** diff --git a/docs-site/README.md b/docs-site/README.md index 5a8da1ec..e435a9e5 100644 --- a/docs-site/README.md +++ b/docs-site/README.md @@ -38,9 +38,17 @@ npm run build:local # the offline reader that ships in a release npm run build:web -- --base=/dev/ # the published site, for one version npm run dev # same as build, then a dev server npm run licenses # licence gate over the installed tree +npm run check:links # internal links, over a build or a whole publish tree +npm run report:external # outbound links — a report, never a failure npm test # script tests (node:test) ``` +Both builds end with `check:links`, so a broken reference fails the build that +produced it. Each build also writes a `.rm-docs-build.json` stamp recording the +base it used: the checker runs in its own shell, where the build's environment +is long gone, and a checker that assumed the wrong base would report every link +in the build as broken. + The adapter **fails the build** on a broken link, naming the source page and the target. diff --git a/docs-site/package.json b/docs-site/package.json index 763b27ad..c0a9a308 100644 --- a/docs-site/package.json +++ b/docs-site/package.json @@ -19,7 +19,9 @@ "licenses": "node scripts/licenses.mjs", "check": "node scripts/check-f1-coverage.mjs", "check:local": "node scripts/check-local-build.mjs", - "test": "node --test \"test/**/*.test.mjs\"" + "test": "node --test \"test/**/*.test.mjs\"", + "check:links": "node scripts/check-links.mjs", + "report:external": "node scripts/report-external-links.mjs" }, "dependencies": { "@astrojs/starlight": "0.36.0", diff --git a/docs-site/scripts/build-local.mjs b/docs-site/scripts/build-local.mjs index 179fa182..f8c2e4f6 100644 --- a/docs-site/scripts/build-local.mjs +++ b/docs-site/scripts/build-local.mjs @@ -22,6 +22,7 @@ // The release packaging job runs this and hands the resulting dist/ to CMake as // a path; CMake never invokes npm. import { spawnSync } from 'node:child_process'; +import { writeFileSync } from 'node:fs'; import { dirname, join } from 'node:path'; import { fileURLToPath } from 'node:url'; @@ -51,6 +52,15 @@ step('adapt docs/user-guide', node, [join(here, 'adapt.mjs')]); step('F1 coverage', node, [join(here, 'check-f1-coverage.mjs')]); step('astro build (file format, search off)', node, [astro, 'build']); step('relativize references', node, [join(here, 'relativize.mjs')]); + +// See build-web.mjs: the checker runs in its own shell and must not have to +// guess. The offline reader's references are relative, so its base is `/`. +writeFileSync( + join(root, 'dist', '.rm-docs-build.json'), + `${JSON.stringify({ target: 'local', base: '/' }, null, 2)}\n`, +); + step('verify the local build', node, [join(here, 'check-local-build.mjs')]); +step('internal links', node, [join(here, 'check-links.mjs')]); console.log('\nbuild:local: dist/ is ready to open from file://'); diff --git a/docs-site/scripts/build-web.mjs b/docs-site/scripts/build-web.mjs index 39067c5c..dfaf2d7d 100644 --- a/docs-site/scripts/build-web.mjs +++ b/docs-site/scripts/build-web.mjs @@ -21,6 +21,7 @@ // Markdown — content Astro passes through untouched). Passing it through one // environment variable is what keeps those two from disagreeing. import { spawnSync } from 'node:child_process'; +import { writeFileSync } from 'node:fs'; import { dirname, join } from 'node:path'; import { fileURLToPath } from 'node:url'; @@ -59,6 +60,18 @@ step('theme tokens', [join(here, 'theme-css.mjs')]); step('adapt docs/user-guide', [join(here, 'adapt.mjs')]); step('F1 coverage', [join(here, 'check-f1-coverage.mjs')]); step('astro build', [join(root, 'node_modules', 'astro', 'astro.js'), 'build']); + +// Record the base in the output. check-links.mjs runs in its own shell — in CI +// and on a contributor's machine — where this environment variable no longer +// exists, and a link checker that assumes the wrong base reports every link in +// the build as broken. The stamp rides along into the published tree, so a +// version directory (and the `latest/` copy of one) stays self-describing. +writeFileSync( + join(root, 'dist', '.rm-docs-build.json'), + `${JSON.stringify({ target: 'web', base }, null, 2)}\n`, +); + step('verify the base', [join(here, 'check-web-build.mjs')]); +step('internal links', [join(here, 'check-links.mjs')]); console.log(`\nbuild:web: dist/ is ready to publish under ${base}`); diff --git a/docs-site/scripts/check-links.mjs b/docs-site/scripts/check-links.mjs new file mode 100644 index 00000000..bc27e9e4 --- /dev/null +++ b/docs-site/scripts/check-links.mjs @@ -0,0 +1,221 @@ +// Copyright 2026 Robomous +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// Internal link and image check over a BUILT tree (ADR-0009 / docs-s4). +// +// Run it on one build, or on the whole assembled publish tree — in which case +// it checks `dev/` and every version directory, each against its own base, and +// the root redirect and manifest too. That is the point: the adapter's check +// runs on source, and a per-build check cannot see a cross-version link or a +// version directory that was published months ago and is still being served. +// +// EXTERNAL LINKS ARE NOT FETCHED HERE. Link rot in somebody else's URL is not a +// reason a merge cannot happen; report-external-links.mjs reports on those +// without failing anything. +import { readFileSync, existsSync, statSync } from 'node:fs'; +import { join, posix, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +import { htmlFiles } from './relativize.mjs'; + +/** Name of the stamp each build drops so its base is knowable afterwards. */ +export const STAMP = '.rm-docs-build.json'; + +/** + * The base a built directory was produced with. + * + * Read from the stamp the build writes, NOT from the environment: this script + * runs in its own shell in CI and in a contributor's terminal, where whatever + * `RM_DOCS_BASE` the build used is long gone. Inferring it from the output + * would be guesswork, and guessing wrong turns every link into a false + * positive. + */ +export function baseOf(dir, fallback = '/') { + const stamp = join(dir, STAMP); + if (existsSync(stamp)) { + try { + const parsed = JSON.parse(readFileSync(stamp, 'utf8')); + if (typeof parsed.base === 'string' && parsed.base.startsWith('/')) return parsed.base; + } catch { + // A corrupt stamp is not worth failing over; fall through. + } + } + return fallback; +} + +/** Roots to check: each version directory of a publish tree, or the build itself. */ +export function findRoots(dir) { + const manifest = join(dir, 'versions.json'); + if (existsSync(manifest)) { + const parsed = JSON.parse(readFileSync(manifest, 'utf8')); + const roots = (parsed.versions ?? []) + .map((version) => version.path) + .filter((path) => existsSync(join(dir, path))) + .map((path) => ({ + label: path, + dir: join(dir, path), + base: baseOf(join(dir, path), `/${path}/`), + })); + // `latest/` is a byte copy of the highest version, so its pages — and its + // stamp — carry THAT version's base, not `/latest/`. Checking it against + // `/latest/` would report every link in it as broken. + // + // The consequence is worth knowing: a reader who opens `/latest/` is moved + // to the pinned version's URL on their first click. Every link resolves — + // `/latest/` is an entry point, not a browsable mirror. + if (existsSync(join(dir, 'latest'))) { + const fallback = parsed.latest ? `/${parsed.latest}/` : '/latest/'; + roots.push({ + label: 'latest', + dir: join(dir, 'latest'), + base: baseOf(join(dir, 'latest'), fallback), + }); + } + return roots; + } + return [{ label: '.', dir, base: baseOf(dir, process.env.RM_DOCS_BASE ?? '/') }]; +} + +/** + * Resolve one reference against a page. + * + * Handles all three link shapes the two builds produce: relative (the offline + * reader), root-absolute under a base (the published site), and the + * extensionless slugs the adapter emits for content links. + */ +export function resolveRef(root, base, pageRel, value) { + // Off-tree schemes answer for themselves. The caller filters these too, to + // keep them out of the reference count — but this function is exported, and + // one that silently called every external URL "broken" would be a trap. + if (/^(https?:|mailto:|tel:|data:|javascript:|rmmanual:)/i.test(value)) { + return { kind: 'external' }; + } + const withoutHash = value.split('#')[0].split('?')[0]; + if (withoutHash === '') return { kind: 'in-page' }; + + let target; + if (withoutHash.startsWith('/')) { + if (withoutHash.startsWith('//')) return { kind: 'external' }; + if (!withoutHash.startsWith(base)) { + return { kind: 'broken', reason: `outside the base ${base}` }; + } + target = withoutHash.slice(base.length); + } else { + target = posix.normalize(posix.join(posix.dirname(pageRel), withoutHash)); + if (target.startsWith('..')) { + return { kind: 'broken', reason: 'escapes the version root' }; + } + } + + const clean = target.replace(/\/+$/, ''); + const candidates = + clean === '' ? ['index.html'] : [clean, `${clean}.html`, `${clean}/index.html`]; + for (const candidate of candidates) { + const full = join(root, candidate); + if (existsSync(full) && statSync(full).isFile()) return { kind: 'ok', file: candidate }; + } + return { kind: 'broken', reason: 'matches no file' }; +} + +/** Check one built root. Returns the problems found. */ +export function checkRoot({ label, dir, base }) { + const problems = []; + let refs = 0; + const pages = htmlFiles(dir); + if (pages.length === 0) { + problems.push(`${label}: contains no HTML pages at all`); + return { problems, pages: 0, refs }; + } + + for (const pageRel of pages) { + const html = readFileSync(join(dir, pageRel), 'utf8'); + for (const match of html.matchAll(/\b(href|src|srcset)="([^"]*)"/g)) { + const [, attr, value] = match; + const urls = + attr === 'srcset' ? value.split(',').map((p) => p.trim().split(/\s+/)[0]) : [value]; + for (const url of urls) { + if (/^(https?:|mailto:|tel:|data:|javascript:|rmmanual:)/i.test(url)) continue; + refs += 1; + const outcome = resolveRef(dir, base, pageRel, url); + if (outcome.kind === 'broken') { + problems.push(`${label}/${pageRel}: ${attr}="${url}" — ${outcome.reason}`); + } + } + } + } + return { problems, pages: pages.length, refs }; +} + +/** The publish tree's own derived files, which no page links but readers hit. */ +export function checkPublishRoot(dir) { + const problems = []; + const manifest = join(dir, 'versions.json'); + if (!existsSync(manifest)) return problems; + + const parsed = JSON.parse(readFileSync(manifest, 'utf8')); + for (const version of parsed.versions ?? []) { + if (!existsSync(join(dir, version.path, 'index.html'))) { + problems.push(`versions.json lists '${version.path}', which has no index.html`); + } + } + if (parsed.latest && !existsSync(join(dir, 'latest', 'index.html'))) { + problems.push(`versions.json names latest='${parsed.latest}' but latest/ has no index.html`); + } + + const rootIndex = join(dir, 'index.html'); + if (!existsSync(rootIndex)) { + problems.push('the publish root has no index.html — nothing to redirect a visitor'); + } else if (parsed.default) { + const html = readFileSync(rootIndex, 'utf8'); + if (!html.includes(`./${parsed.default}/`)) { + problems.push( + `the root redirect does not point at '${parsed.default}/', which versions.json calls the default`, + ); + } + } + return problems; +} + +// ------------------------------------------------------------------ as a script + +const here = dirname(fileURLToPath(import.meta.url)); +if (process.argv[1] && join(process.argv[1]) === fileURLToPath(import.meta.url)) { + const dir = process.argv[2] ?? join(here, '..', 'dist'); + if (!existsSync(dir) || !statSync(dir).isDirectory()) { + console.error(`check-links: nothing at ${dir}`); + process.exit(1); + } + + const roots = findRoots(dir); + const problems = [...checkPublishRoot(dir)]; + let pages = 0; + let refs = 0; + for (const root of roots) { + const result = checkRoot(root); + problems.push(...result.problems); + pages += result.pages; + refs += result.refs; + } + + if (problems.length > 0) { + console.error(`check-links: ${problems.length} broken reference(s)`); + for (const problem of problems.slice(0, 50)) console.error(` ${problem}`); + if (problems.length > 50) console.error(` …and ${problems.length - 50} more`); + process.exit(1); + } + console.log( + `check-links: ${roots.length} root(s) [${roots.map((r) => r.label).join(', ')}], ` + + `${pages} pages, ${refs} internal references, all resolve`, + ); +} diff --git a/docs-site/scripts/report-external-links.mjs b/docs-site/scripts/report-external-links.mjs new file mode 100644 index 00000000..00080a99 --- /dev/null +++ b/docs-site/scripts/report-external-links.mjs @@ -0,0 +1,125 @@ +// Copyright 2026 Robomous +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// Reports outbound links that look dead (ADR-0009 / docs-s4). +// +// DELIBERATELY NON-BLOCKING. It always exits 0, even when every request fails. +// A third party rearranging their site, or rate-limiting a CI runner, is not a +// reason a contributor's merge cannot happen — and a gate that fails for +// reasons nobody in this repository can fix is a gate people learn to ignore, +// which costs more than the link rot it was meant to catch. +// +// The output is a report to read, not a status to satisfy. +import { readFileSync, existsSync, statSync } from 'node:fs'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +import { htmlFiles } from './relativize.mjs'; + +const TIMEOUT_MS = 15_000; +const CONCURRENCY = 8; + +/** Every distinct external URL in a built tree, with the pages that use it. */ +export function collectExternalLinks(dir) { + const found = new Map(); + for (const pageRel of htmlFiles(dir)) { + const html = readFileSync(join(dir, pageRel), 'utf8'); + for (const match of html.matchAll(/\bhref="(https?:\/\/[^"]+)"/g)) { + const url = match[1].split('#')[0]; + if (!found.has(url)) found.set(url, new Set()); + found.get(url).add(pageRel); + } + } + return found; +} + +/** HEAD, falling back to GET — a fair number of hosts refuse HEAD outright. */ +async function probe(url) { + for (const method of ['HEAD', 'GET']) { + try { + const response = await fetch(url, { + method, + redirect: 'follow', + signal: AbortSignal.timeout(TIMEOUT_MS), + headers: { 'user-agent': 'RoadMaker-docs-link-report' }, + }); + if (response.ok) return { ok: true, status: response.status }; + if (method === 'GET') return { ok: false, status: String(response.status) }; + } catch (error) { + if (method === 'GET') return { ok: false, status: error.name ?? 'error' }; + } + } + return { ok: false, status: 'unknown' }; +} + +async function main() { + const here = dirname(fileURLToPath(import.meta.url)); + const dir = process.argv[2] ?? join(here, '..', 'dist'); + if (!existsSync(dir) || !statSync(dir).isDirectory()) { + console.log(`external-links: nothing at ${dir} — skipping the report`); + return; + } + + const links = [...collectExternalLinks(dir).entries()]; + console.log(`external-links: probing ${links.length} distinct outbound URL(s)`); + + const suspect = []; + let index = 0; + const workers = Array.from({ length: Math.min(CONCURRENCY, links.length) }, async () => { + while (index < links.length) { + const [url, pages] = links[index++]; + const result = await probe(url); + if (!result.ok) suspect.push({ url, status: result.status, pages: [...pages] }); + } + }); + await Promise.all(workers); + + if (suspect.length === 0) { + console.log('external-links: every outbound link answered'); + return; + } + + suspect.sort((a, b) => a.url.localeCompare(b.url)); + console.log(`\nexternal-links: ${suspect.length} did not answer (REPORT ONLY — nothing fails):`); + for (const entry of suspect) { + console.log(` [${entry.status}] ${entry.url}`); + for (const page of entry.pages.slice(0, 3)) console.log(` on ${page}`); + if (entry.pages.length > 3) console.log(` …and ${entry.pages.length - 3} more pages`); + } + + const summary = process.env.GITHUB_STEP_SUMMARY; + if (summary) { + const lines = [ + '### Outbound links that did not answer', + '', + 'Report only — this does not fail the build. A third party rearranging their', + 'site is not something a merge should wait on.', + '', + '| Status | URL | Pages |', + '|---|---|---|', + ...suspect.map((e) => `| \`${e.status}\` | ${e.url} | ${e.pages.length} |`), + '', + ]; + const { appendFileSync } = await import('node:fs'); + appendFileSync(summary, lines.join('\n')); + } +} + +// Only when run as a script — importing this module (the tests do, for +// collectExternalLinks) must not fire a few dozen network requests. +if (process.argv[1] && join(process.argv[1]) === fileURLToPath(import.meta.url)) { + await main(); + // Explicit: this script never fails a build. + process.exitCode = 0; +} diff --git a/docs-site/test/check-links.test.mjs b/docs-site/test/check-links.test.mjs new file mode 100644 index 00000000..7a0d92ba --- /dev/null +++ b/docs-site/test/check-links.test.mjs @@ -0,0 +1,174 @@ +// Copyright 2026 Robomous +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { checkRoot, checkPublishRoot, findRoots, resolveRef } from '../scripts/check-links.mjs'; +import { assemble } from '../scripts/assemble.mjs'; +import { collectExternalLinks } from '../scripts/report-external-links.mjs'; + +/** A built version directory whose links are all sound. */ +function makeVersion(label, extraBody = '') { + const dir = mkdtempSync(join(tmpdir(), 'rm-ver-')); + mkdirSync(join(dir, 'reference'), { recursive: true }); + mkdirSync(join(dir, 'img'), { recursive: true }); + writeFileSync(join(dir, 'img', 'shot.png'), ''); + writeFileSync( + join(dir, 'index.html'), + `

${label}

j${extraBody}`, + ); + writeFileSync( + join(dir, 'reference', 'junction.html'), + `home` + + `outhere` + + `bridge`, + ); + return dir; +} + +test('a sound build passes and counts what it checked', () => { + const dir = makeVersion('dev'); + try { + const result = checkRoot({ label: 'dev', dir, base: '/dev/' }); + assert.deepEqual(result.problems, []); + assert.equal(result.pages, 2); + assert.ok(result.refs > 0); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('external, in-page, and rmmanual references are not treated as files', () => { + // rmmanual: is the in-app bridge scheme — it resolves at runtime against the + // packaged manual and can never be a file in this tree. Reporting any of + // these as broken would be a flood of false positives. + for (const url of [ + 'https://x.invalid/y', + 'http://x.invalid/y', + 'mailto:someone@example.invalid', + 'rmmanual:tutorials/getting-around', + '//cdn.invalid/lib.js', + ]) { + assert.equal( + resolveRef('/nowhere', '/dev/', 'index.html', url).kind, + 'external', + `${url} must be treated as external`, + ); + } + assert.equal(resolveRef('/nowhere', '/dev/', 'index.html', '#top').kind, 'in-page'); +}); + +test('a dangling relative link is reported', () => { + const dir = makeVersion('dev', 'x'); + try { + const { problems } = checkRoot({ label: 'dev', dir, base: '/dev/' }); + assert.equal(problems.length, 1); + assert.match(problems[0], /gone\.html/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('a missing image is reported', () => { + const dir = makeVersion('dev', ''); + try { + const { problems } = checkRoot({ label: 'dev', dir, base: '/dev/' }); + assert.equal(problems.length, 1); + assert.match(problems[0], /absent\.png/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('an absolute link that leaves this version is reported', () => { + // The exact shape of a cross-version link written by hand: it would silently + // send a v0.1.0 reader into dev. + const dir = makeVersion('dev', 'other'); + try { + const { problems } = checkRoot({ label: 'dev', dir, base: '/dev/' }); + assert.equal(problems.length, 1); + assert.match(problems[0], /outside the base/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('a relative link climbing out of the version root is reported', () => { + const dir = makeVersion('dev', 'x'); + try { + const { problems } = checkRoot({ label: 'dev', dir, base: '/dev/' }); + assert.equal(problems.length, 1); + assert.match(problems[0], /escapes the version root/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('the assembled tree is checked version by version, latest included', () => { + const root = mkdtempSync(join(tmpdir(), 'rm-pub-')); + const dev = makeVersion('dev'); + const rel = makeVersion('v0.1.0'); + try { + assemble({ publishRoot: root, buildDir: dev, segment: 'dev' }); + assemble({ publishRoot: root, buildDir: rel, segment: 'v0.1.0' }); + + const roots = findRoots(root); + assert.deepEqual( + roots.map((r) => r.label).sort(), + ['dev', 'latest', 'v0.1.0'], + 'every version directory, plus latest/ — which readers land on', + ); + + // latest/ is a BYTE COPY of v0.1.0, so its pages carry v0.1.0's base. + // findRoots has to hand it that base or every link in it reads as broken. + assert.equal(roots.find((r) => r.label === 'latest').base, '/v0.1.0/'); + + for (const one of roots) { + assert.deepEqual(checkRoot(one).problems, [], `${one.label} should be clean`); + } + assert.deepEqual(checkPublishRoot(root), []); + } finally { + for (const d of [root, dev, rel]) rmSync(d, { recursive: true, force: true }); + } +}); + +test('a publish root whose redirect disagrees with the manifest is reported', () => { + const root = mkdtempSync(join(tmpdir(), 'rm-pub-')); + const dev = makeVersion('dev'); + try { + assemble({ publishRoot: root, buildDir: dev, segment: 'dev' }); + writeFileSync(join(root, 'index.html'), '

nowhere in particular

'); + const problems = checkPublishRoot(root); + assert.equal(problems.length, 1); + assert.match(problems[0], /root redirect/); + } finally { + rmSync(root, { recursive: true, force: true }); + rmSync(dev, { recursive: true, force: true }); + } +}); + +test('external links are collected with the pages that use them', () => { + const dir = makeVersion('dev'); + try { + const found = collectExternalLinks(dir); + assert.ok(found.has('https://example.invalid/x')); + assert.deepEqual([...found.get('https://example.invalid/x')], ['reference/junction.html']); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); diff --git a/docs/README.md b/docs/README.md index 1ea71234..1c4abe26 100644 --- a/docs/README.md +++ b/docs/README.md @@ -37,10 +37,12 @@ OpenDRIVE scene in the editor: [Create Road](user-guide/reference/create-road.md 3. [Testing](contributing/testing.md) — GoogleTest/pytest doctrine, headless Qt tests, sanitizers, fuzzing. 4. [CI](contributing/ci.md) — what each gate checks. -5. [Publishing the documentation site](contributing/docs-site-publishing.md) — +5. [Writing user documentation](contributing/documentation.md) — which tier a + page belongs to, and what syntax each one allows. +6. [Publishing the documentation site](contributing/docs-site-publishing.md) — how a merged docs change reaches a live page, and the maintainer runbook for the hosting app. -6. The standards your change must meet: +7. The standards your change must meet: [C++ style](standards/cpp-style.md) · [Cross-platform](standards/cross-platform.md) · [Dependencies & licensing](standards/dependencies.md) · @@ -102,5 +104,9 @@ editing docs: - No page over ~300 lines — split instead. - Diagrams are Mermaid, inline in the page. -The tree is plain Markdown, structured so a static-site generator (e.g. -MkDocs) could be adopted later without moving files. +The tree is plain Markdown. `docs/user-guide/` is additionally the authored +source for two generated outputs — the in-app help book and the documentation +site ([ADR-0009](decisions/0009-documentation-site-tiered-docs.md)) — which +constrains what may be written there; see +[Writing user documentation](contributing/documentation.md). The rest of `docs/` +has no pipeline and no syntax budget. diff --git a/docs/contributing/documentation.md b/docs/contributing/documentation.md new file mode 100644 index 00000000..59ddcd18 --- /dev/null +++ b/docs/contributing/documentation.md @@ -0,0 +1,241 @@ +# Writing user documentation + +*I want to document a thing — where does it go, and what may I write?* + +This page answers both, concretely enough to follow without reading any +generator source. Decided in +[ADR-0009](../decisions/0009-documentation-site-tiered-docs.md). + +It covers `docs/user-guide/` — the documentation **users** read. Contributor +documentation (everything else under `docs/`) has no pipeline and no syntax +budget; write it as you like. + +## The one rule everything else follows from + +`docs/user-guide/` is the **only** place this content is authored. Two +generators read it: + +| Generator | Produces | Reads | +|---|---|---| +| `rm_helpc` (C++) | the in-app `.qch` book that `F1` opens | `index.md` + `reference/` | +| `docs-site/scripts/adapt.mjs` (Node) | the web site and the offline manual | everything | + +**Never hand-edit generated output.** `docs-site/src/content/docs/` and +`docs-site/src/styles/theme.css` are build products and are gitignored; an edit +there is silently lost on the next build. Change the source. + +## Which tier? + +Ask what the page is *for*, not how long it is. + +| | **Reference** | **Guides** | +|---|---|---| +| Where | `docs/user-guide/reference/` | `docs/user-guide/tutorials/` (and `guides/`) | +| What | one tool or one panel: what it does, its parameters, its shortcuts | a walkthrough that strings tools together to build something | +| Rendered by | the in-app book **and** the site | the site only | +| Syntax | strict CommonMark subset — see below | Markdown plus Starlight asides | +| Reached by | `F1` in the editor, and the site | the site | + +A reference page is short and answers "what is this control?" while the user is +looking at it. A guide answers "how do I build a T-junction?" and may be long +and heavily illustrated. + +If a page is genuinely both — a reference head with a tutorial welded on — split +it. `reference/objects-signals.md` is the standing example of one that has not +been split yet. + +### Adding a reference page + +1. Write `docs/user-guide/reference/.md`, starting with a single `# H1`. +2. **Link it from `docs/user-guide/index.md`.** This is the step that is easy to + forget and silent to skip — see the manifest rule below. +3. If the page documents a tool or a panel that `F1` should open it for, add the + mapping in `editor/src/help/help_registry.cpp`. `test_help_registry.cpp` + gates that every tool has one. + +### Adding a guide page + +1. Write `docs/user-guide/tutorials/.md`, starting with a single `# H1`. +2. Link it from `docs/user-guide/index.md` so readers can find it. The `.qhp` + generator skips the `tutorials/` prefix deliberately, so this link orders the + page on the site without pulling it into the in-app book. + +## `index.md` is the ordering manifest, for both pipelines + +`docs/user-guide/index.md` is not a courtesy table of contents. **Both** +generators read its links, in document order: + +- `helpc::build_toc()` ingests exactly the reference pages it links, in that + order, and nothing else; +- `adapt.mjs` derives the site's reference-tier sidebar order from the same + list. + +That is deliberate: one manifest is what stops the two outputs drifting. + +**A reference page not linked from `index.md` is invisible to the in-app book.** +It will not appear, and nothing will fail — the page simply is not in the +collection. If it is also an `F1` target, the coverage gate catches it, because +`test_help_registry.cpp` greps `index.md` for the literal `(.md)`. If it +is not an `F1` target, nothing catches it. Link the page. + +## Syntax budget for `reference/` + +Reference pages pass through **md4c** with `MD_DIALECT_GITHUB`, and the result +is rendered by `QTextBrowser`, which supports a limited HTML subset. The budget +is therefore the intersection of the two, and it is narrow. + +**Works:** headings, paragraphs, emphasis, lists, blockquotes, fenced and +indented code, links, images, **tables**, **strikethrough**, **task lists**, and +bare-URL autolinks. + +**Does not exist — do not use:** + +| Not available | Why | +|---|---| +| Footnotes | md4c has no footnote support at all | +| YAML front matter | nothing strips it; it renders as literal text at the top of the page | +| `:::note` and other admonitions | Starlight syntax; the in-app renderer prints the colons | +| Anything a code fence's *info string* is supposed to trigger | the fence renders as plain preformatted text; the language is not read | +| LaTeX math, wiki links, `__underline__` | not in `MD_DIALECT_GITHUB` | + +### No heading anchors — the one that bites + +The in-app renderer emits **no `id` attributes on headings**. A link written as +`page.md#section` therefore lands at the **top** of the target page in the app, +while working correctly on the site. + +That is not a bug to route around; it is a constraint to design for. Link to a +page, not to a section within it. If you find yourself needing to point at one +section of a long reference page, the page probably wants splitting. + +### `` — settled: use backticks + +Seven pages use `Shift` inline HTML; the rest use `` `Shift` `` for +the same thing. + +**The rule is backticks.** Three reasons: `QTextBrowser` renders `` +unstyled, so in the app it looks exactly like the surrounding prose and the +distinction it was reaching for is lost; backticks are pure CommonMark and +render identically in the app, on the site, and on GitHub; and it is already the +majority spelling. + +Write `` `Shift`+`L` ``, not `Shift+L`. + +*Normalising the existing seven pages is a separate change — this page settles +the rule, it does not apply it.* + +## Syntax budget for guides + +Markdown plus **Starlight asides**: + +```markdown +:::note +Useful but skippable. +::: + +:::caution[Watch out] +Something that will cost you time if you miss it. +::: +``` + +**MDX and JavaScript components stay deferred.** The reason is the offline +reader: the manual bundled in every release opens from `file://`, and keeping +guides to Markdown plus asides keeps that build static, portable, and free of +anything that needs a server or a hydration runtime. Ask before reaching for +more. + +## The reference → guide bridge + +A reference page may end with a section under this exact heading: + +```markdown +## Full guide + +[Shaping lanes](../tutorials/shaping-lanes.md) — the whole cross-section pass. +``` + +The **heading** is the marker, so what you write stays an ordinary relative +Markdown link that works on GitHub. Each generator retargets it: the site emits +a normal link, and the in-app book emits a URL that opens the packaged manual in +your system browser. + +Rules: + +- exactly one bridge section per page, as the **last** section; +- the **first** link in it is the bridge; prose after it is fine; +- the target must be a real page in the guides tier. + +**CI verifies the target exists**, from both sides: +`HelpBridge.EveryBridgeTargetIsAPageThatExists` checks it in C++ against +`docs/user-guide/`, and the site adapter fails on the same broken link. Renaming +a tutorial without updating the pages that bridge to it fails the build twice. + +## Images + +Put the image beside the page that uses it, in that tier's `img/` folder: + +| Page | Image | +|---|---| +| `reference/create-road.md` | `reference/img/create-road.png` | +| `tutorials/shaping-lanes.md` | `tutorials/img/shaping-lanes.png` | +| `index.md` | `img/…` | + +Reference it relatively: `![Alt text](img/create-road.png)`. + +Two constraints come from the in-app pipeline, and both are silent when broken: + +- **`png`, `gif` and `jpg` only.** An `svg` will not be bundled. (A `.gif` is + bundled but will not animate in `QTextBrowser` — treat it as a still.) +- **The `` patterns do not recurse.** `helpc/qhp.cpp` lists + `img/*.png`, `reference/img/*.png` and so on, explicitly. **A new image folder + needs its own patterns added there**, or its images are simply absent from the + in-app book while the page renders fine on the site. + +Always write alt text: it is what a screen reader has, and what shows when an +image is missing. + +## Checking your work + +```sh +cd docs-site +npm ci # once +npm run build:web -- --base=/dev/ # the site, as published +npm run build:local # the offline reader +npm run check:links # every internal link and image resolves +``` + +The adapter **fails** on a broken link and names the page and the target. The +C++ side is covered by the editor test suite: + +```sh +cmake --build --preset dev-macos --target roadmaker_editor_tests +ctest --preset dev-macos -R 'Help|Manual' +``` + +To see your reference page in the app, build the editor and press `F1`. + +## The end-to-end walkthrough + +Adding one page of each tier, start to finish — no steps beyond this page: + +1. `docs/user-guide/reference/my-tool.md`, beginning `# My Tool`. +2. `docs/user-guide/tutorials/using-my-tool.md`, beginning `# Using My Tool`. +3. In `docs/user-guide/index.md`, add a row to the tools table linking + `reference/my-tool.md`, and a row to the tutorials table linking + `tutorials/using-my-tool.md`. +4. Optionally end the reference page with a `## Full guide` section linking + `../tutorials/using-my-tool.md`. +5. `cd docs-site && npm run build:web -- --base=/dev/` — both pages appear on + the site, the reference one in the Reference sidebar at the position its + `index.md` row occupies. +6. Build the editor — the reference page is in the in-app book; the tutorial is + **not**, which is the tier split working. + +## See also + +- [Publishing the documentation site](docs-site-publishing.md) — how a merged + change reaches a live page. +- [`docs-site/README.md`](https://github.com/Robomous/RoadMaker/blob/main/docs-site/README.md) + — the two builds, the adapter, and the licence gate. +- [ADR-0009](../decisions/0009-documentation-site-tiered-docs.md) — why the + model is what it is. diff --git a/docs/roadmap/README.md b/docs/roadmap/README.md index 1e2c835c..c43e4b3c 100644 --- a/docs/roadmap/README.md +++ b/docs/roadmap/README.md @@ -259,8 +259,9 @@ model shrinks the dual-source surface to the reference pages alone. - `docs/user-guide/index.md` is already the `.qhp` pipeline's **ordering manifest** — the generator ingests exactly the pages that page links, in link order — so it keeps that role for the reference tier and the adapter - reads the same order. Guides order by folder structure plus an optional - `_order` manifest the `.qhp` generator ignores. + reads the same order. Guides order by folder structure (Starlight's + `autogenerate`); the `_order` manifest sketched during planning was not + needed and was not built. - Astro Starlight lives in a new top-level `docs-site/` folder. `docs/` remains the public contributor source of truth; `docs-site/` is tooling. Node LTS pinned (`engines` + `.nvmrc`), `package-lock.json` committed, diff --git a/docs/standards/dependencies.md b/docs/standards/dependencies.md index 0001e00e..e28cb5a3 100644 --- a/docs/standards/dependencies.md +++ b/docs/standards/dependencies.md @@ -56,6 +56,60 @@ replacements. Do not "helpfully" add the forbidden one: | 2D triangulation / plan-view ops | CDT (MPL-2.0) + Clipper2 (BSL-1.0) | CGAL; Triangle (non-commercial-only!) | | Scene export | glTF via tinygltf, later OpenUSD | FBX SDK (proprietary; FBX may become an out-of-tree plugin built against the user's own SDK) | +## npm dependencies (`docs-site/` only) + +The documentation site is the one part of this repository with an npm tree +([ADR-0009](../decisions/0009-documentation-site-tiered-docs.md)). The same +licence rules apply, with three additions: + +- **It is scoped to `docs-site/`.** Nothing else in the repository has a + `package.json`, and **CMake never invokes npm** — a developer build of the + kernel or the editor needs no Node at all. +- **`package-lock.json` is committed and CI runs `npm ci`**, so what CI resolves + is exactly what a contributor resolved. +- **`npm run licenses` is a gate.** It reads every *installed* package's own + `package.json` — the lockfile does not record licences — and fails on anything + outside the permitted set. A package that ships a LICENSE file and declares + nothing in `package.json` is read from the file rather than failed, because + the file is still the grant. **Extend that fallback for a similar case; never + weaken the gate.** If a dependency resolves non-permissive and no permissive + alternative exists, stop and ask the maintainer. + +`sharp` is the standing example of the policy biting: the site's framework +lists it as an optional dependency for image processing, and its prebuilt +libvips binaries are **LGPL-3.0-or-later**. Qt is this project's only sanctioned +LGPL dependency, so `package.json` overrides `sharp` to a local no-op stub that +throws if anything imports it, and the site uses a passthrough image service. +`--omit=optional` was not usable — it would also drop a required native binary. + +### Update cadence: monthly + +Once a month, and otherwise only when something needs it: + +```sh +cd docs-site +npm outdated # what has moved +npm update # within the declared ranges +npm install @ # for a major, deliberately +npm run licenses # the gate, before anything else +npm test && npm run build:web -- --base=/dev/ && npm run build:local +``` + +Commit the updated `package-lock.json` with the change. Record any new direct +dependency in `THIRD_PARTY_LICENSES.md`. + +**No npm automation may open pull requests against this repository or fail a job +for the C++ side.** Dependency bots and advisory scanners are deliberately not +enabled here: a docs-site advisory that cannot affect the kernel must never +appear as a red check on a kernel change. That is why the cadence is a documented +human task rather than a robot. + +Judge an advisory by whether it can reach *this* deployment. The site is +statically prerendered with no adapter and no server, so advisories confined to a +dev server, SSR, middleware, server islands, or an image endpoint do not apply to +what is published — note the assessment in the update commit rather than +silently ignoring it. + ## Adding a dependency (checklist — all steps, one commit) 1. **Verify the license file in the upstream repository**, not just the