feat: detect wiki source references invalidated by a file move - #636
Open
Jithin (JM) Mohandas (DecentralizedJM) wants to merge 4 commits into
Open
Conversation
A wiki's most frequent claim is that some behavior lives in a given file. A repository reorganization invalidates every one of those claims without touching the wiki, and because the generated Markdown writes them as inline code spans rather than links, internal-link validation cannot see them. Resolve the repository paths each page cites and report the ones a move explains, naming where the file now lives. A finalize-pass validator stamps them in place so a later update run self-corrects, matching the existing mermaid and link validators. `openwiki doctor` reports the same findings without writing, adds the pages whose cited files changed since the commit the wiki last documented, and exits non-zero when a reference is stale. Both are deterministic and make no model calls. Only citations with a same-named file elsewhere in the repository are reported. A missing file with no such twin is ambiguous -- it may be a path the reader creates in their own repository, a runtime artifact, or a real deletion -- and flagging those costs more trust than it returns. Also fix the internal-link dogfood test, which resolved its root to test/ rather than the repository root and so scanned nothing and passed vacuously.
🦋 Changeset detectedLatest commit: 88b6ac3 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Author
|
Colin Francis (@colifran) Please have a look when you have time |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A wiki's most frequent factual claim is "this behavior lives in this file", and a refactor invalidates every one of them without touching the wiki. Readers follow paths that no longer exist, and an agent using the wiki as memory falls back to crawling the repo -- the expensive behavior the wiki was meant to replace. The generated Markdown writes these claims as inline code spans, not links, so
validateWikiInternalLinkscannot see them.This is not hypothetical. Reconstructing this repository at
9a415b5~1-- after the domain-directory reorg in #513 and before the regeneration in #617 -- and running the new check finds 115 stale source references across all 7 pages, each with the correct destination:Every page on
mainpointed at files that did not exist for 9 commits and 4 days, and it took a full regeneration to correct.What this adds
Finalize-pass validator (
src/agent/source-citation-validator.ts), alongside the mermaid and link validators. It stamps each stale citation in place:The run never fails, and the prompt teaches the agent to repair from that comment, so the wiki self-corrects on the next update.
openwiki doctorreports the same findings without writing, adds the pages whose cited files changed since the commit recorded in.last-update.json, and exits non-zero when a reference is stale so CI can gate on it. It reads only wiki text and git, so it is fast and makes no model calls.Precision over recall
A citation is reported only when its file is missing and a file of the same name exists elsewhere. A missing file with no such twin is ambiguous: it may be a path the reader creates in their own repository (
openwiki/.langsmith.json), a runtime artifact, or a genuine deletion, and only the last is a defect. An earlier version without this gate produced exactly that false positive against the checked-in wiki. A move is self-evidencing and arrives with the correct path in hand, so every report is actionable.Candidate detection is correspondingly conservative: a code span must be slash-separated, carry a letter-initial extension, avoid traversal segments, sit outside fenced code blocks, and begin with a directory that exists at the repository root. That last gate is what excludes the model id
z-ai/glm-5.2, the repo sluglangchain-ai/openwiki, and the media typeapplication/json. Personal-mode runs are skipped entirely, since a personal wiki has no source tree.Drive-by fix
test/agent/wiki-link-validator-dogfood.test.tsresolvedrootDirtotest/instead of the repository root, so it scanned zero files and passed vacuously. It now points at the real repository (and passes), with a comment explaining why that matters.Test plan
pnpm testgreen (2347 passed)pnpm lint:checkandpnpm formatcleanafterAgentwithout failing the runopenwiki doctortests for clean and stale wikis, missing-wiki guidance, read-only behavior, and graceful degradation outside a git repository