The gap
otter-docs' highest-value promise is keeping docs honest, but doc-vs-code staleness detection is not shipped. description.divergence is disabled by default (README: the cosine distance between a symbol's description-vector and code-vector conflates real docstring staleness with "terse code + verbose description" — empirically ~0.53 on nomic-embed, untrustworthy alone). The real fix, confirm_description, is parked at v0.2.
Why it matters
A full manual doc audit of the quorum monorepo (2026-08-12) found real stale claims — TECH_STACK missing an entire machine, an icarus README omitting a shipped product, wrong counts/model versions — all of which confirm_description is designed to catch automatically. This feature is the difference between otter-docs as a metrics-dumper and otter-docs as the thing that keeps docs true.
Design (mirror confirm_redundancy)
An LLM judge that reads BOTH the doc claim and the code and rules accurate | partial | stale | wrong, with a content-addressed verdict cache in graph.db (same shape as confirm_redundancy). High-recall cosine pre-filter → high-precision LLM second pass. Extend beyond docstrings to hand-written docs (README/CLAUDE.md/specs) checked against the symbols they reference.
Acceptance
findings() (default) can surface confirmed stale/wrong doc findings with the code evidence.- Precision measured on a labeled set; verdicts cached; re-runs ~free on unchanged code.
The gap
otter-docs' highest-value promise is keeping docs honest, but doc-vs-code staleness detection is not shipped.
description.divergenceis disabled by default (README: the cosine distance between a symbol's description-vector and code-vector conflates real docstring staleness with "terse code + verbose description" — empirically ~0.53 on nomic-embed, untrustworthy alone). The real fix,confirm_description, is parked at v0.2.Why it matters
A full manual doc audit of the quorum monorepo (2026-08-12) found real stale claims — TECH_STACK missing an entire machine, an icarus README omitting a shipped product, wrong counts/model versions — all of which
confirm_descriptionis designed to catch automatically. This feature is the difference between otter-docs as a metrics-dumper and otter-docs as the thing that keeps docs true.Design (mirror confirm_redundancy)
An LLM judge that reads BOTH the doc claim and the code and rules
accurate | partial | stale | wrong, with a content-addressed verdict cache ingraph.db(same shape asconfirm_redundancy). High-recall cosine pre-filter → high-precision LLM second pass. Extend beyond docstrings to hand-written docs (README/CLAUDE.md/specs) checked against the symbols they reference.Acceptance
findings()(default) can surface confirmedstale/wrongdoc findings with the code evidence.