Repository navigation
feat(citation): measure citation coverage and mark unsupported statements - #164
Merged
Merged
Conversation
…ents Report the one deterministic grounding property: does a sentence carry a citation that resolves to a real span. Deliberately not whether the citation proves the sentence — that is semantic entailment, and a rule that pretended to answer it would produce a wrong "grounded" verdict, which is worse than none. - classifyClaimSupport() (shared, pure) splits the answer and returns cited / uncited / invalid-citation / inference per sentence, plus coverage, unsupportedRate and invalidRate. Marker-only fragments such as "A is true. [1]" attach to the preceding sentence, because models write the marker after the period. - Sentence segmentation moves to shared/utils/sentenceSegmentation so candidates (#155) and coverage agree on what a sentence is. - The prompt asks for a citation on every factual sentence and [inference] for a claim the model adds itself; the remark plugin keeps [inference] out of the rendered body. - AnswerCoverage shows a notice only when something is unsupported; a fully cited answer gets no line. Verified: npm run typecheck; npm test (363 pass); npm run check:design; npm run build; RAG eval output byte-identical to docs/eval/baseline-v1.4.json. Scoped out: inline per-sentence marking (the notice is aggregate), and live-model behaviour. The #75 harness runs no model, so coverage is not measured there — a model-driven answer eval is a separate deliverable.
mrsibe
added
enhancement
This was referenced Sep 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Re-delivers #156 (citation coverage) onto
main.Why?
#163 was merged into
feat/155-exact-passage-citationrather thanmain, because its base was the #155 branch.origin/maincontains #161 and #162 but not #156's changes. The #155 commit on that branch is also the pre-squash SHA, so a normal PR from it would re-show #155.This branch is
origin/main+ a cherry-pick of the #156 commit, so the diff is exactly #156 and nothing else.Related issue
Fixes #156
Related to #154, #163
What changed
Same content as #163:
classifyClaimSupport()(shared, pure):cited/uncited/invalid-citation/inferenceper sentence, pluscoverage,unsupportedRate,invalidRate. Marker-only fragments attach to the preceding sentence.shared/utils/sentenceSegmentationso candidates ([Feat] Exact-passage citation: the model cites source spans, not chunks #155) and coverage agree.[inference]for the model's own claims; the remark plugin keeps[inference]out of the rendered body.AnswerCoverageshows a notice only when something is unsupported.How was this tested?
npm run typecheck— passes.npm test— 363 pass, 0 fail.npm run check:design— no violations.npm run buildand the RAG eval baseline were verified byte-identical on feat(citation): measure citation coverage and mark unsupported statements #163; this branch is a content-identical cherry-pick.Checklist
npm run typecheckpasses.npm run buildpasses.Desktop / build changes