feat(health): surface mentionRate on the health snapshot#710
Merged
Conversation
cnry health was the last citation-only read surface; mention is already live on overview/report/analytics. Add mentionRate alongside citedRate so health/dashboard/overviews can lead with mention while keeping cited. The two signals stay independent: mention counts a (query x provider) pair only when answer_mentioned === true (tri-state; null = "not checked" and is never coerced to false). buildRunData now threads query_snapshots.answer_ mentioned through computeHealth, which emits overallMentionRate + mentioned Pairs + per-provider mentionRate beside the untouched cited fields. - intelligence: Snapshot.answerMentioned + HealthScore mention fields; compute - canonry: buildRunData SELECT + persist the new columns; CLI renders mention first - db: nullable overall_mention_rate / mentioned_pairs columns; idempotent migration v80 (guarded run, table+column existence checks) - contracts: additive HealthSnapshotDto fields (public SDK contract) - api-routes: map/aggregate/empty health rows; coalesce legacy NULL -> 0 Additive contract, idempotent migration, legacy rows read back as 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add
mentionRate(answer-text mention signal) to the health snapshot path socnry healthleads with mention rate alongside cited rate.cnry healthwas the last citation-only read surface; mention is already live on overview/report/analytics.Scope:
overallMentionRate+mentionedPairs+ per-providermentionRate, added beside the untouched cited fields.Why
So health/dashboard/overviews can lead with mention while still showing cited. The two signals are independent and must never be derived from one another.
How
answer_mentioned === true. The field is tri-state (true / false / null);nullmeans "not checked" and is never coerced to false in logic. Legacy persisted rows (no mention columns) coalesce NULL→0 on read.buildRunDatanow threadsquery_snapshots.answer_mentioned(previously dropped) intocomputeHealth, which emits the mention metrics.overall_mention_rate/mentioned_pairsonhealth_snapshots, added by an idempotent migration (v80, guarded by table + column existence checks).HealthSnapshotDtogains the mention fields (additive only — no rename ofoverallCitedRate).--historytable.Additive contract, idempotent migration, legacy rows read back as 0. Minor version bump (4.83.0 → 4.84.0) per the Versioning rule (additive, multi-package, >100 lines).
Tests
intelligence/health.test.ts: explicit mention math (numerator/denominator/rounded), plus anull/undefinedcase proving null is not counted as mentioned.api-routes/health-latest.test.ts: ready-snapshot mention fields, aggregate summingmentionedPairs+ merging per-providermentionRate, and a legacy NULL-column row coalescing to 0 without crashing.canonry: health JSONL + human-render tests (mention leads), and ananalyzeAndPersistround-trip threadinganswerMentionedthrough to the persisted columns.db: round-trip of the new columns + a legacy NULL-column row.🤖 Generated with Claude Code