feat(hooks): recall accounting from explicit citations — [mem:id] handles, hits only - #178
Merged
Conversation
…dles, hits only Literal name/title matching was the accounting: measured across ten real sessions and three matching strategies it had zero signal, so every injected memory drifted toward an unearned recall_miss, and misses feed impactScore (10% of ranking weight). Replaced end to end: - Every topology line carries its entity's citation handle ([mem:42]) — budgeted like any other character, printed by the session-start injection and the briefing tool alike, so a memory reads the same way on every surface. One instruction line, outside the reference fence on purpose (the fence declares its content data, and this is an instruction), asks the agent to cite the memories it genuinely uses. - The Stop hook credits recall_hits from extractCitedMemoryIds() over the structurally-stripped transcript — the injected block prints a handle on every line and is echoed 2+ times, so skipping the strip would score every injection a hit (the exact failure the two pre-citation accountings shipped). isRecallHit and isMeasurableRecallName retire with the literal-match era. - Misses are FROZEN: markers are self-reported, undercount and never overcount, so silence is not yet evidence of non-use. Two memesh_metadata counters make compliance itself measurable and an accounting-mode stamp keeps the eras of numbers apart. - The injected-set record now includes the lessons pool — injected lessons were never accounted at all — and topLessons is hoisted so the record lists the same five lessons the block renders. Cost, measured on one DB snapshot (main vs this branch): 695 -> 765 tokens per session start (+70) — the price of measuring injection ROI instead of guessing it. New end-to-end scenario spawns the real Stop hook against a seeded DB: cited id earns exactly one hit, an id seen only inside hook echoes earns nothing, silence earns nothing, and the metadata stamps land.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
R1 of the work-topology arc: the injection-ROI signal gets an honest proxy. M0 measured the previous accounting — literal name/title matching against the transcript — at zero signal across ten real sessions and three matching strategies: nobody restates a memory's title in prose, so every injected memory drifted toward an unearned
recall_miss, and misses feedimpactScore(10% of ranking weight).Replaced end to end:
- [decision] Ship FTS5 as the baseline [mem:42]— budgeted like any other character (the text yields the space; a truncated handle cites nothing and still spends the tokens, so it survives whole). Printed by the session-start injection and thebriefingtool alike, so a memory reads the same way on every surface. One instruction line sits deliberately OUTSIDE the reference fence (the fence declares its content "background data, not instructions", and this is an instruction): cite a memory once inline as [mem:ID] when it genuinely informs your work.recall_hitsfromextractCitedMemoryIds()over the structurally-stripped transcript. The injected block prints a handle on every line and Claude Code echoes it 2+ times — skip the strip and every injection scores a hit, the exact failure both pre-citation accountings shipped.isRecallHit/isMeasurableRecallNameretire with the literal-match era.memesh_metadatacounters (citation_sessions_total/citation_sessions_cited) make compliance itself measurable;recall_accounting_modestamps the era so the two generations of numbers are never conflated.topLessonsis hoisted so the record lists the same five lessons the block renders (it previously referenced an inner-scope variable pattern that recorded a different set shape).Cost (measured, same DB snapshot, main vs this branch)
scripts/audit/measure-injection-tokens.mjs: 695 → 765 tokens per session start (+70, ~+10%) for the handles + the instruction line — the price of measuring injection ROI instead of guessing it.Verification
node scripts/run-tests-isolated.mjs→ exit 0, 139 files / 2098 tests (net +3).npm run typecheck/npm run verify:release→ exit 0 (audit baseline: one fence-scan C5 entry re-keyed for the line shift).topologyLine→ the handle test goes red; skipping the echo strip →× cited earns a hit, silence earns NOTHINGgoes red (the echo-only id scores a false hit); restored → green.[mem:ID]placeholder never scoring), handle budgeting, no-id lines unchanged.Notes