Skip to content

feat: R2 — honest retrieval metadata on every recall - #182

Merged
kevintseng merged 2 commits into
mainfrom
feat/r2-honest-retrieval
Aug 17, 2026
Merged

feat: R2 — honest retrieval metadata on every recall#182
kevintseng merged 2 commits into
mainfrom
feat/r2-honest-retrieval

Conversation

@kevintseng

Copy link
Copy Markdown
Contributor

R2 — honest retrieval metadata

Every recall now says how it was answered. The silent shape this removes: embeddings configured, sqlite-vec missing or the provider down → recall silently served keyword-only results with nothing in the response saying so; and a limit-full window was indistinguishable from a complete answer.

The retrieval block

Carried on every recall envelope — MCP, HTTP and memesh recall --json alike:

{ "entities": [...], "retrieval": { "mode": "hybrid", "degraded": false, "truncated": false } }
  • modehybrid when the vector supplement actually ran; fts when the answer is keyword-only (by configuration, or no searchable query).
  • degradedtrue means embeddings ARE configured but the vector side could not run right now (provider failure or missing sqlite-vec). Keyword-only-by-configuration is NOT degraded — that distinction is the point.
  • truncatedtrue means results filled limit; more may exist. The difference between "that is all" and "that is all I was allowed to return".

Mechanics: supplementWithVectors reports its outcome (used / unconfigured / degraded) instead of returning void with a silent catch; recallEnhanced returns {entities, retrieval} and recallWithConflicts threads it through, so the three transports cannot drift.

Surface changes

  • CLI human output: a degraded warning line (pointing at memesh doctor) and a (limit reached — more may exist) note.
  • memesh recall --json now always prints the object envelope {entities, retrieval, conflicts?} — previously a bare array normally and an object only when conflicts existed. MCP and HTTP abandoned that bimodal shape in v4.5.x (it broke Gemini CLI); the CLI now matches. Called out in the CHANGELOG. Repo-wide grep found no programmatic consumer of the bare-array shape (docs examples feed LLMs, not parsers).
  • AGENTS.md + the memesh skill teach agents to read retrieval instead of guessing — the natural counterpart of the anti-hallucination clauses shipped in docs(agents): what an agent must never invent #180.

Verification (run in this session, verdicts by exit code)

node scripts/run-tests-isolated.mjs   exit=0   Test Files 147 passed (147) / Tests 2159 passed (2159); 0 "Errors" lines
npm run verify:release                exit=0   (lint, typecheck, version-coherence, generated-mirror, doc-claims, verification-audit, audit:prod all green)

Break-test against the committed baseline: mutating the vector-failure catch back to the old silent swallow (return 'used') turns tests/recall-retrieval-meta.test.ts red (1 failed, exit 1); restoring turns it green (6 passed, exit 0).

The first full-suite run itself caught two real gaps, both fixed in the last commit: the one remaining bare-array consumer of recall --json (flag-validation's merge-strategy pin), and dist/skills-manifest.json lagging the SKILL.md edit (the manifest hashes skill files, not only hooks).

…n every envelope

The silent shape this removes: embeddings configured, sqlite-vec missing
or the provider down → recall silently served keyword-only results with
nothing in the response saying so, and a limit-full window was
indistinguishable from a complete answer.

Every recall envelope (MCP, HTTP, CLI --json) now carries
`retrieval: { mode, degraded, truncated }`:

- mode: 'hybrid' when the vector supplement actually ran, 'fts' when the
  answer is keyword-only (by configuration, or no searchable query).
- degraded: true = embeddings ARE configured but the vector side could
  not run right now — supplementWithVectors reports its outcome
  ('used' | 'unconfigured' | 'degraded') instead of returning void with
  a silent catch. Unconfigured keyword-only is NOT degraded: that is the
  configured behaviour.
- truncated: true = results filled `limit`; more may exist. A small hit
  count is a window, not a graph-wide count.

recallEnhanced returns {entities, retrieval} (two test files updated to
the new shape); recallWithConflicts threads it through, so the three
transports cannot drift. CLI human output prints a degraded warning and
a "(limit reached — more may exist)" note; CLI --json now always prints
the object envelope {entities, retrieval, conflicts?} — the bimodal
bare-array-or-object shape MCP and HTTP already abandoned.

Docs: API_REFERENCE recall section documents the block; AGENTS.md and
the memesh skill teach agents to read it instead of guessing; CHANGELOG
notes the CLI --json shape change. Audit baseline: 4 pure line-shift
re-keys (statements verified unchanged).

Verification (this session):
- vitest recall-retrieval-meta + tools + http + cli/recall-presentation
  + recall-hits-instrumentation + recall-quality: 119 passed, exit 0
- npm run typecheck exit 0; npm run build exit 0
- check-doc-claims exit 0; verification-audit exit 0
…skills manifest

The full-suite run caught the one bare-array consumer the survey missed
(tests/cli/flag-validation.test.ts merge-strategy pin), and
verify:release caught dist/skills-manifest.json lagging the memesh
SKILL.md edit — the manifest hashes skill files, not only hooks.

[Verified-By: npx vitest run tests/cli/flag-validation.test.ts exit=0 — 'Tests 15 passed (15)'; npm run build exit=0]
@kevintseng
kevintseng merged commit 7fc6d02 into main Aug 17, 2026
23 of 24 checks passed
@kevintseng
kevintseng deleted the feat/r2-honest-retrieval branch August 17, 2026 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant