Skip to content

feat(search): expose per-arm participation evidence - #2321

Merged
ohdearquant merged 6 commits into
mainfrom
codex/search-arm-participation
Sep 4, 2026
Merged

feat(search): expose per-arm participation evidence#2321
ohdearquant merged 6 commits into
mainfrom
codex/search-arm-participation

Conversation

@ohdearquant

Copy link
Copy Markdown
Owner

Summary

  • add typed arm_participation evidence for the text and vector KG search arms with ran, skipped, and error status plus bounded final candidate counts
  • preserve the evidence on complete, partial, zero-match, degraded-empty, presentation, and frame-budget paths without changing filtering, fusion, or deterministic ranking
  • document exact-name presence checks and align degraded arm status with ADR-130 / fix(mcp): classify and pace backend search timeouts #2176 bounded backend cause vocabulary
  • cover exact-name text hits, long keyword-dense zero hits, vector and both-source counting, partial results, degraded-empty failures, and diagnostic omission

Verification

  • cargo test --workspace
  • cargo test -p khive-mcp --lib (449 passed)
  • cargo test -p kkernel coordinator::tests (63 passed)
  • cargo check -p kkernel --all-targets
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo fmt --all -- --check
  • git diff --check

Closes #1935

@ohdearquant

Copy link
Copy Markdown
Owner Author

Both arm statuses are derived from an undifferentiated backend error, and the vector discriminator is a config flag

The statuses are computed at crates/khive-mcp/src/server.rs:196-203:

let text_failed = result.per_backend.iter().any(|b| b.error.is_some());
let vector_failed = result.per_backend.iter().any(|b| b.vector_selected && b.error.is_some());

text_failed is "any backend errored" — it discriminates neither by arm nor by backend. And
vector_selected is not an execution fact: crates/kkernel/src/coordinator/service.rs:162-166
derives it from entry.runtime.config().embedding_model.is_some(), i.e. whether that backend is
configured with an embedding model, regardless of whether the vector arm ever ran.

Put those together in a coordinator fan-out — the case from_result exists to serve, per its own
doc comment at server.rs:160-163. Backend A succeeds, backend B fails. The response still carries
A's hits, but text is reported error, and vector is reported error as well whenever B merely
has an embedding model configured, whatever actually failed there. Relatedly, vector_selected at
server.rs:179-182 is an .any() across backends, so a single embedding-configured backend marks
the vector arm ran for a response in which other backends never ran it.

This is the failure mode the field exists to prevent. The evidence is right in the common
single-backend case and wrong exactly when a backend is degraded, which is when someone reads it.
The fix is to carry each arm's own outcome from the point that arm finishes, per backend, rather
than reconstructing it from an aggregate error plus a configuration flag — with a regression test
for a fan-out in which one backend fails.

One thing to note so the mechanism is not mis-stated: within a single backend, a vector failure
aborts the whole hybrid search (crates/khive-runtime/src/retrieval.rs:746 propagates with ?), so
the text hits are discarded rather than surviving beside a vector error. The cross-backend path
above is the reachable one.

Documented cause vocabulary does not match the code

docs/adr/ADR-130-search-response-completeness-and-ranking-evidence.md:802 and
docs/guide/api-reference.md:483 both promise a closed timeout | backend_error vocabulary. The
only production emitter, backend_errors_value at crates/khive-mcp/src/server.rs:395, hardcodes
"kind": "backend_error", and no production site emits timeout — the other occurrences in the
tree are test code. A caller cannot distinguish a timeout from any other backend failure. Either
propagate a typed cause or drop the two-value promise from both documents.

Verified and sound

  • No filtering, fusion, ordering or short-circuit behaviour changed. Evidence gathering reads an
    already-built result and cannot force an arm to run or to complete.
  • Zero is distinguishable across the three cases when the operation completes: an arm that ran and
    matched nothing is ran, 0, a skipped vector arm is skipped, 0, and a degraded selected arm is
    error, 0. An arm that ran, matched rows, and had them all filtered out reports ran, 0.
  • Evidence is attached on the success, coordinator-partial, complete-empty and incomplete paths, and
    arm_participation is in the survivor key list for frame omission (server.rs:4012-4025), so it
    survives a trimmed frame.
  • observe_result sets only the counts; it does not override the statuses.

Non-blocking notes

  • Saturation is invisible. search_arm_candidate_counts (server.rs:312-325) folds over the final
    serialized result, so each count is bounded by the caller's limit — which
    docs/guide/api-reference.md:484-486 does document. What neither documents nor marks is the
    boundary case: a count that equals limit is indistinguishable from one that was clamped there,
    and nothing in the response says which. The sibling bounded-count work in this area publishes an
    explicit saturation flag for exactly this reason; the same treatment would fit here.
  • Tests cover envelope placement, source counting, complete-empty, degraded-empty and presentation.
    They do not cover a fan-out with one failing backend, a count sitting at the limit, or an
    end-to-end frame-budget trim; the frame tests are synthetic helper tests.

No builds or tests were run for this pass; every finding above is from source reading.

@ohdearquant
ohdearquant marked this pull request as ready for review September 1, 2026 16:39

@ohdearquant ohdearquant left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review. Posted by this repository's automated pull-request review pipeline; this is not a human read and does not gate the merge by itself.

Verdict on head 821eb94: REQUEST-CHANGES, 1 blocking finding. Finding details are delivered to the review's recipients rather than posted here. Do not merge this head while blocking findings are outstanding; a pipeline comment on a newer head supersedes this one.

…strate

A backend whose vector leg fails now keeps its text hits and reports
status: "complete" with arm_participation.vector.status: "error" instead
of discarding the whole backend's contribution. This holds for the
coordinated multi-backend fan-out path on both entity and note searches;
the single-backend default path (no coordinator installed) still fails
the whole search call on a vector-arm failure and returns no
arm_participation.

The note substrate gets a fan-out outcome path mirroring the existing
entity one (search_notes_outcome, tolerating a vector-arm error while
still failing loud on a text-arm error), wired into both the
single-backend and multi-backend branches of the coordinator's fan-out
dispatch. The MCP envelope's vector-failure detection no longer depends
solely on a per-backend vector_selected flag, which could miss a
recorded vector_error; a populated vector_error is now sufficient proof
on its own. The wire-level join from the dispatch result to the
coordinator service's JSON-facing type is covered end to end, including
a mutation check on the single line that carries it.

Frame-budget omission is amended to fold arm_participation into
error.search alongside the other search diagnostics it already
relocates there, matching the documented contract. The API reference
and ADR-130 are updated to state the per-arm (not per-backend) meaning
of the error status consistently in one place, and to name where
arm_participation and backend_errors land once an oversized response is
trimmed.
…ocs and normalise emphasis

rustdoc refuses a public item whose docs link a private one; the mention
stays as plain code. Two markdown emphasis spans take the formatter's
underscore form.
The arm-participation text said backend causes use the two-value
timeout | backend_error vocabulary, but v0.8.0 emits the single constant
backend_error; the two-value form ships in v0.9.0 per the Compatibility
section. Say so in the ADR amendment and the API reference.

@ohdearquant ohdearquant left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review. Posted by this repository's automated pull-request review pipeline; this is not a human read and does not gate the merge.

Verdict on current head: APPROVE, zero blocking findings. This is a comment, not an approval — a human reviewer decides whether to approve and merge.

@oceanwaves630 oceanwaves630 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

STAMPING, not reviewing.

This approval executes the standing khive-oss stamp lane on a mechanical gate; it is not an independent code review, and it carries no judgment of my own about the change.

Gate read at head deb224e98fb085c3bb3b5d8c7c37bf346bf4ef21:

  • complete daemon verdict at this exact head (.md/.json/.process.jsonl all present)
  • decision APPROVE, degraded False
  • 0 blocking; 1 graded finding, maintainability/minor, non-blocking — read it before merging, "0 blocking" is not "no findings"
  • CI at this head: 21 passing, 0 failing, 0 pending
  • base main, open, not a draft
  • no CHANGES_REQUESTED outstanding from any account

Merge with --match-head-commit deb224e98fb085c3bb3b5d8c7c37bf346bf4ef21; if the head moves, this stamp does not carry.

@ohdearquant
ohdearquant merged commit 3e249f7 into main Sep 4, 2026
29 checks passed
@ohdearquant
ohdearquant deleted the codex/search-arm-participation branch September 4, 2026 01:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

search: FTS arm can contribute zero candidates on long keyword-dense entity queries, with no per-arm participation signal

2 participants