Skip to content

ADR-0001 D2's mention payoff never reaches recall(): a graph candidate cannot survive the corpus quota #21

Description

@wfaithfull

ADR-0001 D2 is the argument for the whole corpus↔graph join:

a retrieved chat fact seeds an entity which pulls in the Helios architecture doc even when the
query wording matches that document not at all
. That is the multi-hop win.

Every part of that now exists — entity_mentions populates, pgkg maintain --task mentions runs
it, and pgkg_graph_candidates emits the document. The last step does not happen: the candidate
never survives to the caller, at any corpus size.

Mechanism, located

Two independent effects meet, which is why there is no corpus size where it works.

Above ~38 competing passages. A graph candidate scores w_graph (0.5) × MIN(seed fused score)
— migration 010's pgkg_fuse, the "neighbour floor" 043 names — so it is by construction below
every keyword and vector candidate. pgkg_apply_quotas then keeps only
floor(k_rerank × corpus_fraction) = floor(64 × 0.6) = 38 corpus items. So the
mention-added passage is cut before the cross-encoder, which is the only stage that could
promote it on merit.

Below that size. The passage is inside the arms' own k_initial = 200 candidate list, which
makes it a seed — and 043's per_seed mention branch excludes seeds from expansion with
NOT EXISTS. So it gets no graph contribution at all.

Measured

  • 11 items: recall() scores byte-identical with the entity_mentions rows present and deleted.
  • 310 items: the passage is absent from recall() entirely.
  • pgkg_graph_candidates called directly with the real seed set does emit it, raw score
    0.003861 = MIN(seed fused score).
  • Sweeping k_rerank at 64 / 200 / 400 / 1000: the row appears only at 1000, the value at
    which the corpus quota (600) exceeds the 244 corpus candidates — i.e. only when the quota does
    no cutting at all.

Why the suite never saw it — and the second half of this issue

tests/test_entity_mentions.py::test_a_chat_fact_pulls_in_the_document_that_defines_what_it_names
tests D2 and passes. Its docstring says "D2, end to end, over the real ingest pipelines". It
cannot catch this
: it calls pgkg_retrieve with q_text and no q_embedding, so the vector
arm returns nothing, the corpus chunk is not a candidate, and it is therefore not a seed. That is
the only geometry in which a graph candidate survives, and it is not the geometry the product
uses — Memory.recall always passes an embedding.

The test is not wrong about the SQL mechanism, and its control arm (expand_graph=false) does not
help. But anyone reading it as proof that D2 works end to end is reading more than it says, and
that is how this survived a full adversarial verification pass.

So a fix has two parts, and the first without the second just rebuilds the illusion:

  1. Make a mention-derived candidate reachable through recall() at product geometry.
  2. Give that test a query embedding and a corpus larger than the candidate pool, so it exercises
    the geometry the product runs at. It becomes a slow test; that is the cost of it meaning
    something.

The decision

Three options, all migrations, all changing ranking for every query — which is why this is an
issue and not a patch:

  • The graph arm stops scoring at the seed floor. 043 chose that floor deliberately and says so,
    so this is a reversal, not an oversight.
  • pgkg_apply_quotas stops ordering purely by fused score.
  • Graph candidates get a reserved slot in the rerank budget — the same shape as D1's
    personal-memory floor, and the option I would look at first: the floor exists precisely because
    rank-by-score starves a class that matters for reasons score cannot see.

Whatever lands, the cross-encoder is the stage that can judge a mention-derived passage on merit,
so the goal is getting it there, not scoring it higher outright.

Found by the end-to-end regression loop after #16#19. The measurement is in the
_GRAPH_ARM_SQL comment in scripts/e2e_mcp.py and in CHANGELOG so it does not have to be
retaken.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions