Skip to content

search: query-time candidate-set resolver over raw edges (#86) - #93

Merged
IceRhymers merged 4 commits into
integration/knowledge-graph-reference-edgesfrom
feat/86-reference-resolver
Jul 23, 2026
Merged

search: query-time candidate-set resolver over raw edges (#86)#93
IceRhymers merged 4 commits into
integration/knowledge-graph-reference-edgesfrom
feat/86-reference-resolver

Conversation

@IceRhymers

Copy link
Copy Markdown
Owner

Summary

Part of #82 / umbrella #89. Closes #86.

Adds a query-time resolver that turns raw reference_edges rows (#83/#84/#85: unresolved call/import sites, no FK to symbols) into ranked candidate-set matches against symbols, entirely at serve time — no migration, no indexer change, no extraction-time symbol FK, no INDEX_SEMANTICS_VERSION bump. MCP tool registration (find_references/list_imports as callable tools) is a separate, later child (#87) — this PR ships the resolver module and the app.service payload builders it backs, not the tool wiring.

Caveat: since #87 hasn't landed yet, find_references_payload/list_imports_payload are not reachable from any MCP tool or the web UI in this PR — they're exercised only by the unit/integration test suites and the offline measurement script below. #87 will wire them up.

  • app/search/references.py (new): two-query resolver (edge sites, then a SQL-window-bounded candidate scan per name) — deliberately not one joined query, avoiding the same self-join auto-correlation hazard app/search/symbols.py avoids for sym:. Candidate fetch is bounded in SQL per name (ROW_NUMBER() OVER (PARTITION BY symbols.name ...), cap 32), while COUNT(*) OVER carries the TRUE pre-cap count so resolution (unique/ambiguous/unresolved) is never wrong even when the returned list is capped. Ranking (same-repo → kind-appropriate → same-file → deterministic tiebreak) is membership-preserving: ambiguity is never silently collapsed to one answer. Import edges resolve on the exact full dotted path only (no last-segment split) — a deliberate, documented decision (external/stdlib imports should read as unresolved, not manufacture false ambiguity).
  • app/service.py: additive find_references_payload (corpus-wide, call edges) and list_imports_payload (repo-required, import edges, structured repo_known miss — never a silent empty). Repo-id→name resolution runs in its own post-leg transaction, mirroring search_code_payload's _repo_name_map handling.
  • scripts/measure_reference_resolution.py (new): offline AC4 measurement CLI that reuses the resolver's own build_candidate_count_select/classify_resolution, so the reported distribution agrees with the serve path by construction rather than re-implementing the join.
  • docs/runbooks/reference-edges.md: new §4 documenting the resolver design, the import-as-external decision, repo_known semantics, branch-scoping parity with search_code/get_file, and the recorded measurement below.
  • Tests: tests/unit/test_references.py, tests/unit/test_measure_reference_resolution.py, tests/integration/test_references.py, plus extensions to tests/unit/test_service.py / tests/integration/test_service.py.

Gates (fresh run against this diff)

  • make lint (ruff check + format --check + mypy app indexer webui): clean
  • make test (unit + observability): 1028 passed
  • make test-integration: new/touched tests (test_references.py ×16, test_service.py ×19 incl. 3 new) all pass against a real local Postgres. The suite's pre-existing failures/errors (lakebase_tokenizer/lakebase_ann/lakebase_bm25 extensions absent from a vanilla Postgres image, and role-provisioning-dependent grant tests) are documented, environment-only gaps unrelated to this change — confirmed pre-existing by reproducing one (test_commit_search.py::test_scoped_commit_equals_repo_branch_query_non_default) on a clean stash of this branch.
  • Independent adversarial code review (fresh agent, plan-fidelity + correctness + security + test-quality focus): APPROVE, no CRITICAL/HIGH findings. Two LOW test-quality findings were fixed (strengthened two near-tautological SQL-shape assertions in test_references.py to actually verify no last-segment splitting and a distinct aliased second files join in the count builder).

Measurement (AC4)

Self-indexed this repo's own git-tracked source (206 files, 2,947 symbols, 15,412 reference edges across Python/JS/TS/TSX) into a local Postgres via the real indexer pipeline, then ran scripts/measure_reference_resolution.py --edge-kind both --use-resolver:

call edges -- HEADLINE AC4 metric (n=14226):
  unique         4144   29.1%   (baseline 28.8%)
  ambiguous      4208   29.6%   (baseline 33.4%)
  unresolved     5874   41.3%   (baseline 37.8%)

import edges -- informational, expected ~0% resolution (validates D3) (n=1186):
  unique           15    1.3%
  ambiguous         8    0.7%
  unresolved     1163   98.1%

The re-measured call-edge distribution tracks the prior-art baseline closely (within ~4 points on every bucket); import edges resolve at ~2% total, confirming they're overwhelmingly external/stdlib targets, matching the exact-dotted-match design (D3).

Acceptance-criteria mapping

  • AC1 (ranked candidate sets; ambiguity never collapsed): _rank_candidates + true candidate_count from COUNT(*) OVER; unit ambiguity/cap tests, integration cross-repo/same-repo-duplicate/hot-name tests.
  • AC2 (statement_timeout guard; truncation flags; deterministic ordering): shared transaction-local timeout → QueryTooBroadError; row_cap/candidates_truncated flags; ORDER BY total order ending in a PK tiebreak on both queries; integration timeout + determinism tests.
  • AC3 (branch/default-branch scoping matching search_code): shared _branch_predicate, byte-identical to get_file_payload's, applied to both the edge site's file and each candidate's file; unit byte-identical-fragment assertions + integration branch-parity tests.
  • AC4 (resolution distribution measured & recorded): scripts/measure_reference_resolution.py + the real self-indexed measurement above, recorded in docs/runbooks/reference-edges.md §4.

Test plan

  • make lint
  • make test
  • make test-integration (new/touched suites, against a real Postgres)
  • Real resolution-distribution measurement against a self-indexed corpus
  • Independent code review pass, findings fixed
  • Not merging this PR — leaving open per operator instruction

Two-query resolver (edge sites, then a SQL-window-bounded candidate scan
per name) that resolves raw reference_edges.target_name to ranked symbols
candidates without a self-join, mirroring symbols.py's auto-correlation
avoidance. Ranking is membership-preserving so ambiguity is never
silently collapsed, and candidate_count always carries the true pre-cap
total even when the fetch is capped. Serve-only: no migration, no
extraction-time symbol FK, no indexer change.
Additive app.service builders over the new resolver: find_references_payload
(corpus-wide, call edges) and list_imports_payload (repo-scoped, import
edges, structured repo_known miss). Repo-id-to-name resolution runs in a
separate post-leg transaction, mirroring search_code_payload's
_repo_name_map handling. MCP tool registration is a later child (#87).
Offline AC4 measurement CLI reusing the resolver's own
build_candidate_count_select/classify_resolution so the reported
distribution agrees with the serve path by construction. call edges are
the headline metric compared against the epic's baseline; import edges
are reported separately as informational.
…book (#86)

Adds §4 covering the resolver's two-query design, candidate-set/resolution
semantics, the exact-dotted-match import decision, repo_known for
list_imports, branch-scoping parity with search_code/get_file, and the
measurement script with a recorded distribution from self-indexing this
repo's own tracked source.
@IceRhymers
IceRhymers merged commit 548c1a1 into integration/knowledge-graph-reference-edges Jul 23, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant