integration: knowledge-graph reference edges (#82) - #89
Merged
Conversation
* db: raw reference-edge schema with lifecycle-safe grants (#83) Adds the reference_edges table (migration 0005) for raw, unresolved call/import edges extracted per file. Deliberately no FK to symbols -- resolution to a concrete symbol happens at query time by name-join in a later child of epic #82. FKs to repos/files only, both ON DELETE CASCADE, so the existing sweep/reconcile cascade paths in indexer/store.py cover the new table without any behavior change; their docstrings are updated to say so truthfully. Grant builders in app/db/grants.py are schema-wide and need no code change to cover the new table. * test: cover reference_edges schema, cascades, and grant lifecycle (#83) Unit: source-level tripwires for migration 0005 (revision chain, no symbols FK, no app import, no stray CREATE EXTENSION) plus an ORM-metadata tripwire (test_reference_edge_model.py) so a future models.py edit can't silently reintroduce a symbols FK or loosen a NOT NULL column. Extends the durable-core-tables tripwire in test_db_client.py. Integration: adds migrated_edges_capable, a fixture that reaches migration head on stock dev Postgres too (by pre-seeding a stub chunks table before 0004, the same idempotency guard test_0004_guard_preserves_preexisting_chunks already exercises) so the reference_edges shape/cascade/EXPLAIN/downgrade tests and both ADP same-role/different-role grant-lifecycle proofs run without a live Lakebase branch. Extends test_reconcile.py and test_store.py to seed reference_edges rows and assert they cascade through the existing sweep/reconcile paths. * docs: add reference-edges runbook and update key-file tables (#83) New docs/runbooks/reference-edges.md: schema summary, index-to-consumer mapping, and the deploy/grant-coupling section (same shape as multi-branch.md and semantic-enablement.md) with the ADP same-role/ different-role rule and the has_table_privilege verification query. Links it from README's further-reading list. Updates app/db/AGENTS.md and app/alembic/AGENTS.md key-file tables and the 0001->0005 chain. * Address review pass 1: quote-agnostic FK tripwire, re-export ReferenceEdge, assert job role has no DDL Fresh code-reviewer and security-reviewer passes both returned APPROVE with no blockers/majors; three low-cost nits/informational findings addressed: - test_0005_no_symbol_fk now matches symbols.id via regex instead of a quote-literal substring, so it survives formatter drift. - app/db/__init__.py re-exports ReferenceEdge alongside the other models. - The ADP same-role grant test now also asserts the job role's grants are DML-only (TRUNCATE raises InsufficientPrivilege), matching build_job_grants' least-privilege intent.
This was referenced Jul 23, 2026
…on (#84) (#91) extract_file() extends the existing tree-sitter symbol walk in indexer/symbols.py to also emit call/import reference edges in the same single pass (indexer/languages.py's new EDGE_NODE_KINDS map, Python-only for now). Call targets resolve to the rightmost identifier of the callee; import targets are the full dotted path as written, alias-insensitive, with source-faithful relative-import and wildcard handling. Each edge attributes to the innermost named enclosing definition on the walk stack, computed in O(1) with no second walk. indexer/store.py's index_repo writes reference_edges exactly like symbols: an unconditional per-file delete followed by a bulk reinsert inside the same per-(repo, branch) transaction, so a file whose edges all vanish still sheds its stale rows. IndexCounts gains an edges count; indexer/job.py switches to extract_file and logs it. INDEX_SEMANTICS_VERSION bumps 2 -> 3 so every already-indexed branch re-indexes once to backfill reference_edges.
This was referenced Jul 23, 2026
…#92) Adds per-language call/import extractors (JS/TS/TSX share one pair; Go, Java, and Rust each get their own) and a dispatch table so extract_file resolves the pair once per file instead of hardcoding Python. Rust use declarations are handled by a recursive use-tree descent to cover scoped, grouped, nested, self, and wildcard forms; Java constructor targets resolve generic and scoped type names since object_creation_expression has no simple name field. Bumps INDEX_SEMANTICS_VERSION 3 -> 4 so every already-indexed branch re-indexes once and backfills the new languages.
This was referenced Jul 23, 2026
* search: add query-time candidate-set resolver over reference_edges (#86) Two-query resolver (edge sites, then a SQL-window-bounded candidate scan per name) that resolves raw reference_edges.target_name to ranked symbols candidates without a self-join, mirroring symbols.py's auto-correlation avoidance. Ranking is membership-preserving so ambiguity is never silently collapsed, and candidate_count always carries the true pre-cap total even when the fetch is capped. Serve-only: no migration, no extraction-time symbol FK, no indexer change. * search: add find_references/list_imports payload builders (#86) Additive app.service builders over the new resolver: find_references_payload (corpus-wide, call edges) and list_imports_payload (repo-scoped, import edges, structured repo_known miss). Repo-id-to-name resolution runs in a separate post-leg transaction, mirroring search_code_payload's _repo_name_map handling. MCP tool registration is a later child (#87). * scripts: measure reference-edge resolution distribution (#86) Offline AC4 measurement CLI reusing the resolver's own build_candidate_count_select/classify_resolution so the reported distribution agrees with the serve path by construction. call edges are the headline metric compared against the epic's baseline; import edges are reported separately as informational. * docs: document query-time reference resolution in reference-edges runbook (#86) Adds §4 covering the resolver's two-query design, candidate-set/resolution semantics, the exact-dotted-match import decision, repo_known for list_imports, branch-scoping parity with search_code/get_file, and the measurement script with a recorded distribution from self-indexing this repo's own tracked source.
This was referenced Jul 23, 2026
Add find_references and list_imports as thin FastMCP tool registrations backed only by app/service.py builders. find_references_payload is unchanged; list_imports_payload gains an additive target/direction extension (imports|imported_by) with deterministic pre-DB structured validation. Extends the MCP e2e fixture, smoke checks, docstrings, and docs for the new tools and the "what tests cover symbol X" composition pattern.
This was referenced Jul 23, 2026
* webui: add /api/references and /api/imports passthrough routes (#88) Thin passthrough routes over the existing app.service find_references_payload/ list_imports_payload builders -- the same builders the MCP find_references/list_imports tools wrap. DataError is the only exception mapped to 400; every other recoverable condition (query_too_broad, unsupported_direction, missing_repo, missing_target, repo_known=False, truncation) passes through as a 200 body unchanged. * webui: add Graph tab for references and imports lookups (#88) One GraphPage covering both /references and /imports modes (mode from the pathname), sharing a SiteList candidate-set renderer -- both routes' sites use the identical _site_payload shape. App.tsx mounts GraphPage with key={route.mode} so switching modes fully remounts the page and re-arms the mount-time auto-run guard. Entry points: a refs link next to each symbol match in search results, and an imports link on each repo row. * webui: rebuild frontend dist for the Graph tab (#88) make webui-build after the router/App/GraphPage/SiteList/client changes. * test: add webui graph parity integration test (#88) Extracts seed_reference_corpus out of test_mcp_server.py::seeded_schema as a pure, behavior-preserving, connection-parameterized helper, then reuses it in a new parity suite that proves /api/references and /api/imports return payloads byte-identical to a direct app.service builder call (clamp_limit applied identically) over the same seeded corpus. Transitively, since app/main.py's MCP tools are pure wrappers around the same builders (independently pinned by test_reference_tools_streamable_http), this proves webui and MCP serve identical payloads. * docs: document webui graph tools and fix stale INDEX_SEMANTICS_VERSION notes (#88) README and docs/runbooks/webui.md document the new Graph tab/routes and their parity contract with the MCP tools. docs/runbooks/reference-edges.md and app/db/AGENTS.md corrected from a stale INDEX_SEMANTICS_VERSION=3 to the current 4 (bumped 3->4 by #85).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Knowledge-graph reference edges — integration branch
This draft integration PR tracks the staged delivery of #82. Child PRs may merge into
integration/knowledge-graph-reference-edges; this PR remains draft until the complete epic is integrated, verified, and ready for human review intomaster.Planned delivery
Guardrails
symbol_idforeign keys.masterremains protected and is never agent-merged.Integration exit criteria