indexer: extend typed reference edges to JS/TS/TSX/Go/Java/Rust (#85) - #92
Merged
IceRhymers merged 1 commit intoJul 23, 2026
Conversation
Adds per-language call/import extractors (JS/TS/TSX share one pair; Go, Java, and Rust each get their own) and a dispatch table so extract_file resolves the pair once per file instead of hardcoding Python. Rust use declarations are handled by a recursive use-tree descent to cover scoped, grouped, nested, self, and wildcard forms; Java constructor targets resolve generic and scoped type names since object_creation_expression has no simple name field. Bumps INDEX_SEMANTICS_VERSION 3 -> 4 so every already-indexed branch re-indexes once and backfills the new languages.
IceRhymers
merged commit Jul 23, 2026
0ff53f4
into
integration/knowledge-graph-reference-edges
4 checks passed
This was referenced Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part of #82 (umbrella #89, still draft). Implements #85: extends the typed call/import reference-edge extraction landed for Python in #84 (PR #91) to the remaining six supported languages — JavaScript, TypeScript, TSX, Go, Java, Rust — so all 7
SYMBOL_KINDSlanguages now emitreference_edgesfrom the same single-parse/single-walk pipeline.Caveat on
Closes #85: this covers the issue's three acceptance checkboxes for all 7 languages. GitHub will auto-close #85 on merge; if the issue should stay open until the full epic (#82) lands, strip the closing keyword before merge.What changed
indexer/languages.py— six newEDGE_NODE_KINDSentries: JS/TS/TSX (call_expression,new_expression→call;import_statement→import), Go (call_expression→call;import_spec→import, mapped per-spec rather than per-declaration so grouped imports get per-spec line anchors), Java (method_invocation,object_creation_expression→call;import_declaration→import), Rust (call_expression→call;use_declaration→import).indexer/symbols.py— one_<lang>_call_edge/_<lang>_import_edgespair per language (JS/TS/TSX share one pair — identical grammar shapes), plus_EDGE_EXTRACTORS, adict[str, tuple[call_fn, import_fn]]dispatch table.extract_fileresolves the pair once per file (not per node) right after the combined symbol/edge map lookup, replacing the hardcoded Python call sites.member_expression.property(a.b.f()/obj?.m()→f/m; optional chaining is transparent); Goselector_expression.field; Javamethod_invocation.name(object/type-argument-independent) plus a dedicated_java_type_namehelper forobject_creation_expression, since that node has no simple name field — it descends throughgeneric_typeand takes the rightmosttype_identifierof ascoped_type_identifier(new java.util.ArrayList<String>()→ArrayList); Rustscoped_identifier.name/field_expression.field.import type, TSimport x = require(...)), each anchored per-specifier where the grammar allows it. Go per-import_specanchoring handles single and grouped forms, including dot/blank aliases (target stays the package path) and emptyimport ()(zero edges, no special-casing). Java covers plain/static/wildcard forms. Rustuseis handled by_rust_use_tree_edges, a recursive descent over the use-tree (scoped, grouped, nested groups, prefix-less bare groups,self, wildcard,as-renamed) that accumulates a::-joined prefix and anchors each emitted edge at its own leaf node.macro_invocation(Rust), dynamicimport(...)(JS/TS/TSX), andexport ... fromre-exports are deliberately unmapped/skipped — not calls or imports in this schema.app/db/models.py—INDEX_SEMANTICS_VERSIONbumps3 -> 4with a docstring entry: every already-indexed branch re-indexes once on its next run to backfill edges for the six new languages.tests/unit/test_edges.py— per-language shape fixtures (calls: bare/dotted/optional-chain/constructor/generic/static/macro-negative; imports: every specifier form per grammar, incl. negatives — re-export, dynamic import, empty Go import group, Rust macro) and enclosing-attribution fixtures (function/method/class/module scope per language). Split the old combined Python/JS smoke test into alang=Nonecase and a real JS positive fixture.tests/unit/test_languages.py— three mandatory parity/invariant guards: everySYMBOL_KINDSlanguage has anEDGE_NODE_KINDSentry, everySYMBOL_KINDSlanguage has an_EDGE_EXTRACTORSentry (a missing one is a runtimeKeyErrorfor every file of that language), and the symbol/edge node-type sets are disjoint per language (guards the lossless-merge assumption_combined_kindsdepends on).Out of scope (per the epic's binding plan)
indexer/store.py(writesFileExtraction.edgesgenerically already), DB schema/migrations, resolver (#86), MCP tools (#87), Web UI, lexical query grammar/SYMBOL_KINDS, grants.Grammar-shape verification
Every node-type/field-name assumption below (
constructorvsfunction, Go'sinterpreted_string_literal_content, Rust'spath/list/argumentfields, Java's fieldlessscoped_type_identifier, TS'simport_require_clause.source, etc.) was verified against livetree-sitter-language-pack==1.13.3parse trees before being coded, not assumed from grammar familiarity — every fixture in the table below passed on the first attempt.Test plan
make lint— ruff check + format --check + mypy (app, indexer, webui) — clean.make test(pytest -m "unit or observability") — 987 passed (previously 984; +3 from a fix-round described below), including 96 tests acrosstest_edges.py/test_languages.py.make test-integration— deferred to CI, not run locally. This project is Lakebase-only (no local/CI Postgres image; seeREADME.md"Local development" anddocs/runbooks/ci-lakebase.md), and the integration suite needs a live ephemeral Lakebase branch viascripts/ci_branch.py. Checkedci-lakebase.yml's run history on the last several related PRs ([Epic] Knowledge graph: typed reference edges in Lakebase with agent-facing graph tools #82 umbrella, db: add raw reference-edge schema with lifecycle-safe grants #83, indexer: emit typed call/import edges with enclosing-symbol attribution (Python) #84) — every run isskipped, becausevars.CI_LAKEBASE_ENABLEDisn't provisioned yet (a pre-existing gap, unrelated to this change; the prerequisites indocs/runbooks/ci-lakebase.mdare still unmet). This diff touches no schema/store/DB-write code — only extraction — so the risk this defers is low, but it is a real gap the CI Lakebase job should close once provisioned.Independent pre-PR code review (separate agent pass, not self-approved): one MEDIUM finding, fixed.
_rust_use_tree_edgesdidn't handle a bare, prefix-lessuse_listargument (use {std::io, std::fmt};— valid per the Rust reference grammar'sUseTree production, distinct fromscoped_use_list), silently dropping to zero edges. Added the missing branch plus a regression fixture (test_rust_use_bare_prefix_less_group). Two non-blocking observations were also addressed: addedtest_rust_import_attributes_to_enclosing_function(Rustusecan appear inside a function body, unlike JS/Go/Java's file-top-level-only imports) andtest_javascript_default_and_namespace_import_both_target_the_module, which documents (rather than "fixes") thatimport d, * as ns from 'm'emits two identical module-target edges — both clauses are module-class bindings with no name-class target to disambiguate, so this is intentional, not a dedup bug.JS/TS-heavy benchmark (A9, manual timing, not committed — per indexer: emit typed call/import edges with enclosing-symbol attribution (Python) #84's precedent, no repo benchmark harness):
extract_file(with edges) vs a walk-only baseline (edge branch stubbed, same tree walk and symbol extraction), best-of-5time.perf_counter, over real corpora per language (this repo'swebui/frontend/srcfor TS/TSX — 21 files, 2654 lines; Go/Java/Rust standard-library sources —/usr/lib/go/src, the JDK'ssrc.zip, andrustup component add rust-src'slibrary/— since this repo has no Go/Java/Rust source of its own; JS shares the TS/TSX code path exactly, so no separate corpus was needed):All languages, and the JS/TS delta specifically (the one the epic's ≤~15% budget is keyed to), land comfortably under budget — consistent with the analytic argument that added cost is a bounded number of
child_by_field_name/text.decodecalls per edge node, not a change to the walk itself.Acceptance-criteria mapping (issue #85)
EDGE_NODE_KINDSentries for the 6 new languages + existing Python;test_edges.pyshape fixtures per language; enclosing-attribution fixtures per language (function/method/class/module scope)SYMBOL_KINDS/edge-map coveragetest_languages.py::test_every_symbol_language_has_an_edge_map,::test_every_symbol_language_has_an_edge_extractor,::test_symbol_and_edge_node_types_are_disjoint_per_languagetest_edges.pyper-language shape matrix (bare/dotted/optional-chain/constructor/generic calls; every import specifier form per grammar) plus negative fixtures (dynamic import, re-export, empty Go import group, Rust macro, bare prefix-lessusegroup)Not merging — left ready for orchestrator CI verification and merge.