Skip to content

Commit b77bf49

Browse files
Wolfvinworker-1
andauthored
feat(graph): true graph data model (nodes + edges) — closes #8 (#24)
Phase 1 Step 1: add graph_nodes + graph_edges SQLite tables alongside flat registry (non-breaking). Populate during full scan. Migrate trace engine as pilot with --use-graph/--no-graph A/B flag. 20 new tests pass, A/B parity verified on clean_app fixture. Schema: - graph_nodes(id, node_id, file:line:fn, node_type, name, file, line, extra_json) - graph_edges(source_id, target_id, edge_type, file, line, confidence, extra_json) - 6 indexes on lookup paths - CALLS edges only in pilot; IMPORTS/DEFINES/INHERITS/IMPLEMENTS/USES_TYPE reserved Co-authored-by: worker-1 <worker@codeassistant.local>
1 parent 2715492 commit b77bf49

9 files changed

Lines changed: 1791 additions & 35 deletions

File tree

CHANGELOG.md

Lines changed: 88 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,96 @@
22

33
All notable changes to CodeLens will be documented in this file.
44

5-
The format is based on [Keep a Changelog](https://keepa.changelog.com/en/1.1.0/),
5+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0/html).
77

8+
## [8.2.0] — Unreleased
9+
10+
### Graph Data Model (issue #8)
11+
12+
Replaces the ad-hoc flat-registry graph traversal with a true node + edge graph
13+
backed by SQLite. This unblocks structural queries like "who calls this function
14+
across the entire codebase", "blast radius if I rename this class", and
15+
"circular dependency chains" — engines no longer need to reimplement partial
16+
graph traversal logic.
17+
18+
### Added
19+
20+
- **`scripts/graph_model.py`** — New module implementing the graph data model:
21+
- `init_graph_schema(conn)` — Creates `graph_nodes` + `graph_edges` tables
22+
and 6 indexes (idempotent, called during database initialization).
23+
- `populate_graph_tables(workspace, db_path)` — Reads the flat backend
24+
registry and bulk-inserts all nodes + edges in a single transaction.
25+
Clears stale rows first so re-scans don't duplicate.
26+
- `query_callers(node_id, db_path, max_depth=1)` — BFS over CALLS edges
27+
in reverse (who calls this node).
28+
- `query_callees(node_id, db_path, max_depth=1)` — BFS over CALLS edges
29+
forward (what this node calls).
30+
- `clear_graph_tables(db_path)` — DELETE FROM both tables.
31+
- `find_nodes_by_name`, `graph_tables_exist`, `graph_tables_populated`,
32+
`graph_stats` — introspection helpers for engines and tests.
33+
34+
- **Graph schema** (additive, prefixed `graph_` to avoid collisions):
35+
```sql
36+
graph_nodes(id, node_id UNIQUE, node_type, name, file, line, extra_json)
37+
graph_edges(id, source_id, target_id, edge_type, file, line,
38+
confidence, extra_json)
39+
```
40+
Node types: `function|class|file|module|route|type|interface`
41+
Edge types: `CALLS|IMPORTS|DEFINES|INHERITS|IMPLEMENTS|USES_TYPE`
42+
(Only `CALLS` is populated in v8.2; other types are reserved for future
43+
engine migrations — `impact`, `circular`, `dependents`.)
44+
45+
- **`trace --use-graph` / `--no-graph` flags** — The `trace` command now
46+
queries the graph tables by default, with the flat-registry path retained
47+
as fallback. Use `--no-graph` to force the flat path for A/B testing.
48+
49+
- **`tests/test_graph_model.py`** — 20 test cases covering schema init,
50+
population, query_callers, query_callees, re-population idempotency, and
51+
the trace pilot A/B comparison.
52+
53+
### Changed
54+
55+
- **`scripts/persistent_registry.py`** — Calls `init_graph_schema(conn)`
56+
during `_init_schema` so the graph tables always exist by the time any
57+
engine tries to query them. Additive — existing tables untouched.
58+
- **`scripts/commands/scan.py`** — After the flat backend registry is built,
59+
calls `populate_graph_tables(workspace, db_path)` to populate the graph
60+
tables in a single bulk transaction. Scan output now includes a `graph`
61+
field with node + edge counts.
62+
- **`scripts/trace_engine.py`** — Pilot engine migration: `trace_symbol` is
63+
now a dispatcher that picks between `trace_via_graph` (default) and
64+
`trace_via_flat` (fallback). Falls back to flat automatically when graph
65+
tables are empty (pre-8.2 databases). Output shape is identical regardless
66+
of backend — callers and formatters don't need to know which backend ran.
67+
68+
### Non-Breaking
69+
70+
- All 56 existing CLI commands continue to work unchanged.
71+
- Existing flat tables (`symbols`, `refs`, `files`, `analysis_cache`,
72+
`scan_metadata`) and JSON registries (`frontend.json`, `backend.json`)
73+
are untouched.
74+
- The graph tables are additive — no existing table or column was modified.
75+
- Scan performance impact is negligible (single bulk INSERT in one
76+
transaction; <5ms on the clean_app fixture with 31 nodes + 97 edges).
77+
78+
### Migration Notes for Engine Authors
79+
80+
The flat registry remains the source of truth during scan. The graph tables
81+
are a derived projection that engines can query for structural traversals.
82+
To migrate an engine to the graph backend:
83+
84+
1. Check `graph_model.graph_tables_populated(db_path)` — if False, fall back
85+
to the flat path (don't hard-fail).
86+
2. Use `graph_model.find_nodes_by_name(name, db_path)` to find start nodes.
87+
3. Use `graph_model.query_callers` / `query_callees` for BFS traversal.
88+
4. Preserve the existing flat-path output shape so callers and formatters
89+
don't break. See `trace_engine.trace_via_graph` for a reference impl.
90+
91+
Future engine migrations (post-v8.2): `impact`, `circular`, `dependents`.
92+
93+
---
94+
895
## [8.1.0] — 2026-06-13
996

1097
### F1 Benchmark Improvements

README.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
1-
# CodeLens v8.1 — AI-Native Code Intelligence
1+
# CodeLens v8.2 — AI-Native Code Intelligence
22

33
> **Before an AI writes a new class/id/function, CodeLens must be checked. This is not optional.**
44
5-
CodeLens is an AI-native code intelligence platform that gives AI agents **full visibility** into a codebase before they write any code. It prevents collision, overwrite of existing logic, security vulnerabilities, and dead code through 56 CLI commands, an MCP server with 54 tools (49 static + 5 dynamic), AST-based taint analysis, live CVE/OSV scanning, and a plugin system with OWASP Top 10 + Compliance rule packs.
5+
CodeLens is an AI-native code intelligence platform that gives AI agents **full visibility** into a codebase before they write any code. It prevents collision, overwrite of existing logic, security vulnerabilities, and dead code through 56 CLI commands, an MCP server with 54 tools (49 static + 5 dynamic), AST-based taint analysis, live CVE/OSV scanning, a plugin system with OWASP Top 10 + Compliance rule packs, and a true graph data model (nodes + edges) for structural code queries.
66

77
## Features
88

@@ -11,6 +11,7 @@ CodeLens is an AI-native code intelligence platform that gives AI agents **full
1111
- **AST Taint Engine** — Tree-sitter based taint analysis with return-value propagation, scope hierarchy, and branch condition refinement
1212
- **Live CVE/OSV Scanning** — Real-time vulnerability data from OSV.dev API with SQLite cache, 9 ecosystems (PyPI, npm, crates.io, Go, Maven, NuGet, RubyGems, Pub, Hex)
1313
- **Cross-File Call Graph** — Workspace-wide call graph with import resolution and bidirectional taint propagation
14+
- **Graph Data Model (v8.2)** — True node + edge graph (`graph_nodes` + `graph_edges` SQLite tables) for structural queries: callers, callees, blast radius, circular chains. Populated during scan; `trace` engine migrated to use it by default with `--use-graph` / `--no-graph` flag for A/B testing
1415
- **Plugin System** — 4 plugin types (rule_pack/engine/formatter/command), 3-tier discovery (local → user → built-in), OWASP Top 10 (36 rules) + Compliance (53 rules: PCI-DSS v4.0 + HIPAA)
1516
- **VS Code Extension** — Diagnostics Provider, Code Actions, Guard hooks, Health status bar
1617
- **CI/CD Integration** — GitHub Actions workflows, SARIF v2.1.0 output, PR decoration, `check` quality-gate command
@@ -227,6 +228,7 @@ codelens/
227228
│ ├── framework_detect.py # Framework auto-detection
228229
│ ├── incremental.py # Incremental scan support
229230
│ ├── edge_resolver.py # Cross-file edge resolution
231+
│ ├── graph_model.py # Graph data model (nodes + edges) — issue #8
230232
│ ├── search_engine.py # Regex code search
231233
│ ├── trace_engine.py # Call chain tracing
232234
│ ├── impact_engine.py # Change impact analysis

references/agent-integration.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1169,6 +1169,7 @@ workspace/
11691169
codelens.config.json ← Configuration
11701170
frontend.json ← Frontend registry (classes + ids)
11711171
backend.json ← Backend registry (nodes + edges)
1172+
codelens.db ← SQLite: symbols/refs/files/analysis_cache + graph_nodes/graph_edges (v8.2+)
11721173
mtimes.json ← File modification times cache
11731174
```
11741175

@@ -1228,6 +1229,51 @@ def quick_lookup(workspace, name):
12281229
return results
12291230
```
12301231

1232+
### 10.4 Graph Data Model (v8.2+)
1233+
1234+
For structural queries (callers, callees, blast radius, circular chains),
1235+
agents should prefer the graph data model over iterating the flat registry.
1236+
The graph tables (`graph_nodes` + `graph_edges`) are populated automatically
1237+
during `scan` and live in `.codelens/codelens.db` alongside the existing
1238+
SQLite tables.
1239+
1240+
```python
1241+
from graph_model import (
1242+
find_nodes_by_name, query_callers, query_callees,
1243+
graph_tables_populated, graph_stats,
1244+
)
1245+
1246+
db_path = os.path.join(workspace, '.codelens', 'codelens.db')
1247+
1248+
# Always check population first — pre-8.2 dbs won't have graph data.
1249+
if not graph_tables_populated(db_path):
1250+
# Fall back to flat registry iteration (see 10.3 above).
1251+
pass
1252+
1253+
# Find a function node by name (case-insensitive + fuzzy match).
1254+
nodes = find_nodes_by_name('my_function', db_path)
1255+
1256+
# Who calls this function? (BFS over CALLS edges in reverse)
1257+
callers = query_callers(nodes[0]['node_id'], db_path, max_depth=3)
1258+
1259+
# What does this function call? (BFS over CALLS edges forward)
1260+
callees = query_callees(nodes[0]['node_id'], db_path, max_depth=3)
1261+
```
1262+
1263+
**When to use the graph vs the flat registry:**
1264+
1265+
| Use case | Backend |
1266+
|----------|---------|
1267+
| Structural traversal (callers/callees/cycles/blast-radius) | **Graph** (`graph_model`) |
1268+
| Single-symbol lookup (does `foo` exist?) | Flat (`backend.json`) |
1269+
| Frontend class/id reference lookup | Flat (`frontend.json`) |
1270+
| Cross-language edge resolution (Tauri IPC) | Flat (edge_resolver) |
1271+
1272+
The graph is a derived projection of the flat registry — it is rebuilt from
1273+
`backend.json` on every scan. Engines that need the original node metadata
1274+
(`status`, `async`, `impl_for`, `component`) can read it from the graph
1275+
node's `extra` field, which preserves all non-id/fn/type/file/line fields.
1276+
12311277
---
12321278

12331279
## 11. Multi-Agent Coordination

scripts/commands/scan.py

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -905,6 +905,22 @@ def cmd_scan(workspace: str, incremental: bool = False, plugins: Optional[list]
905905
}
906906
save_backend_registry(workspace, backend_registry)
907907

908+
# ─── Graph Data Model Population (issue #8) ─────────────────
909+
# After the flat backend registry is built, populate the graph_nodes +
910+
# graph_edges tables from it in a single bulk transaction. This is
911+
# additive and non-breaking — the flat registry remains the source of
912+
# truth; the graph tables are a derived projection that engines can
913+
# query for structural traversals (callers/callees/cycles/blast-radius).
914+
# Failures here MUST NOT break the scan — the graph is an optimization
915+
# layer; engines fall back to the flat registry if it's missing.
916+
graph_population = {"nodes": 0, "edges": 0}
917+
try:
918+
from graph_model import populate_graph_tables, _default_db_path
919+
db_path = _default_db_path(workspace)
920+
graph_population = populate_graph_tables(workspace, db_path)
921+
except Exception:
922+
logger.warning("Graph table population failed", exc_info=True)
923+
908924
# Update mtimes cache
909925
all_files = []
910926
for file_list in files.values():
@@ -998,6 +1014,10 @@ def cmd_scan(workspace: str, incremental: bool = False, plugins: Optional[list]
9981014
"changed_files_count": len(changed_files) if changed_files else 0,
9991015
"unsupported_langs": fw.get("unsupported_langs", []) if fw else [],
10001016
"lang_note": _build_lang_note(fw) if fw else None,
1017+
"graph": {
1018+
"nodes": graph_population.get("nodes", 0),
1019+
"edges": graph_population.get("edges", 0),
1020+
},
10011021
}
10021022

10031023
# Add plugin rules data if plugins were requested

scripts/commands/trace.py

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55

66

77
def add_args(parser):
8+
"""Add trace-specific arguments to the parser."""
89
parser.add_argument("name", help="Symbol name to trace")
910
parser.add_argument("workspace", nargs="?", default=None,
1011
help="Path to workspace root (auto-detected if omitted)")
@@ -15,15 +16,36 @@ def add_args(parser):
1516
help="Domain to trace")
1617
parser.add_argument("--max-results", type=int, default=MAX_CHAIN_RESULTS,
1718
help=f"Max chain entries to return (default {MAX_CHAIN_RESULTS})")
19+
# v8.2 (issue #8): toggle between the new graph backend (default) and the
20+
# legacy flat-registry backend. Default is graph with automatic fallback
21+
# to flat when the graph tables are empty. Use --no-graph to force the
22+
# flat path for A/B comparison.
23+
parser.add_argument(
24+
"--use-graph",
25+
dest="use_graph",
26+
action="store_true",
27+
default=True,
28+
help="Use the graph_nodes/graph_edges backend (default). "
29+
"Falls back to flat registry if graph tables are empty.",
30+
)
31+
parser.add_argument(
32+
"--no-graph",
33+
dest="use_graph",
34+
action="store_false",
35+
help="Force the legacy flat-registry backend (A/B testing).",
36+
)
1837

1938

2039
def execute(args, workspace):
40+
"""Execute the trace command."""
41+
use_graph = getattr(args, "use_graph", True)
2142
return trace_symbol(
2243
args.name, workspace,
2344
direction=args.direction,
2445
max_depth=args.depth,
2546
domain=args.domain,
26-
max_results=args.max_results
47+
max_results=args.max_results,
48+
use_graph=use_graph,
2749
)
2850

2951

0 commit comments

Comments
 (0)