Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion app/db/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Database connectivity and schema truth for the code-search corpus. `client.py` i
| File | Description |
|------|-------------|
| `client.py` | `create_db_engine()`: Lakebase-endpoint-wins-over-`PGHOST` dual-mode selection; Lakebase mode closes one `WorkspaceClient` over a `do_connect` handler that mints a fresh OAuth token as the password on every physical connect (never logged); server defaults `pool_size=5`, `pool_recycle=2700` (45 min, under the ~1h token TTL), `pool_pre_ping=True`; SDK import is lazy so the local path never touches it |
| `models.py` | ORM models + `INDEX_SEMANTICS_VERSION` (currently 2; bump on any indexing-meaning change — CI tripwires it). `File` is content-deduped per (repo_id, path, content_sha) with a `branches ARRAY(Text)` membership column; `RepoBranch` is the authoritative per-(repo, branch) CAS stamp; `repos`' own stamp columns and `files.commit` are deprecated/ambiguous — never add readers. `ReferenceEdge` (0005, epic #82) is a raw unresolved call/import edge, deliberately with NO FK to `symbols` (resolution happens at query time by name-join in a later child); FKs to `repos`/`files` only, both `ON DELETE CASCADE`. Declares the trgm + branches GIN indexes so autogenerate can't drift |
| `models.py` | ORM models + `INDEX_SEMANTICS_VERSION` (currently 3; bump on any indexing-meaning change — CI tripwires it). `File` is content-deduped per (repo_id, path, content_sha) with a `branches ARRAY(Text)` membership column; `RepoBranch` is the authoritative per-(repo, branch) CAS stamp; `repos`' own stamp columns and `files.commit` are deprecated/ambiguous — never add readers. `ReferenceEdge` (0005, epic #82) is a raw unresolved call/import edge, deliberately with NO FK to `symbols` (resolution happens at query time by name-join in a later child); FKs to `repos`/`files` only, both `ON DELETE CASCADE`. Declares the trgm + branches GIN indexes so autogenerate can't drift |
| `semantic.py` | Standalone Core `Table` for `chunks` (BigInteger PK, `Vector(SEMANTIC_EMBEDDING_DIM)` embedding, generated `ts` tsvector, nullable `start_line`/`end_line`) in its own `semantic_metadata` — a typed description only; the real DDL is owned by migration `0004` |
| `grants.py` | Pure SQL-string builders `build_app_grants` (read-only) / `build_job_grants` (CRUD + sequences, no DDL); identifiers validated against `^[A-Za-z0-9_-]+$` (1..63 chars) then psycopg-quoted. Execution lives in `scripts/migrate.py` |
| `__init__.py` | Re-exports `Base`, `File`, `Repo`, `Symbol`, `create_db_engine` |
Expand Down
6 changes: 5 additions & 1 deletion app/db/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,17 @@
from sqlalchemy.dialects.postgresql import ARRAY
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship

INDEX_SEMANTICS_VERSION = 2
INDEX_SEMANTICS_VERSION = 3
"""Version of the indexing semantics the current code produces.

2: semantic search default-on -- every already-indexed branch must re-index once so
``chunks`` backfills (the skip seam compares ``(head_sha, INDEX_SEMANTICS_VERSION)``,
and without a bump a repo already at HEAD would skip forever and never get chunks).

3: reference edges -- every already-indexed branch must re-index once so
``reference_edges`` backfills; without a bump a branch at HEAD would skip forever
and never get edges.

Bump this whenever the *meaning* of what gets written changes: any change to
``indexer/symbols.py``, to ``indexer/parse.py``'s chunking, or to
``indexer/languages.py``'s extraction contract. A bump forces every repo to
Expand Down
37 changes: 32 additions & 5 deletions docs/runbooks/reference-edges.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,42 @@ joins through `files` with the same `coalesce(default_branch,'HEAD')` conjunct u
everywhere else), and `files.commit` is documented-ambiguous under multi-branch dedup and
must gain no new readers.

**This table is dormant until #84 ships a writer.** `0005` only creates the schema; no
code path inserts rows yet. `indexer/store.py`'s cascade-owning functions (`index_repo`'s
membership sweep, `reconcile_retired_branches`, `reconcile_removed_repos`) already
enumerate `reference_edges` alongside `symbols`/`chunks` in their docstrings and rely on
the same FK-cascade mechanism proven in §7.2 of the design doc and in
**#84 shipped the writer.** `indexer/symbols.py`'s `extract_file` walks each file's parse
tree once, emitting both `symbols` and `reference_edges` candidates from the same pass
(`indexer/languages.py`'s `EDGE_NODE_KINDS`, Python-only for now — the other six languages
land in #85). `indexer/store.py::index_repo` writes them exactly like `symbols`: an
unconditional per-file `DELETE` followed by a bulk reinsert, inside the same transaction as
the rest of that file's row, so a file whose edges all vanish still sheds its stale rows.
`indexer/store.py`'s cascade-owning functions (`index_repo`'s membership sweep,
`reconcile_retired_branches`, `reconcile_removed_repos`) already enumerate
`reference_edges` alongside `symbols`/`chunks` in their docstrings and rely on the same
FK-cascade mechanism proven in §7.2 of the design doc and in
`tests/integration/test_reconcile.py` / `test_store.py` — no behavior change was needed to
make the cascade correct, because both `repos -> reference_edges` and
`files -> reference_edges` are `ON DELETE CASCADE` foreign keys.

**What gets extracted (Python, #84):**

- **`call`** edges target the rightmost identifier of the callee: `f()` / `a.b.f()` /
`self.f()` all target `f`. Callees with no rightmost identifier (`xs[0]()`, the outer call
of `f()()`) are skipped — candidate-set semantics, not full resolution.
- **`import`** edges target the full dotted path as written, alias-insensitive:
`import a.b.c as d` targets `a.b.c`, not `d`. `from a.b import c, d as e` yields two edges
(`a.b.c`, `a.b.d`). Relative imports preserve source fidelity (`from . import x` ->
`.x`; `from ..p import q` -> `..p.q`). A wildcard `from a.b import *` yields one edge for
the module itself (`a.b`).
- **Enclosing attribution** is the innermost *named* definition on the walk stack when the
call/import node is visited (`None` = module/top-level scope) — a call in a class body
outside any method attributes to the class, not to `None`.
- Duplicate sites (the same target called twice on one line) are two rows by design; there
is no uniqueness constraint, and the query-time resolver (#86) ranks candidates.

**Operational consequence of the `INDEX_SEMANTICS_VERSION` bump (2 -> 3, #84):** every
already-indexed branch's stored `(head_sha, index_semantics_version)` stamp now mismatches
the running code's version, so the *next* run of every branch is a full re-index (not a
skip) purely to backfill `reference_edges` — expected, one-time, and already how the `2`
bump behaved for `chunks`.

## 2. Indexes

| Index | Serves |
Expand Down
Loading
Loading