docs: clarify FastNeighborInsert algorithm, edge-role invariants, and Query.ts rework scope

PLAN.md:
- FastNeighborInsert row: use Williams-cutoff distance (not K), lazy Daydreamer
  reconnection, cosine=discovery+Bayesian vs Hebbian=TSP traversal; DESIGN.md cross-ref
- Query.ts/QueryResult.ts: status "Needs Rework"; existing flat top-K code is superseded

TODO.md:
- P1-C1: neighbors found within Williams-cutoff distance; Daydreamer builds additional
  edges lazily; edge-role invariant (SemanticNeighbor.cosineSimilarity vs edges_hebbian)
- P1-C3: add test that FastNeighborInsert does NOT create Hebbian edges
- P1-E: "Rewrite" not "Upgrade"; note flat-scoring code path is fully superseded;
  add Hebbian edge traversal in P1-E1; recommended order updated

DESIGN.md:
- SemanticNeighbor: add edge-role distinction table; remove misleading "TSP-ready" comment
- SemanticNeighborSubgraph: inline note that TSP uses Hebbian weights for tour traversal
- Incremental Strategy: Williams-cutoff distance; Daydreamer lazy reconnection

Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>

Author: Copilot

DESIGN.md

@@ -434,11 +434,28 @@ Sparse radius-graph edge connecting pages with high cosine similarity. Used for
 
 > **Note:** The current codebase names this type `MetroidNeighbor` — this is an architectural naming error introduced by early conceptual drift. The correct term is `SemanticNeighbor` (or equivalent). A code-level rename is tracked in the TODO. The edge is a proximity concept, not a Metroid concept.
 
+**Critical distinction — two edge types, two roles:**
+
+| Edge type | Storage | Role |
+|-----------|---------|------|
+| `SemanticNeighbor` | `neighbor_graph` IDB store | Neighbor discovery during ingest; Bayesian belief updates |
+| Hebbian edge (`Edge`) | `edges_hebbian` IDB store | TSP tour traversal distance; LTP/LTD strengthening/decay |
+
+`SemanticNeighbor.cosineSimilarity` drives:
+- Which pages become neighbors during `FastNeighborInsert` (Williams-cutoff distance, not a fixed K)
+- Bayesian belief score updates for the retrieved page set
+
+Hebbian `Edge.weight` drives:
+- The distance metric used by `OpenTSPSolver` when ordering pages into a coherent narrative path
+- Strength of connection for LTP/LTD during Daydreamer consolidation
+
+These two edge types must **never** be conflated or substituted for one another.
+
 ```typescript
 interface SemanticNeighbor {
   neighborPageId: Hash;
   cosineSimilarity: number;
-  distance: number;           // 1 - cosineSimilarity (TSP-ready)
+  distance: number;           // 1 - cosineSimilarity; used for subgraph edge weight
 }
 ```
 
@@ -450,6 +467,9 @@ Induced subgraph for BFS-based coherence path expansion.
 ```typescript
 interface SemanticNeighborSubgraph {
   nodes: Hash[];
+  // distance: 1 - cosineSimilarity; used for BFS expansion candidate selection.
+  // OpenTSPSolver uses Hebbian edge weights (from edges_hebbian) as the tour
+  // traversal distance to determine how far to walk — not these cosine distances.
   edges: { from: Hash; to: Hash; distance: number }[];
 }
 ```
@@ -556,7 +576,9 @@ Rather than returning nearest neighbors by similarity, Cortex traces a coherent
 7. **Mark Dirty** — Flag volumes for full recalc by Daydreamer
 
 **Incremental Strategy:**
-Fast local semantic neighbor insertion keeps query-time latency low. Full neighborhood recalculation is deferred to idle Daydreamer passes. Hotpath admission runs at ingest time for new pages and hierarchy prototypes.
+Fast local semantic neighbor insertion keeps ingest-time latency low. At ingest time, only the initial forward and reverse edges are created — neighbors are selected by cosine similarity within Williams-cutoff **distance** (not a fixed K; the cutoff is derived from `HotpathPolicy`). On degree overflow, the lowest-cosine-similarity neighbor is evicted.
+
+Full cross-edge reconnection is intentionally deferred: Daydreamer walks the graph during idle passes to build additional edges, strengthening or pruning connections via LTP/LTD. This avoids a full graph recalculation on every insert while still converging to a well-connected graph over time. Hotpath admission runs at ingest time for new pages and hierarchy prototypes.
 
 ## Consolidation Design
 

PLAN.md

@@ -81,7 +81,7 @@ This document tracks the implementation status of each major module in CORTEX. I
 | Page Builder | ✅ Complete | `hippocampus/PageBuilder.ts` | Builds signed `Page` entities with `contentHash`, `vectorHash`, `prevPageId`/`nextPageId` linkage; covered by `tests/hippocampus/PageBuilder.test.ts` |
 | Ingest Orchestrator | 🟡 Partial | `hippocampus/Ingest.ts` | `ingestText()` implemented: chunk → embed → persist pages + PageActivity → create Book → run hotpath promotion sweep. **Missing:** hierarchy building (Volume/Shelf), semantic neighbor insertion. |
 | Hierarchy Builder | ❌ Missing | `hippocampus/HierarchyBuilder.ts` (planned) | Construct/update Books, Volumes, Shelves; attempt tier-quota hotpath admission for each level's medoid/prototype; Williams-derived fanout bounds; trigger split via ClusterStability when bounds exceeded |
-| Fast Semantic Neighbor Insert | ❌ Missing | `hippocampus/FastNeighborInsert.ts` (planned) | Incremental semantic neighbor graph update; max degree derived from HotpathPolicy (not hardcoded K); evict lowest-cosine-similarity neighbor on degree overflow; check new page for hotpath admission. **Note:** Not to be confused with Metroid construction, which is a CORTEX retrieval concern. |
+| Fast Semantic Neighbor Insert | ❌ Missing | `hippocampus/FastNeighborInsert.ts` (planned) | Cosine-nearest neighbors within Williams-cutoff distance (not fixed K). Degree overflow evicts lowest-cosine-similarity neighbor. Initial edges only at ingest; Daydreamer builds additional edges lazily. `SemanticNeighbor.cosineSimilarity` drives discovery + Bayesian updates; Hebbian weights (separate) drive TSP traversal. See DESIGN.md §Graph Structures for the full edge-role invariant. |
 
 **Hippocampus Status:** 2.5/5 complete (50%)
 
@@ -100,8 +100,8 @@ This document tracks the implementation status of each major module in CORTEX. I
 | Seed Selection | ❌ Missing | `cortex/SeedSelection.ts` (planned) | Threshold-based top-k page selection from ranking output |
 | Subgraph Expansion | 🟡 Partial | `storage/IndexedDbMetadataStore.ts` (`getInducedMetroidSubgraph` — to be renamed `getInducedNeighborSubgraph`) | BFS expansion implemented in storage layer; needs dynamic Williams bounds; needs orchestration wrapper |
 | Open TSP Solver | ❌ Missing | `cortex/OpenTSPSolver.ts` (planned) | Dummy-node open-path heuristic for coherent ordering |
-| Query Orchestrator | 🟡 Partial | `cortex/Query.ts` | `query()` implemented: hotpath-first scoring → warm/cold spill → PageActivity update → promotion sweep. **Missing:** MetroidBuilder, dialectical zone scoring, subgraph expansion, TSP coherence, query cost meter. |
-| Result DTO | 🟡 Partial | `cortex/QueryResult.ts` | Minimal DTO: `pages`, `scores`, `metadata`. **Missing:** `coherencePath`, `metroid`, `knowledgeGap`, `provenance` fields. |
+| Query Orchestrator | 🟡 Needs Rework | `cortex/Query.ts` | Flat top-K scoring implemented (hotpath-first → warm/cold spill → PageActivity update → promotion sweep). **Must be substantially reworked** to implement the full dialectical pipeline: replace flat scoring with hierarchical resident-first ranking, add MetroidBuilder, dialectical zone scoring (thesis/antithesis/synthesis), subgraph expansion with dynamic Williams bounds, TSP coherence path, and query cost meter. The existing implementation does not use Hebbian edges or cosine-similarity-bounded subgraph expansion; it is a functional placeholder only. |
+| Result DTO | 🟡 Needs Rework | `cortex/QueryResult.ts` | Minimal DTO (`pages`, `scores`, `metadata`). **Must be reworked** to add `coherencePath: Hash[]`, `metroid?: { m1, m2, centroid }`, `knowledgeGap?: KnowledgeGap`, and `provenance: { subgraphSize, hopCount, edgeWeights, vectorOpCost, earlyStop }`. |
 
 **Cortex Status:** 1.5/9 complete (17%)
 

TODO.md

@@ -308,11 +308,12 @@ These items add hierarchical routing and coherent path ordering. They transform
 **Why:** Need a sparse semantic neighbor graph for coherent path tracing. This graph connects pages with high cosine similarity and is used for BFS subgraph expansion during retrieval. Degree must be bounded by `HotpathPolicy` to prevent unbounded graph mass growth. **This is not related to Metroid construction** — the semantic neighbor graph is a proximity concept, not a dialectical probe concept.
 
 - [ ] **P1-C1:** Implement `hippocampus/FastNeighborInsert.ts`
-  - For each new page, compute similarity to existing pages
-  - Derive max neighbors per page from `HotpathPolicy` constant (not hardcoded K)
-  - Insert forward edges (page → neighbors) as `SemanticNeighbor` records
-  - Insert reverse edges (neighbors → page), respecting max degree
+  - For each new page, find cosine-nearest neighbors within Williams-cutoff **distance** (not a fixed K); derive the cutoff radius from `HotpathPolicy` rather than a hardcoded constant
+  - Insert forward edges (page → neighbors) as `SemanticNeighbor` records, respecting max degree
+  - Insert reverse edges (neighbors → page), respecting max degree per direction
   - If a page is already at max degree, evict the neighbor with the lowest cosine similarity
+  - Insert only initial edges at ingest time; do not attempt full cross-edge reconnection — Daydreamer walks the graph during idle passes to build additional edges (avoids full graph recalc on every insert)
+  - **Edge role invariant:** `SemanticNeighbor.cosineSimilarity` is used for neighbor discovery and Bayesian belief updates. Hebbian edge weights (in `edges_hebbian`) are used for TSP tour traversal. These are separate edge types with separate roles; do not mix them.
   - Mark affected volumes as dirty for full Daydreamer recalc
   - After insertion, check new page for hotpath admission via `SalienceEngine`
 
@@ -321,10 +322,11 @@ These items add hierarchical routing and coherent path ordering. They transform
 
 - [ ] **P1-C3:** Add semantic neighbor insert test coverage
   - `tests/hippocampus/FastNeighborInsert.test.ts`
-  - Test neighbor lists are bounded by the policy-derived max degree
+  - Test neighbor lists are bounded by Williams-cutoff distance (not a fixed K)
   - Test symmetry (if A→B, then B→A)
-  - Test that degree overflow evicts lowest-similarity neighbor, not a random one
+  - Test that degree overflow evicts lowest-cosine-similarity neighbor, not a random one
   - Test that new page is considered for hotpath admission after insertion
+  - Test that `edges_hebbian` records are NOT created by FastNeighborInsert (Hebbian is Daydreamer's concern)
 
 **Exit Criteria:** Semantic neighbor graph is maintained during ingest with policy-bounded degree.
 
@@ -420,19 +422,21 @@ These items add hierarchical routing and coherent path ordering. They transform
 
 **Why:** This is the "aha" moment — return memories in natural narrative order through the resident hotpath via dialectical Metroid exploration, with dynamic, sublinear expansion bounds.
 
-- [ ] **P1-E1:** Upgrade `cortex/Query.ts` (full version)
+> **Note on scope:** The existing `cortex/Query.ts` is a flat top-K scorer that does not use MetroidBuilder, Hebbian edge traversal, or cosine-similarity-bounded subgraph expansion. It must be **substantially reworked** — not merely extended — to implement the dialectical pipeline described below. The same applies to `cortex/QueryResult.ts`. Do not attempt to preserve the flat-scoring code path; it is superseded entirely.
+
+- [ ] **P1-E1:** Rewrite `cortex/Query.ts` (full dialectical version)
   - Use resident-first hierarchical ranking to select topic medoid (m1)
   - Call `MetroidBuilder` to construct `{ m1, m2, c }`
   - If knowledge gap detected, include in result and continue with partial Metroid (m1 only)
   - Use centroid `c` as the primary scoring anchor for page selection
   - Derive dynamic subgraph bounds from `HotpathPolicy` (`maxSubgraphSize`, `maxHops`, `perHopBranching`)
-  - Call `MetadataStore.getInducedNeighborSubgraph(seedPages, maxHops)` using dynamic `maxHops`
+  - Call `MetadataStore.getInducedNeighborSubgraph(seedPages, maxHops)` using dynamic `maxHops`; traverse edges using Hebbian weights for tour distance (not cosine similarity)
   - Call `OpenTSPSolver.solve(subgraph)`
   - Return ordered page list via coherent path
   - **Query cost meter:** count vector operations; early-stop and return best-so-far if cost exceeds Williams-derived budget
   - Include provenance metadata (hop count, edge weights, subgraph size, cost, Metroid details)
 
-- [ ] **P1-E2:** Upgrade `cortex/QueryResult.ts`
+- [ ] **P1-E2:** Rewrite `cortex/QueryResult.ts`
   - Add `coherencePath: Hash[]` (ordered page IDs)
   - Add `metroid?: { m1: Hash; m2: Hash | null; centroid: Float32Array | null }` (Metroid used for this query)
   - Add `knowledgeGap?: KnowledgeGap` (if antithesis discovery failed)
@@ -847,7 +851,7 @@ If you're reading this and want to know "what do I work on right now?", here's t
 7. **P1-M1/M2:** Implement `cortex/MetroidBuilder.ts` with Matryoshka unwinding
 8. **P1-N1/N2:** Implement `cortex/KnowledgeGapDetector.ts`
 9. **P1-D1:** Implement `cortex/OpenTSPSolver.ts`
-10. **P1-E1:** Upgrade `cortex/Query.ts` to full dialectical orchestrator
+10. **P1-E1:** Rewrite `cortex/Query.ts` to full dialectical orchestrator (substantial rework; not backward-compatible with flat top-K version)
 
 ---