FTS-based session/conversation search for episodic recall

[bm-clawd]

Idea

Agents need to recall not just stored knowledge (notes) but past conversations — "did we discuss X last week?" or "what was the approach we decided on?"

Inspired by Hermes Agent's session_search which stores all sessions in SQLite with FTS5 indexing, groups results by session, and summarizes relevant matches before returning them.

Problem

Currently, agent conversation history is ephemeral — it lives in the session and disappears after compaction. Daily memory notes capture some of this, but:

• Coverage depends on the agent remembering to write things down
• Search is keyword-based over markdown files
• No way to search across conversation turns directly
• No session-level grouping ("find the conversation where we discussed X")

Proposal

Add an episodic recall layer to BM:

1. Conversation ingest — Agents can write conversation summaries/turns as a structured note type (or a dedicated table)
2. FTS indexing — Full-text search over conversation content
3. Session grouping — Results grouped by conversation/session, not individual messages
4. Smart truncation — Return relevant windows around matches, not entire transcripts
5. Optional summarization — Summarize matched sessions before returning (could be agent-side)

This complements BM's existing knowledge graph (which is great for structured facts and relations) with temporal, episodic recall (which is great for "when did we..." and "what did we decide about...").

Open Questions

• Should this be a new note type (conversation) or a separate store?
• How does this interact with Momentum's interaction/session model?
• Could BM Cloud aggregate conversations across multiple agents?

References

• Hermes Agent session_search (SQLite + FTS5 + LLM summarization)
• Relates to Momentum interactions design

[claude]

Claude finished @bm-clawd's task in 1m 10s —— View job

---

Issue Triage

• Analyze issue
• Review existing search/FTS implementation
• Check for duplicate issues
• Apply labels and provide triage

---

Analysis

Type: Feature Request — new episodic recall layer for conversation/session search

Priority: Medium — valuable capability enhancement, no existing functionality broken

Component: MCP tools, Database/Search, possibly new note type

Complexity: Complex — requires architectural decisions and significant new functionality

---

Triage Notes

This is a well-specified feature request for episodic memory — the ability to search across past conversations and sessions, not just stored knowledge notes. The issue outlines a clear problem:

• Current note-taking is agent-dependent (coverage gaps)
• No way to search conversation turns directly
• No session-level grouping for temporal queries like "what did we decide about X last week?"

Related Issue: This is complementary to #669 - Coding agent sidecar, which addresses the ingestion side (watching session files → structured notes). Issue #669 would produce the data that this issue's search would query over. They're not duplicates — together they form a complete episodic memory pipeline:

• Feature: Coding agent sidecar — watch session transcripts, build knowledge graph #669 = write path (transcript → BM notes)
• FTS-based session/conversation search for episodic recall #687 = read path (FTS search + session grouping over those notes)

Existing FTS Infrastructure: BM already has robust FTS support in src/basic_memory/repository/sqlite_search_repository.py and postgres_search_repository.py. The key additions needed would be:

1. Session/conversation note type — a structured note type (or FTS-indexed table) for conversation turns, compatible with the session summary format proposed in Feature: Coding agent sidecar — watch session transcripts, build knowledge graph #669
2. Session grouping — aggregate search hits by session/conversation ID rather than returning individual message hits
3. Context windowing — smart truncation returning relevant snippets around matches
4. New MCP tool — search_conversations(query, timeframe, session_id) or extending search_notes with a conversation type filter

Open Questions to Resolve:

1. Should conversation sessions be a new note type: conversation (fits existing architecture, simpler) or a dedicated DB table (more performant for high-volume session ingestion)?
2. How should session grouping work — by frontmatter session_id field? By parent directory?
3. Should this tool return raw snippets or summarized sessions? (Summarization could be agent-side to keep BM lean)

Suggested Next Steps:

• Implement type: conversation note support in the existing FTS search with session-level grouping
• Coordinate with Feature: Coding agent sidecar — watch session transcripts, build knowledge graph #669 to align the session note schema
• Start with snippet-based results (no LLM summarization in BM core — keep it lean)

Labels applied: enhancement