[Priority 3] Implement memory consolidation (pattern synthesis)

[marknutter]

Problem

Memory store can accumulate redundant/overlapping entries:

• 5 separate 'auth bug' findings → clutters recall results
• Related concepts scattered across entries → harder to see patterns
• No synthesis of accumulated knowledge over time

Proposal (from AIGNE paper analysis)

Periodic memory consolidation that:

1. Analyzes related memories (similarity detection via keywords/embeddings)
2. Synthesizes patterns across entries
3. Creates new consolidated entry tagged synthesis with references to sources
4. Example: 5 'auth bug' entries → 1 'authentication weaknesses summary' + provenance

Implementation

from rlm.consolidator import find_related_entries, synthesize_pattern

# Run weekly or on-demand
related_groups = find_related_entries(min_similarity=0.7)
for group in related_groups:
    if len(group) >= 3:  # Worth consolidating
        synthesis = synthesize_pattern(group)
        store_synthesis(synthesis, source_ids=[e.id for e in group])

Strategy

• Similarity detection: keyword overlap + semantic similarity (if vectors available)
• LLM synthesis: 'Given these related memories, extract common patterns'
• Provenance: link synthesis to source entries
• Retention: keep originals (for audit), prioritize synthesis in recall

Impact

• Memory quality improves over time
• Reduces redundancy
• Surfaces patterns not visible in individual entries
• Better recall results (consolidated knowledge ranks higher)

Effort

4-5 days

Related

• Memory lifecycle from 'Everything is Context' paper
• Self-improving strategies (learned_patterns.md)