Distributed long agent memory

[lukemartinlogan]

We want to add distributed semantic search to iowarp.

Update CAE

We need to add a summarization operator to the context assimilation engine

Let's make a "Summary" operator to the CAE. The summary operator should take as input the "description" blob. This operator will use a local LLM to summarize the the "description" blob. I want to compare different models here, so we should maybe have an inference server that executes elsewhere. We can use environment variables to choose which endpoint to use for summary operators. I want to compare a 270M gemma model with 3.5b qwen model.

The general idea is that we produce a small 4-8 words summary of the "description" blob and then use that to build a knowledge graph.

Update CTE

UpdateKnowledgeGraph

A new chimod method.

Takes as input:

1. TagId
2. Summary to store in the knowledge graph

Add a compiler macro called WRP_CTE_ENABLE_KNOWLEDGE_GRAPH.
TagInfo should store the summary as a std::string.

The knowledge graph should be implemented using some existing knowledge graph design. Let's implement it ourselves though, don't use external library for this. The knowledge graph should be a new class stored in context transport primitives under context-transport-primitives/include/hermes_shm/data_structures/priv/knowledge_graph. It can use the STL for now for its implementation, that's fine.

The knowledge graph should store key-value pairs. The knowledge graph should implement the following basic methods:

1. Add(Key key, Value value)
2. std::vector Search(std::string prompt, int top_k)

SemanticQuery

A new chimod method.

Takes as input:

1. A prompt
2. The maximum number of entries to collect match

Returns:
A vector of TagId representing the number of entries.
They should be sorted in order of confidence.

Future Work

We will need to think about how to prune the knowledge graph. What happens when a tag gets deleted? Should we store reference counts?

[lukemartinlogan]

For the SemanticQuery, the task should support Broadcast. Broadcast needs the Aggregate method in the SemanticQueryTask implemented. The top k vector should also include confidence scores (cosine similarity). When aggregating, we should keep only the top k from the original and new tasks.