Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 26 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,19 +72,40 @@ that skips enforcement is the entire reason the policy layer exists.

## Remote engines

The `tinymemory-remote` crate supports the self-hosted native APIs of
Supermemory, Mem0, and Cognee. Each adapter stores TinyMemory's key, category,
session, and provenance in backend metadata (or a Cognee raw-data envelope), so
exact CRUD and portability survive the seam while recall remains engine-native.
The `tinymemory-remote` crate supports the managed and self-hosted native APIs
of Supermemory and Cognee, plus self-hosted Mem0. Each adapter stores
TinyMemory's key, category, session, and provenance in backend metadata (or a
Cognee raw-data envelope), so exact CRUD and portability survive the seam while
recall remains engine-native. Provider-facing dataset names, container tags,
and filenames are bounded stable hashes, so every namespace and key accepted by
the TinyMemory contract remains valid on the remote API.

```rust
use tinymemory_remote::{SupermemoryMemory, supermemory_provider};

let memory = SupermemoryMemory::new("http://localhost:6767", Some("sm_..."))?;
let memory = SupermemoryMemory::self_hosted("http://localhost:6767", "sm_...")?;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

priority medium critique uncertain

Restore new or add the new constructors before documenting them

The README example changes the constructor from SupermemoryMemory::new to SupermemoryMemory::self_hosted, and adds cloud and api constructors for both adapters. However, the repository context shows existing code calling SupermemoryMemory::new and CogneeMemory::new, and no evidence of self_hosted, cloud, or api methods. These examples will not compile for users following them. Update the README after the corresponding Rust code is added or revert the example to use the existing new constructor.

[RULE] doc-example-mismatch ·

let provider = supermemory_provider(memory);
# Ok::<_, anyhow::Error>(provider)
```

Managed APIs have explicit constructors so their authentication cannot be
confused with a self-hosted token:

```rust
use tinymemory_remote::{CogneeMemory, SupermemoryMemory};

let cognee = CogneeMemory::cloud("cognee-api-key")?;
let supermemory = SupermemoryMemory::cloud("sm_...")?;

// Cognee also issues tenant-specific API origins.
let tenant = CogneeMemory::api("https://tenant.example.cognee.ai", "api-key")?;
# Ok::<_, anyhow::Error>((cognee, supermemory, tenant))
```

Cognee Cloud uses `X-Api-Key`; authenticated self-hosted Cognee uses a bearer
access token. Supermemory uses bearer API keys for both deployment modes. All
constructors redact credentials from `Debug` output and transport errors.

All three advertise the mandatory Core, Recall, and Portability families. The
live Docker harness and conformance command are documented in
[`integration/remote-engines/`](integration/remote-engines/README.md).
Expand Down
10 changes: 9 additions & 1 deletion adapters/remote/examples/conformance.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ use tinymemory_remote::{

/// Builds the command-line usage error returned for invalid arguments.
fn usage() -> anyhow::Error {
anyhow::anyhow!("usage: conformance <supermemory|mem0|cognee> <endpoint> [credential]")
anyhow::anyhow!(
"usage: conformance <supermemory|mem0|cognee|cognee-api> <endpoint> [credential]"
)
}

#[tokio::main]
Expand All @@ -36,6 +38,12 @@ async fn main() -> anyhow::Result<()> {
&endpoint,
credential.as_deref(),
)?)),
"cognee-api" => Arc::new(cognee_provider(CogneeMemory::api(
&endpoint,
credential
.as_deref()
.ok_or_else(|| anyhow::anyhow!("cognee-api requires a credential"))?,
)?)),
_ => return Err(usage()),
};

Expand Down
98 changes: 77 additions & 21 deletions adapters/remote/src/cognee.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,19 +8,22 @@ use tinymemory_api::recall::RecallOpts;
use tinymemory_api::traits::Memory;
use tinymemory_api::types::MemoryTaint;

use crate::common::{encode, Dialect, HttpClient, RemoteMemory, StoredEntry};
use crate::common::{stable_id, Dialect, HttpClient, RemoteMemory, StoredEntry};

/// Stable driver id used by configuration and status output.
pub use tinymemory::registry::COGNEE_DRIVER_ID;

/// A self-hosted Cognee server exposed through TinyMemory's storage contract.
/// Default base URL for Cognee's managed API.
pub const COGNEE_API_ENDPOINT: &str = "https://api.cognee.ai";

/// A Cognee managed or self-hosted service exposed through TinyMemory's contract.
#[derive(Debug)]
pub struct CogneeMemory {
inner: RemoteMemory<CogneeDialect>,
}

impl CogneeMemory {
/// Connect to a Cognee server.
/// Connect to a self-hosted Cognee server.
///
/// `access_token` is sent as a bearer token. Local deployments with
/// backend access control disabled may pass `None`.
Expand All @@ -29,12 +32,53 @@ impl CogneeMemory {
///
/// Returns an error when `endpoint` is not an HTTP(S) URL.
pub fn new(endpoint: &str, access_token: Option<&str>) -> anyhow::Result<Self> {
Self::self_hosted(endpoint, access_token)
}

/// Connect to a self-hosted Cognee server.
///
/// `access_token` is sent as a bearer token. Local deployments with
/// authentication disabled may pass `None`.
///
/// # Errors
///
/// Returns an error when `endpoint` is not an HTTP(S) URL.
pub fn self_hosted(endpoint: &str, access_token: Option<&str>) -> anyhow::Result<Self> {
Ok(Self {
inner: RemoteMemory::new(CogneeDialect {
client: HttpClient::bearer(endpoint, access_token)?,
}),
})
}

/// Connect to a Cognee managed API using `X-Api-Key` authentication.
///
/// This accepts a custom endpoint because Cognee Cloud may issue a
/// tenant-specific base URL. Use [`Self::cloud`] for the shared default.
///
/// # Errors
///
/// Returns an error when `endpoint` is invalid or `api_key` is blank.
pub fn api(endpoint: &str, api_key: &str) -> anyhow::Result<Self> {
anyhow::ensure!(
!api_key.trim().is_empty(),
"cognee API key must not be empty"
);
Ok(Self {
inner: RemoteMemory::new(CogneeDialect {
client: HttpClient::api_key(endpoint, Some(api_key))?,
}),
})
}

/// Connect to Cognee's shared managed API endpoint.
///
/// # Errors
///
/// Returns an error when `api_key` is blank.
pub fn cloud(api_key: &str) -> anyhow::Result<Self> {
Self::api(COGNEE_API_ENDPOINT, api_key)
}
}

#[async_trait]
Expand Down Expand Up @@ -128,11 +172,11 @@ struct Dataset {
impl CogneeDialect {
/// Encodes a TinyMemory namespace as a collision-free Cognee dataset name.
fn dataset_name(namespace: &str) -> String {
format!("tinymemory__{}", encode(namespace))
format!("tinymemory__{}", stable_id("dataset", namespace))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

priority medium critique uncertain

Preserve existing dataset names when switching ID encoding

Changing the dataset name from format!("tinymemory__{}", encode(namespace)) to format!("tinymemory__{}", stable_id("dataset", namespace)) alters the generated dataset identifiers. Existing Cognee deployments that stored data under the old naming scheme will no longer match datasets, causing stored records to be invisible and possibly duplicated on re-upload. Consider a migration strategy or keep the previous encoding for existing namespaces.

[RULE] backward-compat ·

}
/// Encodes a TinyMemory key as the uploaded envelope's filename.
fn filename(key: &str) -> String {
format!("{}.tinymemory.json", encode(key))
format!("{}.tinymemory.json", stable_id("key", key))
}

/// Discovers only datasets owned by the TinyMemory adapter.
Expand Down Expand Up @@ -239,33 +283,45 @@ impl Dialect for CogneeDialect {

/// Replaces an existing envelope and uploads the new exact record.
async fn upsert(&self, entry: StoredEntry) -> anyhow::Result<()> {
if let Some(existing) = self
let existing = self
.entries()
.await?
.into_iter()
.find(|item| item.namespace == entry.namespace && item.key == entry.key)
{
self.delete_entry(&existing).await?;
}
.find(|item| item.namespace == entry.namespace && item.key == entry.key);
let body = serde_json::to_vec(&entry)?;
let form = multipart::Form::new()
.text("datasetName", Self::dataset_name(&entry.namespace))
.text("run_in_background", "false")
.part(
"data",
multipart::Part::bytes(body)
.file_name(Self::filename(&entry.key))
.mime_str("application/json")?,
);
let form = multipart::Form::new().part(
"data",
multipart::Part::bytes(body)
.file_name(Self::filename(&entry.key))
.mime_str("application/json")?,
);
let (method, path, form) = if let Some(existing) = existing {
let (dataset_id, data_id) = existing
.remote_id
.split_once(':')
.ok_or_else(|| anyhow!("Cognee record has no dataset id"))?;
(
Method::PATCH,
format!("api/v1/update?data_id={data_id}&dataset_id={dataset_id}"),
form,
)
} else {
(
Method::POST,
"api/v1/remember".to_owned(),
form.text("datasetName", Self::dataset_name(&entry.namespace))
.text("run_in_background", "false"),
)
};
let response = self
.client
.multipart("api/v1/remember")?
.multipart(method, &path)?
.multipart(form)
.send()
.await?;
if !response.status().is_success() {
return Err(anyhow!(
"memory API api/v1/remember returned HTTP {}",
"memory API {path} returned HTTP {}",
response.status()
));
}
Expand Down
88 changes: 83 additions & 5 deletions adapters/remote/src/cognee_test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,16 @@ use std::sync::{Arc, Mutex};

use axum::{
extract::{Multipart, State},
http::StatusCode,
http::{HeaderMap, StatusCode},
response::IntoResponse,
routing::{delete, get, post},
routing::{delete, get, patch, post},
Json, Router,
};
use serde_json::{json, Value};
use tinymemory_api::{
provider::{MemoryCore, MemoryProvider, MemoryRecall},
recall::OwnedRecallOpts,
traits::Memory,
types::{MemoryCategory, MemoryTaint},
};

Expand All @@ -23,7 +24,10 @@ struct AppState(Arc<Mutex<Option<Vec<u8>>>>);

async fn datasets(State(state): State<AppState>) -> Json<Value> {
let values = if state.0.lock().expect("state lock").is_some() {
vec![json!({"id": "dataset-1", "name": "tinymemory__70726f6a656374"})]
vec![json!({
"id": "dataset-1",
"name": super::CogneeDialect::dataset_name("project")
})]
} else {
vec![]
};
Expand Down Expand Up @@ -70,6 +74,67 @@ async fn recall(State(state): State<AppState>) -> Json<Value> {
Json(Value::Array(records))
}

async fn capture_auth(State(state): State<Arc<Mutex<Value>>>, headers: HeaderMap) -> StatusCode {
*state.lock().expect("state lock") = json!({
"authorization": headers
.get("authorization")
.and_then(|value| value.to_str().ok()),
"api_key": headers
.get("x-api-key")
.and_then(|value| value.to_str().ok()),
});
StatusCode::OK
}

#[tokio::test]
async fn cognee_supports_cloud_api_keys_and_self_hosted_bearer_tokens() {
let captured = Arc::new(Mutex::new(Value::Null));
let app = Router::new()
.route("/health", get(capture_auth))
.with_state(captured.clone());
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind");
let endpoint = format!("http://{}", listener.local_addr().expect("address"));
tokio::spawn(async move {
axum::serve(listener, app).await.expect("serve");
});

let api = super::CogneeMemory::api(&endpoint, "cloud-secret").expect("api client");
assert!(api.health_check().await);
let api_headers = captured.lock().expect("state lock").clone();
assert_eq!(api_headers["api_key"], "cloud-secret");
assert!(api_headers["authorization"].is_null());

let hosted = super::CogneeMemory::self_hosted(&endpoint, Some("local-secret"))
.expect("self-hosted client");
assert!(hosted.health_check().await);
let hosted_headers = captured.lock().expect("state lock").clone();
assert_eq!(hosted_headers["authorization"], "Bearer local-secret");
assert!(hosted_headers["api_key"].is_null());

let debug = format!("{api:?}");
assert!(!debug.contains("cloud-secret"));
assert!(super::CogneeMemory::api(&endpoint, " ").is_err());
}

#[test]
fn cognee_remote_names_are_bounded_and_safe_for_arbitrary_contract_keys() {
let unusual = format!("tenant / 🧠 / {}", "x".repeat(500));
let dataset = super::CogneeDialect::dataset_name(&unusual);
let filename = super::CogneeDialect::filename(&unusual);

assert!(dataset.starts_with("tinymemory__tm_"));
assert!(dataset.len() < 100);
assert!(dataset
.bytes()
.all(|byte| byte.is_ascii_alphanumeric() || byte == b'_'));
assert!(filename.starts_with("tm_"));
assert!(filename.ends_with(".tinymemory.json"));
assert!(filename.len() < 100);
assert_eq!(dataset, super::CogneeDialect::dataset_name(&unusual));
}

#[tokio::test]
async fn native_cognee_round_trips_the_tinymemory_contract() {
let state = AppState::default();
Expand All @@ -79,6 +144,7 @@ async fn native_cognee_round_trips_the_tinymemory_contract() {
.route("/api/v1/datasets/{dataset}/data/{data}/raw", get(raw))
.route("/api/v1/datasets/{dataset}/data/{data}", delete(remove))
.route("/api/v1/remember", post(remember))
.route("/api/v1/update", patch(remember))
.route("/api/v1/recall", post(recall))
.route("/health", get(|| async { StatusCode::OK }))
.with_state(state);
Expand All @@ -90,7 +156,8 @@ async fn native_cognee_round_trips_the_tinymemory_contract() {
axum::serve(listener, app).await.expect("serve");
});

let driver = crate::cognee_provider(super::CogneeMemory::new(&endpoint, None).expect("client"));
let driver =
crate::cognee_provider(super::CogneeMemory::self_hosted(&endpoint, None).expect("client"));
tinymemory_api::provider::audit_provider(&driver).expect("honest capabilities");
driver
.store(
Expand All @@ -103,12 +170,23 @@ async fn native_cognee_round_trips_the_tinymemory_contract() {
)
.await
.expect("store");
driver
.store(
"project",
"key",
"updated knowledge graph",
MemoryCategory::Conversation,
Some("session"),
MemoryTaint::ExternalSync,
)
.await
.expect("upsert");
let entry = driver
.get("project", "key")
.await
.expect("get")
.expect("entry");
assert_eq!(entry.content, "knowledge graph");
assert_eq!(entry.content, "updated knowledge graph");
assert_eq!(entry.taint, MemoryTaint::ExternalSync);
assert_eq!(
driver
Expand Down
6 changes: 3 additions & 3 deletions adapters/remote/src/common.rs
Original file line number Diff line number Diff line change
Expand Up @@ -144,9 +144,9 @@ impl HttpClient {
Ok(status)
}

/// Starts an authenticated multipart POST request.
pub(crate) fn multipart(&self, path: &str) -> anyhow::Result<RequestBuilder> {
self.request(Method::POST, path)
/// Starts an authenticated multipart request.
pub(crate) fn multipart(&self, method: Method, path: &str) -> anyhow::Result<RequestBuilder> {
self.request(method, path)
}

/// Reports whether a GET endpoint responds successfully.
Expand Down
4 changes: 2 additions & 2 deletions adapters/remote/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ mod common;
pub mod mem0;
pub mod supermemory;

pub use cognee::{CogneeMemory, COGNEE_DRIVER_ID};
pub use cognee::{CogneeMemory, COGNEE_API_ENDPOINT, COGNEE_DRIVER_ID};
pub use mem0::{Mem0Memory, MEM0_DRIVER_ID};
pub use supermemory::{SupermemoryMemory, SUPERMEMORY_DRIVER_ID};
pub use supermemory::{SupermemoryMemory, SUPERMEMORY_API_ENDPOINT, SUPERMEMORY_DRIVER_ID};

use std::sync::Arc;

Expand Down
Loading