Five contract doors for the openhuman engine shed

crates/tinymemory-api/src/provider/chunks.rs

@@ -42,7 +42,8 @@ use crate::provider::types::SourceScope;
 // able to name them without compiling this trait — and re-exported here so
 // every historical path keeps resolving and the types stay the same types.
 pub use tinymemory_bus::provider::chunks::{
-    ChunkDetail, ChunkEmbedding, ChunkListRow, ChunkQuery, SourceTotal,
+    ChunkDetail, ChunkEmbedding, ChunkListRow, ChunkQuery, ChunkScore, ChunkScoreSignals,
+    SourceIngestQuery, SourceIngestStatus, SourceTotal, DEFAULT_DROP_THRESHOLD,
 };
 
 /// Direct read access to the chunk tier.
@@ -238,4 +239,108 @@ pub trait MemoryChunks: Send + Sync {
         chunk_ids: &[String],
         model_signature: &str,
     ) -> Result<Vec<ChunkEmbedding>, MemoryError>;
+
+    /// One chunk's admission decision, and the signals it was reached from.
+    ///
+    /// The scorer's own row: what each signal measured, what they summed to,
+    /// whether the chunk was kept, and why. A diagnostic read for "this
+    /// document is in memory and that one is not" — not an input to ranking,
+    /// which [`MemoryRetrieval`](super::MemoryRetrieval) owns and which happens
+    /// per query rather than once at ingest.
+    ///
+    /// # Why the driver has to answer this
+    ///
+    /// The decision is a row in the driver's own score table, written at
+    /// admission time under the policy in force then. Nothing in the chunk tier
+    /// records it: [`Self::chunk_detail`] can say a chunk exists and is marked
+    /// dropped, and cannot say what it scored or which signal it failed on. A
+    /// caller cannot re-derive it either — re-running the scorer today would
+    /// answer under today's policy, and produce a number the store never used.
+    ///
+    /// # `None` is "never scored", not "scored zero"
+    ///
+    /// A chunk with no score row was not judged, which is a different fact from
+    /// a chunk judged uninteresting and kept anyway. Collapsing the two — by
+    /// defaulting to a zeroed [`ChunkScore`] — reports a verdict that was never
+    /// reached, on a screen whose entire purpose is to explain verdicts.
+    ///
+    /// # Errors
+    ///
+    /// [`MemoryError::Unsupported`] from a driver that implements this family
+    /// but keeps no admission record. That is the honest answer for a driver
+    /// that admits everything, and it is deliberately not defaulted to
+    /// `Ok(None)`: "this driver does not score" and "this chunk was not scored"
+    /// are different answers, and only the first is true of every chunk.
+    ///
+    /// Otherwise backend failures; an unknown chunk id yields `Ok(None)`, the
+    /// same as a known one with no score row — a caller inspecting a chunk it
+    /// just listed cannot tell those apart and does not need to.
+    async fn chunk_score(&self, chunk_id: &str) -> Result<Option<ChunkScore>, MemoryError> {
+        let _ = chunk_id;
+        Err(MemoryError::unsupported(Capability::Chunks))
+    }
+
+    /// How far ingest has got for each of the sources named in
+    /// `source_prefixes`.
+    ///
+    /// One row per query, in the order asked, echoing
+    /// [`SourceIngestQuery::source_id`] so a caller can pair them by value
+    /// rather than by position. **A query whose prefix matches nothing still
+    /// gets a row**, zero-filled — see below.
+    ///
+    /// # Why the caller supplies the prefix
+    ///
+    /// Because the caller is the only party that can. The prefix is derived
+    /// from a configured source's kind, toolkit and connection id, which live
+    /// in the host's source registry; a driver asked to derive it would need
+    /// that registry, which is precisely the coupling this contract exists to
+    /// remove. So the host states the key and the driver counts the rows under
+    /// it — each side answering from what it actually holds.
+    ///
+    /// # Why this is not [`Self::source_totals`]
+    ///
+    /// Three differences, and a caller that substituted one for the other would
+    /// get a result that renders as a healthy store.
+    ///
+    /// 1. [`SourceTotal`] has no pending count and none can be derived from it.
+    ///    The predicate spans the embedding sidecar and the re-embed skip
+    ///    ledger as well as the chunk's own lifecycle column, so a caller
+    ///    reading only the chunk tier reports nothing in flight — which is what
+    ///    a finished sync looks like.
+    /// 2. `source_totals` returns the groups that *exist*. A configured source
+    ///    that has never synced forms no group, so it vanishes from the answer
+    ///    rather than appearing idle — and a source missing from a dashboard
+    ///    reads as one that was never set up.
+    /// 3. [`SourceTotal::source_id`] is the ingest key the chunk rows carry;
+    ///    [`SourceIngestQuery::source_id`] is the registry entry a user
+    ///    configured. For a connector source the two share no substring, so
+    ///    matching them up is not a formatting difference a caller can paper
+    ///    over.
+    ///
+    /// # Freshness is deliberately absent
+    ///
+    /// An `Active`/`Recent`/`Idle` label is arithmetic over
+    /// [`SourceIngestStatus::last_chunk_at_ms`] and the current time. Answering
+    /// it here would freeze the driver's clock into the reply, so a panel
+    /// rendering the label a minute later would show how fresh the source was
+    /// when the driver looked. The caller has the timestamp and its own clock.
+    ///
+    /// # Errors
+    ///
+    /// [`MemoryError::Unsupported`] from a driver that implements this family
+    /// but tracks no per-source ingest state — not defaulted to zero-filled
+    /// rows, which would report every configured source as never synced.
+    ///
+    /// Otherwise backend failures, for the whole batch rather than per row: the
+    /// counts come from one store, so a read that fails fails for all of them,
+    /// and a partial answer would be indistinguishable from a set of genuinely
+    /// empty sources. An empty `source_prefixes` yields an empty vector without
+    /// touching the store.
+    async fn source_ingest_status(
+        &self,
+        source_prefixes: &[SourceIngestQuery],
+    ) -> Result<Vec<SourceIngestStatus>, MemoryError> {
+        let _ = source_prefixes;
+        Err(MemoryError::unsupported(Capability::Chunks))
+    }
 }

crates/tinymemory-api/src/provider/content.rs

@@ -25,6 +25,15 @@ use crate::provider::types::{IngestItem, IngestOutcome, SourceScope};
 use crate::tree::{IngestRequest, QueryResult, SummaryForest, TreeLeaf, TreeStatus};
 use crate::types::{NamespaceDocumentInput, NamespaceRetrievalContext, StoredMemoryDocument};
 
+// The value types the summariser door exchanges. They are defined in
+// `tinymemory-bus` — they cross the module boundary, and a host that only makes
+// calls must be able to name them without compiling this trait — and
+// re-exported here so the family's vocabulary is reachable from the family, the
+// same arrangement `provider::chunks` uses. They are re-exported at
+// `crate::tree` too, alongside the rest of the tree vocabulary; both paths name
+// the same items, not twins of them.
+pub use tinymemory_bus::tree::{RootSummary, SummaryContext, SummaryInput, SummaryOutput};
+
 /// Bulk content ingestion — the driver owns chunking and embedding.
 ///
 /// The distinction from [`crate::provider::MemoryCore::store`] is ownership of
@@ -410,4 +419,122 @@ pub trait MemoryTree: Send + Sync {
     async fn flush_source_tree(&self, _source_scope: &str) -> Result<u64, MemoryError> {
         Err(MemoryError::unsupported(Capability::Tree))
     }
+
+    /// Fold `inputs` into one parent summary, using the driver's own chat
+    /// provider, and report what that call cost.
+    ///
+    /// This is the LLM step of a seal, exposed on its own. Everything else in
+    /// this family either writes content ([`Self::append`]), navigates what is
+    /// already sealed, or asks the driver to run a whole seal/cascade pass
+    /// ([`Self::seal`], [`Self::cascade`], [`Self::flush_source_tree`]). This
+    /// one does a single fold and hands the text back, which is what a caller
+    /// driving its own cascade needs and what none of the others can be made to
+    /// answer: they return tree *state*, and the summary they produced is never
+    /// in it.
+    ///
+    /// # Why the provider is the driver's and not the caller's
+    ///
+    /// The summariser is configured where the engine is — model, temperature,
+    /// output language, rate card. A caller reaching memory over a module has
+    /// none of those, so a fold it performed itself would use a different model
+    /// than every fold the scheduler performs, and the two would disagree about
+    /// the shape of a summary in the same tree. Passing the configuration
+    /// across instead is not an option: no signature in this contract names a
+    /// config type, for the reasons in [`crate::provider`].
+    ///
+    /// The consequence is that the usage numbers on [`SummaryOutput`] are the
+    /// only record of the spend. Nothing on the caller's side saw the request.
+    ///
+    /// # The budgets and the ask are inputs, not hints
+    ///
+    /// A driver applies [`SummaryContext`]'s three token budgets exactly as
+    /// given and selects its prompt from [`SummaryContext::ask`]. It does not
+    /// substitute its own defaults for a budget it finds implausible: the
+    /// caller owns the level it is sealing and therefore owns the budget for
+    /// it, and a driver that quietly widened one would produce a node that
+    /// overruns the level above. See that type for what each budget bounds and
+    /// what a zero does.
+    ///
+    /// # Errors
+    ///
+    /// [`MemoryError::Unsupported`] from a driver with a tree family but no
+    /// provider-backed summariser — deliberately not an empty summary, which a
+    /// caller would seal as a real, blank node.
+    ///
+    /// [`MemoryError::Invalid`] for a [`SummaryContext::tree_kind`] the driver
+    /// does not recognise. Refusing beats folding under a guessed kind: the
+    /// summary is written either way and nothing afterwards records which
+    /// prompt produced it.
+    ///
+    /// Otherwise a backend failure, which here includes the provider call — a
+    /// model that errors, times out, or refuses. That is a real and recurring
+    /// outcome rather than an exceptional one, and the caller is expected to
+    /// have a deterministic fallback for it; the driver does not silently
+    /// substitute one, because a caller cannot tell a fallback summary from a
+    /// model's own work once it is in the tree.
+    ///
+    /// Nothing to fold is **not** an error: an empty slice, or one whose inputs
+    /// are all blank, returns a default [`SummaryOutput`] with empty content and
+    /// no usage. That is the same idempotence [`Self::seal`] has, and it is what
+    /// lets a cascade call this unconditionally at every level.
+    async fn summarise(
+        &self,
+        _inputs: &[SummaryInput],
+        _context: &SummaryContext,
+    ) -> Result<SummaryOutput, MemoryError> {
+        Err(MemoryError::unsupported(Capability::Tree))
+    }
+
+    /// Every namespace's root summary, truncated to a per-namespace cap and a
+    /// total cap.
+    ///
+    /// The top of the markdown time tree, read across all namespaces at once.
+    /// Its caller is a prompt builder: this is the block of standing context a
+    /// host puts in front of a model, which is why the bounds are in
+    /// **characters** rather than in rows, and why the truncation happens
+    /// driver-side rather than after the read. A caller that fetched whole
+    /// roots and clipped them itself would pay for text it then threw away, and
+    /// would clip at a boundary the driver did not choose.
+    ///
+    /// # Why not [`Self::summary_forest`]
+    ///
+    /// Different tier and different shape. The forest walks the *sealed summary
+    /// forest* — one tree per ingest source, levelled by seal generation — and
+    /// returns structure: ids, parents, children, no bodies. This returns the
+    /// **markdown time tree**'s root body, one per namespace, and bodies are
+    /// the entire point. [`crate::tree`] describes why the two live side by
+    /// side.
+    ///
+    /// # Bounds
+    ///
+    /// `per_namespace_cap` clips each namespace's body; `total_cap` stops the
+    /// walk once the accumulated bodies reach it, so the last body included may
+    /// itself be clipped short of its own cap. Both are applied in namespace
+    /// order, which is stable and alphabetical — so a total cap that binds
+    /// drops the *tail* of the namespace list rather than sampling across it,
+    /// exactly the reading [`SummaryForest::truncated`] warns about. A caller
+    /// that needs a particular namespace represented cannot rely on a small
+    /// total cap to include it.
+    ///
+    /// A clipped body ends in a `[... truncated]` marker; see
+    /// [`RootSummary::body`].
+    ///
+    /// # Errors
+    ///
+    /// [`MemoryError::Unsupported`] from a driver with a tree family but no
+    /// markdown time tree.
+    ///
+    /// Otherwise this is deliberately hard to fail. The read is a best-effort
+    /// filesystem scan: a namespace whose root cannot be read is skipped and
+    /// the rest are returned, and a workspace with no tree at all is an empty
+    /// vector. That is the engine's own behaviour and the door does not
+    /// manufacture an error it never produced — the result is a prompt block,
+    /// and one unreadable namespace is worth less than failing the turn.
+    async fn root_summaries_with_caps(
+        &self,
+        _per_namespace_cap: usize,
+        _total_cap: usize,
+    ) -> Result<Vec<RootSummary>, MemoryError> {
+        Err(MemoryError::unsupported(Capability::Tree))
+    }
 }

crates/tinymemory-api/src/provider/mod.rs

@@ -90,9 +90,13 @@ pub use tinymemory_bus::provider::{diagnosis, types};
 
 pub use audit::{audit_provider, CapabilityAudit};
 pub use chunks::{
-    ChunkDetail, ChunkEmbedding, ChunkListRow, ChunkQuery, MemoryChunks, SourceTotal,
+    ChunkDetail, ChunkEmbedding, ChunkListRow, ChunkQuery, ChunkScore, ChunkScoreSignals,
+    MemoryChunks, SourceIngestQuery, SourceIngestStatus, SourceTotal, DEFAULT_DROP_THRESHOLD,
+};
+pub use content::{
+    MemoryDocuments, MemoryIngest, MemoryTree, RootSummary, SummaryContext, SummaryInput,
+    SummaryOutput,
 };
-pub use content::{MemoryDocuments, MemoryIngest, MemoryTree};
 pub use diagnosis::{
     DegradedCapabilities, Diagnosis, DiagnosisCounters, DiagnosisFailure, DiagnosisStage,
 };

crates/tinymemory-api/src/provider/records.rs

@@ -19,7 +19,7 @@ use async_trait::async_trait;
 use crate::capabilities::Capability;
 use crate::error::MemoryError;
 use crate::goals::GoalsDoc;
-use crate::provider::diagnosis::Diagnosis;
+use crate::provider::diagnosis::{DegradedCapabilities, Diagnosis};
 use crate::provider::types::{
     FlushOutcome, ForgetOutcome, ForgetSelector, IngestOutcome, MaintenanceReport, PurgeOutcome,
     QueueFailure, QueueStats, ResetOutcome, SourceItem, StoreStats,
@@ -470,4 +470,54 @@ pub trait MemoryMaintenance: Send + Sync {
     async fn diagnose(&self) -> Result<Diagnosis, MemoryError> {
         Err(MemoryError::unsupported(Capability::Maintenance))
     }
+
+    /// Which capabilities are currently running in a reduced mode.
+    ///
+    /// The degradation flags on their own: semantic recall fallen back to
+    /// recency, extraction producing no structure, the storage path unusable —
+    /// and the cause of the most severe of those, when the driver knows it.
+    ///
+    /// # Why this is not [`Self::diagnose`] with the rest thrown away
+    ///
+    /// Cost, and the difference is not marginal. A [`Diagnosis`] is a full
+    /// pass: it counts chunks, counts jobs in three states, measures extraction
+    /// coverage over the whole store, and inspects the configuration of every
+    /// pipeline stage. This is a read of flags the pipeline sets as it runs —
+    /// no query, no configuration walk, nothing that touches storage.
+    ///
+    /// That matters because of who calls it. A diagnosis is asked for once,
+    /// deliberately, by someone looking at a problem. Degradation is polled: it
+    /// is what a status indicator shows continuously, and driving that from a
+    /// full pass would put an aggregate query over the chunk table on a
+    /// repeating timer. The two members exist so a caller can ask the cheap
+    /// question without paying for the expensive one — and, just as important,
+    /// so it is not tempted to poll the expensive one and cache the answer,
+    /// which is how a status light ends up reporting a degradation that cleared
+    /// minutes ago.
+    ///
+    /// [`Diagnosis::degraded`] carries the same shape, from the same source, so
+    /// a caller that has just run a diagnosis has no reason to call this too.
+    ///
+    /// # Why a caller cannot compute it
+    ///
+    /// The flags are set inside the driver, by the embed and extract stages, as
+    /// they fail. Nothing observable from outside distinguishes a recall that
+    /// ranked semantically from one that fell back to recency — both return
+    /// rows, in an order, with no marker on them. A caller with no engine would
+    /// report an all-clear, which is not a stale answer but a confidently wrong
+    /// one.
+    ///
+    /// # Errors
+    ///
+    /// [`MemoryError::Unsupported`] from a driver that tracks no degradation
+    /// state. Deliberately not defaulted to
+    /// [`DegradedCapabilities::default()`], which is all-clear: a driver that
+    /// has never looked would report that everything is fine, and the whole
+    /// purpose of this member is to be believed when it says that.
+    ///
+    /// Otherwise backend failures — though a driver reading in-process flags
+    /// has no failure path and should not invent one.
+    async fn degraded_state(&self) -> Result<DegradedCapabilities, MemoryError> {
+        Err(MemoryError::unsupported(Capability::Maintenance))
+    }
 }

crates/tinymemory-bus/src/lib.rs

@@ -2,7 +2,7 @@
 //! the members that carry them.
 //!
 //! TinyMemory ships as a loadable `TinyBus` module: `crates/tinymemory-module`
-//! exports one object with 123 members on it, built as a `cdylib`. A host that
+//! exports one object (`tinymemory_bus::METHODS.len()` members) built as a `cdylib`. A host that
 //! loads it — OpenHuman — can call into it but cannot `use` anything out of it,
 //! so the payload vocabulary has to be published as an ordinary library. This
 //! is that library.