Skip to content

Nothing re-summarises segments left unsummarised by a failed recap, and the marker is invisible over the contract #6186

Description

@YellowSnnowmann

Summary

#6156 stopped the archivist from persisting, embedding and goal-enriching a heuristic recap when the LLM summariser fails (PR #6183). A segment whose recap failed is now left status='closed' with a NULL summary, which is deliberately the needs_resummary marker — it is exactly what segments_pending_summary (WHERE status='closed') already selects for.

Two things are still missing for that marker to be worth anything.

1. The marker is invisible over the contract

tinymemory-core distinguishes three segment states — 'open', 'closed', 'summarised' (store/namespace_store/segments.rs:64). The contract DTO does not:

// tinymemory-bus/src/provider/episodic.rs:81
pub struct ConversationSegment {
    ...
    pub summary: Option<String>,
    // no status field
}

and the engine collapses it on the way out:

// tinymemory-tinycortex/src/engine/mod.rs:4778
open: matches!(segment.status, SegmentStatus::Open),

So 'closed' and 'summarised' both arrive at a host as open: false. A host-side pass cannot tell an unsummarised segment from a summarised one, which means it cannot find the segments that need re-summarising.

Either the DTO grows a field (a tinymemory contract change — release, re-pin, forwarder, per the module release-pin gate), or the pass runs inside the engine.

2. Nothing ever runs the queue

segments_pending_summary (segments.rs:442) has zero production callers — only segments_tests.rs. It is a correct query with no scheduler behind it. Until something drives it, a segment left 'closed' stays that way forever; the raw turns survive, so nothing is lost, but the quality is never recovered either.

Expected behaviour

  • A background pass that picks up status='closed' segments and re-runs summarize_entries against them once a summariser answers, writing through set_segment_summary on success only (same rule as the finalize path).
  • Bounded: a batch cap and a floor on retry age, so a permanently-unsummarisable segment is not retried forever.
  • Whichever side it lands on, decide (1) first — the placement of the pass follows from whether the host can see the marker.

Notes

  • Retry-with-backoff inside summarize_entries was considered and declined for LLM recap fails silently and falls back to heuristic summarisation when provider is unreachable #6156. Worth restating why: segment finalize is off the user-visible turn path (post-turn hooks are tokio::spawned detached, agent/hooks.rs:297), but flush_open_segment is awaited unbounded at session wind-down (session/turn/session_io_impl_01_part_02.rs:294), so an inline backoff loop adds latency exactly in the failure case. A durable retry — this issue — is the better shape.
  • segment_embeddings is currently a write-only table (segment_embedding_get has no production callers), so re-summarising has no recall effect until something reads it. That is a separate gap, but it means this work should be sequenced behind, or together with, a reader.

Follow-up to #6156 / #6183.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

memoryMemory store, memory tree, recall, summarization, and embeddings in src/openhuman/memory/.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions