You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#6156 stopped the archivist from persisting, embedding and goal-enriching a heuristic recap when the LLM summariser fails (PR #6183). A segment whose recap failed is now left status='closed' with a NULL summary, which is deliberately the needs_resummary marker — it is exactly what segments_pending_summary (WHERE status='closed') already selects for.
Two things are still missing for that marker to be worth anything.
1. The marker is invisible over the contract
tinymemory-core distinguishes three segment states — 'open', 'closed', 'summarised' (store/namespace_store/segments.rs:64). The contract DTO does not:
// tinymemory-bus/src/provider/episodic.rs:81pubstructConversationSegment{
...
pubsummary:Option<String>,// no status field}
So 'closed' and 'summarised' both arrive at a host as open: false. A host-side pass cannot tell an unsummarised segment from a summarised one, which means it cannot find the segments that need re-summarising.
Either the DTO grows a field (a tinymemory contract change — release, re-pin, forwarder, per the module release-pin gate), or the pass runs inside the engine.
2. Nothing ever runs the queue
segments_pending_summary (segments.rs:442) has zero production callers — only segments_tests.rs. It is a correct query with no scheduler behind it. Until something drives it, a segment left 'closed' stays that way forever; the raw turns survive, so nothing is lost, but the quality is never recovered either.
Expected behaviour
A background pass that picks up status='closed' segments and re-runs summarize_entries against them once a summariser answers, writing through set_segment_summary on success only (same rule as the finalize path).
Bounded: a batch cap and a floor on retry age, so a permanently-unsummarisable segment is not retried forever.
Whichever side it lands on, decide (1) first — the placement of the pass follows from whether the host can see the marker.
Notes
Retry-with-backoff inside summarize_entries was considered and declined for LLM recap fails silently and falls back to heuristic summarisation when provider is unreachable #6156. Worth restating why: segment finalize is off the user-visible turn path (post-turn hooks are tokio::spawned detached, agent/hooks.rs:297), but flush_open_segmentis awaited unbounded at session wind-down (session/turn/session_io_impl_01_part_02.rs:294), so an inline backoff loop adds latency exactly in the failure case. A durable retry — this issue — is the better shape.
segment_embeddings is currently a write-only table (segment_embedding_get has no production callers), so re-summarising has no recall effect until something reads it. That is a separate gap, but it means this work should be sequenced behind, or together with, a reader.
Summary
#6156 stopped the archivist from persisting, embedding and goal-enriching a heuristic recap when the LLM summariser fails (PR #6183). A segment whose recap failed is now left
status='closed'with a NULL summary, which is deliberately theneeds_resummarymarker — it is exactly whatsegments_pending_summary(WHERE status='closed') already selects for.Two things are still missing for that marker to be worth anything.
1. The marker is invisible over the contract
tinymemory-coredistinguishes three segment states —'open','closed','summarised'(store/namespace_store/segments.rs:64). The contract DTO does not:and the engine collapses it on the way out:
So
'closed'and'summarised'both arrive at a host asopen: false. A host-side pass cannot tell an unsummarised segment from a summarised one, which means it cannot find the segments that need re-summarising.Either the DTO grows a field (a
tinymemorycontract change — release, re-pin, forwarder, per the module release-pin gate), or the pass runs inside the engine.2. Nothing ever runs the queue
segments_pending_summary(segments.rs:442) has zero production callers — onlysegments_tests.rs. It is a correct query with no scheduler behind it. Until something drives it, a segment left'closed'stays that way forever; the raw turns survive, so nothing is lost, but the quality is never recovered either.Expected behaviour
status='closed'segments and re-runssummarize_entriesagainst them once a summariser answers, writing throughset_segment_summaryon success only (same rule as the finalize path).Notes
summarize_entrieswas considered and declined for LLM recap fails silently and falls back to heuristic summarisation when provider is unreachable #6156. Worth restating why: segment finalize is off the user-visible turn path (post-turn hooks aretokio::spawned detached,agent/hooks.rs:297), butflush_open_segmentis awaited unbounded at session wind-down (session/turn/session_io_impl_01_part_02.rs:294), so an inline backoff loop adds latency exactly in the failure case. A durable retry — this issue — is the better shape.segment_embeddingsis currently a write-only table (segment_embedding_gethas no production callers), so re-summarising has no recall effect until something reads it. That is a separate gap, but it means this work should be sequenced behind, or together with, a reader.Follow-up to #6156 / #6183.