feat: show vectorization status of memories + stats count + force re-vectorize#140
Merged
rahilp merged 12 commits intoJun 9, 2026
Conversation
Add configurable grace window for distinguishing in-flight embeddings from permanent failures, enabling proper vectorization status tracking.
Adds vector_ids field to the SELECT clause in buildEntryFilterQuery so /list endpoint includes vectorization status for each entry. This enables the frontend to compute the badge state for the recent card (Task 5).
Adds endpoint to re-embed memories that silently failed vectorization (vector_ids still "[]" past the grace window), processing up to 25 entries per call and returning processed/failed/remaining counts for UI looping.
- Add error logging in the /vectorize-pending catch block to catch silent errors - Assert remaining count in partial failure test when entry lacks vectors
…rors Add two safeguards to runVectorize: 1. Check res.ok and throw on 4xx/5xx to surface HTTP errors instead of silently failing 2. Break the loop if a batch processes zero entries while remaining > 0 to prevent infinite spinning during Worker AI outages This prevents infinite loops when entries keep failing to vectorize and ensures error states are properly surfaced to the user.
Coverage Report
File Coverage
|
||||||||||||||||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
unvectorizedcount andvectorize_grace_msto/stats, and exposesvector_idson/listresponsesPOST /vectorize-pendingendpoint that re-embeds up to 25 past-grace unvectorized entries per call, returning{ processed, failed, remaining }/vectorize-pendinguntil all failed memories are re-indexed, then refreshes the UIHow it works
vector_ids = '[]'is the de-facto failure signal — it's permanent whenstoreEntrythrows silently insidectx.waitUntil. A configurable grace window (VECTORIZE_GRACE_MS, default 5 min) distinguishes in-flight embeds from actual failures, so only past-grace rows are counted as failed and eligible for re-embedding.Test Plan
UPDATE entries SET vector_ids='[]', created_at=created_at-600000) → card shows Not indexed/vectorize-pendingand not counted inunvectorizednpm test→ 285/285 passing