As a reminder, in multivector mode, here is the flow for a DB you've saved in the past and are revisiting:
- Chatbot loaded with RagSettings object in the config. If rag_settings.rag_mode is True, we do the following rag magic!
- Chatbot.intialize_rag() is called
- This has a bunch of internal mutations, but what's important is that it's the child-docs that get loaded into the vectorstore that your retriever references. What this means, practically, is that if you have a textbook as your source material, and you've made a summary for every chapter, it's the summaries that will have vector embeddings associated with them! At runtime, this is cheaper, faster, and for use cases that involve retrieving the parent doc (The actual pdf chapter), works really well with downstream processing!
- Up until now, we're doing stellar, but but the problem is, since we've just began a new session, our parent docs were previously loaded into memory, associated with our child_docs, and discarded when the session ended.
This is okay almost all the time, and works perfectly with local documents, but the tradeoff of time spent re-indexing at runtime vs. space on disk is not a great one. I'd definitely prefer storing the chunked parent documents on disk and have an option to easily disable it with a config setting.
Solution:
- When the parent documents are chunked, they should be stored in an sqlite or something easy to pull from the next time they need to be associated with a child doc.
As a reminder, in multivector mode, here is the flow for a DB you've saved in the past and are revisiting:
This is okay almost all the time, and works perfectly with local documents, but the tradeoff of time spent re-indexing at runtime vs. space on disk is not a great one. I'd definitely prefer storing the chunked parent documents on disk and have an option to easily disable it with a config setting.
Solution: