UniRAG

A local, modular retrieval-augmented generation system built from first principles using Ollama, NumPy, PyMuPDF4LLM, and Streamlit.

UniRAG runs entirely on your computer: documents, embeddings, retrieval, and answer generation remain local.

Features

• Upload and index PDF documents
• Switch between multiple indexed documents
• Local embedding and answer generation through Ollama
• PDF-to-Markdown conversion
• Overlapping text chunking
• NumPy cosine-similarity search
• Persistent document indexes
• Streamed, context-grounded answers
• Chunk-level citations
• Web and command-line interfaces
• No LangChain or external API required

Architecture

Document indexing

PDF
  → Markdown conversion
  → overlapping chunks
  → embedding model
  → normalized vectors
  → persistent NumPy vector store

Question answering

User question
  → query embedding
  → cosine-similarity search
  → top-k document chunks
  → context assembly
  → local Qwen model
  → streamed answer with citations

The query itself is embedded but not chunked. Its embedding is compared against all stored document-chunk embeddings.

Project structure

UniRAG/
├── cli/
│   ├── __init__.py
│   ├── build_index.py
│   └── chat.py
├── app.py
├── loader.py
├── chunker.py
├── embeddings.py
├── vector_store.py
├── requirements.txt
├── README.md
└── .gitignore

Core components

• loader.py: converts PDFs into Markdown.
• chunker.py: divides Markdown into overlapping chunks.
• embeddings.py: generates normalized embeddings through Ollama.
• vector_store.py: stores vectors and performs cosine-similarity search.
• app.py: provides the Streamlit web interface.
• cli/build_index.py: builds a document index from the terminal.
• cli/chat.py: queries a saved index from the terminal.

Requirements

• Python 3.11 or newer
• Ollama
• Approximately 8 GB of available memory at minimum
• Approximately 16 GB of RAM recommended for comfortably running qwen3.5:9b

A smaller Ollama generation model can be configured on systems with less memory.

Installation

Clone the repository:

git clone https://github.com/Aditya12340/UniRAG.git
cd UniRAG

Create a virtual environment:

python3 -m venv .venv

Activate it on macOS or Linux:

source .venv/bin/activate

Activate it on Windows:

.venv\Scripts\activate

Install the Python dependencies:

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Download the default Ollama models:

ollama pull embeddinggemma
ollama pull qwen3.5:9b

Web application

The Streamlit application provides:

• PDF uploading
• automatic document indexing
• a dropdown for selecting an indexed document
• a chat interface for asking questions
• streamed answers with chunk citations

1. Start Ollama

In one terminal:

ollama serve

Leave this terminal running.

If the Ollama desktop application is already running, you may not need to run ollama serve.

2. Start UniRAG

In another terminal:

source .venv/bin/activate
python -m streamlit run app.py

Open the following address if it does not open automatically:

http://localhost:8501

3. Use the application

1. Upload a PDF from the sidebar.
2. Press Add document.
3. Wait for the document to be converted, chunked, and embedded.
4. Select the document from the dropdown.
5. Ask questions using the chat input.

Documents are indexed only once. Later questions load the saved vector index rather than converting and embedding the PDF again.

Uploaded PDFs and generated indexes are stored locally under app_data/ and are excluded from Git.

Command-line interface

The CLI provides the same underlying RAG functionality without Streamlit.

Run all CLI commands from the project root.

Build an index

python -m cli.build_index "/path/to/document.pdf"

By default, the index is saved to:

vector_data/

To select another output directory:

python -m cli.build_index "/path/to/document.pdf" \
  --output my_index

Query the saved index

For the default vector_data/ index:

python -m cli.chat

Enter questions at the prompt and type exit or quit to stop.

The command-line workflow is:

python -m cli.build_index document.pdf
python -m cli.chat

The indexing command only needs to be rerun when adding a different document, changing the embedding model, or rebuilding the index.

Model configuration

The default models are:

Embedding model: embeddinggemma
Generation model: qwen3.5:9b

They can be changed with environment variables:

UNIRAG_EMBEDDING_MODEL=embeddinggemma \
UNIRAG_CHAT_MODEL=qwen:latest \
python -m streamlit run app.py

Available configuration variables:

Variable	Default	Purpose
UNIRAG_OLLAMA_HOST	http://localhost:11434	Ollama server address
UNIRAG_EMBEDDING_MODEL	embeddinggemma	Document and query embedding model
UNIRAG_CHAT_MODEL	qwen3.5:9b	Answer-generation model
UNIRAG_TOP_K	8	Number of chunks retrieved per question

If the embedding model is changed, existing documents must be re-indexed because embeddings from different models cannot be compared.

Local data

The following directories contain local or generated data and are not committed:

.venv/
app_data/
pdfs/
vector_data/

Do not commit private course documents, textbooks, generated embeddings, or uploaded PDFs.

Current limitations

• Only PDF documents are supported.
• Image-only or scanned PDFs may require OCR.
• Complex equations and diagrams may not always convert cleanly to Markdown.
• Retrieval currently uses dense cosine similarity without keyword search or reranking.
• Citations refer to chunk numbers rather than original PDF page numbers.

Future improvements

• Page-number citations
• OCR support
• Hybrid semantic and keyword retrieval
• Optional reranking
• Document deletion and index management
• Configurable chunking from the interface
• Automated retrieval evaluation
• Packaged macOS launcher