Adaptive RAG Engine

A RAG system that dynamically selects its retrieval strategy based on query analysis. Instead of using the same search approach for every question, it classifies each query and routes it to the optimal retrieval pipeline.

"What is the rate for Toptal?"           → Direct Retrieval (fast, precise)
"Explain the rate scaling strategy"      → Analytical (broader context, more depth)
"What skills do I need for Phase 4?"     → Multi-Hop (decompose → parallel search → merge)
"What's the weather today?"              → Refusal (knows when it can't answer)

Architecture

User Query
    │
    ▼
┌──────────────┐
│ Query        │ Classifies: factual, analytical, multi_hop, comparative, unanswerable
│ Analyzer     │ Extracts entities, estimates complexity, generates sub-queries
└──────┬───────┘
       │
       ▼
┌──────────────┐
│ Strategy     │ Routes to optimal retrieval pipeline
│ Router       │
└──┬───┬───┬───┘
   │   │   │
   ▼   ▼   ▼
┌─────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐
│Direct│ │Analytical│ │Multi-Hop│ │ Refusal │
│      │ │          │ │         │ │         │
│single│ │ broader  │ │decompose│ │ "I don't│
│search│ │ context  │ │→ N srch │ │  know"  │
│      │ │          │ │→ merge  │ │         │
└──┬───┘ └────┬─────┘ └────┬────┘ └────┬────┘
   │          │             │            │
   └──────────┴──────┬──────┘            │
                     ▼                   │
              ┌──────────────┐           │
              │ Hybrid Search│           │
              │ Vector + BM25│           │
              │ + Reranking  │           │
              └──────┬───────┘           │
                     │                   │
                     ▼                   ▼
              ┌──────────────┐    ┌──────────┐
              │  Generator   │    │ Refusal  │
              │ Claude API   │    │ Response │
              │ + Citations  │    └──────────┘
              └──────┬───────┘
                     │
                     ▼
              ┌──────────────┐
              │ Self-Eval    │
              │ Faithfulness │
              │ Relevancy    │
              │ Completeness │
              └──────┬───────┘
                     │
                     ▼
              Response + Citations + Quality Scores

Quick Start

1. Clone and configure

git clone https://github.com/OriginalKazdov/adaptive-rag.git
cd adaptive-rag
cp .env.example .env
# Edit .env with your API keys

2. Run with Docker Compose

docker-compose up

• Frontend: http://localhost
• API docs: http://localhost:8000/docs
• Qdrant dashboard: http://localhost:6333/dashboard

3. Ingest documents

# Via API
curl -X POST http://localhost:8000/ingest \
  -H "Content-Type: application/json" \
  -d '{"source": "path/to/doc.md", "source_type": "markdown"}'

# Via CLI
python -m scripts.ingest --source doc.pdf --type pdf

4. Ask questions

curl -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"query": "What are the freelance rates by platform?"}'

Run without Docker (development)

# Backend
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
docker run -d -p 6333:6333 qdrant/qdrant  # just Qdrant
uvicorn src.api:app --reload

# Frontend
cd frontend && npm install && npm run dev

Run tests

pip install -r requirements.txt
pytest tests/ -v

All tests run offline (no API keys or Qdrant needed).

Key Design Decisions

Decision	Why
Adaptive strategy selection	One retrieval approach doesn't fit all query types. Factual questions need precision; analytical questions need breadth; multi-hop questions need decomposition.
Hybrid search (Vector + BM25)	Vector search understands semantics but misses exact terms. BM25 finds exact keywords but misses synonyms. Combining both consistently outperforms either alone.
Reciprocal Rank Fusion	Merges rankings without comparing incompatible scores. Uses position only, not scores. Robust and parameter-free.
Self-evaluation	Every response is automatically scored for faithfulness, relevancy, and completeness. In production, you can't improve what you don't measure.
Refusal path	The system explicitly says "I don't know" when it can't answer. A RAG that refuses when uncertain is more trustworthy than one that always guesses.
Semantic chunking	Documents are split by structure (headers, sections), not fixed token counts. This preserves context within chunks and improves retrieval quality.

Tech Stack

• Backend: Python 3.12, FastAPI
• LLM: Anthropic Claude API (generation + analysis)
• Embeddings: OpenAI text-embedding-3-small
• Vector DB: Qdrant
• Keyword Search: BM25 (rank_bm25)
• Reranking: Cohere Rerank / cross-encoder / passthrough
• Evaluation: RAGAS-style metrics + heuristic evaluator
• Frontend: React 18, Tailwind CSS, Recharts
• Infra: Docker Compose

API Endpoints

Method	Endpoint	Description
POST	/query	Ask a question. Returns answer + citations + scores
POST	/ingest	Ingest a document (path or URL)
POST	/ingest/upload	Upload a file (PDF/Markdown)
GET	/metrics	Aggregated quality metrics
GET	/history	Query history with scores
GET	/health	Health check
GET	/docs	Interactive API documentation (Swagger)

Project Structure

adaptive-rag/
├── src/
│   ├── api.py              # FastAPI endpoints
│   ├── rag.py              # Main orchestrator: analyze → route → generate → evaluate
│   ├── config.py           # Settings from .env
│   ├── ingestion/
│   │   ├── parsers.py      # PDF, URL, Markdown parsers
│   │   ├── chunker.py      # Semantic chunking (structure-aware)
│   │   └── pipeline.py     # Ingestion orchestrator
│   ├── storage/
│   │   ├── vector_store.py # Qdrant interface
│   │   └── bm25_store.py   # BM25 keyword index
│   ├── search/
│   │   ├── hybrid.py       # Hybrid search + Reciprocal Rank Fusion
│   │   └── reranker.py     # Cohere / cross-encoder / passthrough
│   ├── query/
│   │   ├── models.py       # QueryType, QueryAnalysis
│   │   ├── analyzer.py     # LLM-based + rule-based query classification
│   │   └── strategies.py   # Direct, Analytical, MultiHop, Refusal strategies
│   ├── generation/
│   │   ├── models.py       # RAGResponse, Citation, EvaluationScores
│   │   ├── generator.py    # Claude generator + mock generator
│   │   └── evaluator.py    # LLM evaluator + heuristic evaluator
│   └── models/
│       └── documents.py    # RawDocument, Chunk, StoredChunk
├── tests/                  # 88 tests, all run offline
├── frontend/               # React + Tailwind + Recharts
├── scripts/
│   └── ingest.py           # CLI ingestion tool
├── docker-compose.yml      # Full stack: Qdrant + Backend + Frontend
├── SCOPE.md                # Detailed project scope and phases
└── LEARNINGS.md            # Learning resources for each concept