RAG Document Processing System

This project implements a Retrieval-Augmented Generation (RAG) system that allows users to upload PDF files and images, extract text using advanced OCR with layout detection, and then query the contents using Claude. The system combines PaddleOCR for document understanding, LayoutLMv3 for reading order, and Claude VLM for visual element analysis.

Description

The RAG Document Processing System is designed to:

1. Extract Text from Documents: Utilize PaddleOCR to accurately extract text from PDFs and images with layout awareness.
2. Detect Document Structure: Identify tables, figures, forms, and text regions using layout detection.
3. Determine Reading Order: Use LayoutLMv3 to determine the natural reading sequence of content.
4. Analyze Visual Elements: Use Claude VLM to extract content from tables, charts, and forms.
5. Create a Knowledge Base: Store document chunks in Pinecone vector database for semantic search.
6. Generate Responses: Use Claude to generate context-aware responses to user queries.

Key Components and Technologies Used

• PaddleOCR: For text extraction and layout detection from documents.
• LayoutLMv3: For determining natural reading order of text regions.
• Claude VLM: For analyzing tables, charts, figures, and forms.
• Pinecone: Vector database for storing and retrieving document embeddings.
• sentence-transformers: For generating text embeddings locally (free, no API needed).
• Claude: For generating intelligent responses to user queries.
• Click: For building the command-line interface.

Why This Project?

• Advanced Document Understanding: Goes beyond simple text extraction to understand document structure.
• Visual Element Analysis: Tables, charts, and forms are analyzed using vision-language models.
• Natural Reading Order: Content is extracted in the order humans would naturally read it.
• Fast Mode Available: Skip OCR for text-heavy PDFs and extract directly for speed.
• Interactive Q&A: Ask questions about your documents and get accurate, sourced answers.

How to Run

Prerequisites

• Python 3.10 or higher
• Anthropic API Key (for Claude)
• Pinecone API Key (for vector storage)

Installation

1. Clone the repository:

   git clone https://github.com/your-username/rag-document-processing.git
   cd rag-document-processing

2. Create a virtual environment and activate it:

   python -m venv .venv
   source .venv/bin/activate  # On Windows use `.venv\Scripts\activate`

3. Install the required packages:

   pip install -r requirements.txt

4. Create a .env file in the project root directory:

   ANTHROPIC_API_KEY=your_anthropic_api_key_here
   PINECONE_API_KEY=your_pinecone_api_key_here
   

Running the Application

1. Quick Start - Full Pipeline (recommended):

   # For text-heavy PDFs (fast mode)
   python rag_system.py pipeline document.pdf --fast -n my_docs
   
   # For documents with tables/charts (uses VLM)
   python rag_system.py pipeline document.pdf -n my_docs

2. Step-by-Step Usage:

   # Step 1: Extract content from document
   python rag_document_extractor.py extract report.pdf -o ./output
   
   # Step 2: Index chunks into Pinecone
   python rag_system.py index ./output/chunks.json -n my_docs
   
   # Step 3: Ask questions
   python rag_system.py ask "What is the main topic?" -n my_docs
   
   # Or start interactive chat
   python rag_system.py chat -n my_docs

3. Available Commands:

   Document Extractor:

   python rag_document_extractor.py extract <file> -o <output>  # Extract document
   python rag_document_extractor.py extract <file> --no-vlm     # Without VLM (faster)
   python rag_document_extractor.py test report                 # Test with sample
   python rag_document_extractor.py list                        # List test documents

   RAG System:

   python rag_system.py pipeline <file> --fast    # Full pipeline (fast mode)
   python rag_system.py index <chunks.json>       # Index chunks
   python rag_system.py ask "question"            # Ask single question
   python rag_system.py chat                      # Interactive chat
   python rag_system.py stats                     # View index stats

Notes

• Use --fast flag for text-heavy PDFs (scripts, articles) - processes 100+ pages in seconds.
• Use default mode (without --fast) for documents with tables, charts, or forms.
• The --no-vlm flag disables visual analysis for faster processing.
• Namespaces (-n) help organize different documents in Pinecone.

Project Structure

rag-document-processing/
├── rag_document_extractor.py   # Document extraction pipeline
├── rag_system.py               # RAG system (embed, index, Q&A)
├── requirements.txt            # Python dependencies
├── .env                        # API keys (create this file)
├── .gitignore                  # Git ignore rules
└── README.md                   # This file

License

MIT License