RAG ChatBot

A Next.js application for chatting with uploaded PDF documents using retrieval-augmented generation. Users can upload documents, index their text into a vector database, and ask questions against that knowledge base through a streaming chat interface.

Features

• Authentication with Clerk-protected routes
• PDF upload flow for indexing document content
• Vector search over embedded document chunks stored in Neon Postgres with pgvector
• Streaming chat powered by the AI SDK and OpenAI models
• Simple landing page with navigation to chat and upload flows

Tech Stack

• Next.js 16 with the App Router
• Clerk for authentication
• OpenAI via the AI SDK
• Neon serverless Postgres
• Drizzle ORM and drizzle-kit
• Tailwind CSS and Biome

Project Structure

• src/app/page.tsx - landing page
• src/app/chat/page.tsx - chat UI
• src/app/upload/page.tsx - PDF upload UI
• src/app/upload/actions.ts - server action that parses PDFs, chunks text, and stores embeddings
• src/app/api/chat/route.ts - streaming chat endpoint with a knowledge-base search tool
• src/lib/db-config.ts - database connection setup
• src/lib/db-schema.ts - document table and vector index definition
• src/lib/search.ts - similarity search over stored embeddings
• src/lib/embeddings.ts - OpenAI embedding helpers
• src/lib/chunking.ts - text chunking logic

Prerequisites

• Node.js 18 or newer
• A Neon Postgres database with the vector extension available
• Clerk application credentials
• OpenAI API access

Environment Variables

Create a .env.local file in the project root with at least the following values:

NEON_DATABASE_URL=your_neon_database_connection_string
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=your_clerk_publishable_key
CLERK_SECRET_KEY=your_clerk_secret_key
OPENAI_API_KEY=your_openai_api_key

Getting Started

Install dependencies:

npm install

Run the development server:

npm run dev

Open http://localhost:3000 in your browser.

Database Setup

Apply the Drizzle migrations to your Neon database before using the app:

npx drizzle-kit push

If your workflow uses generated migration files instead of pushing the schema directly, run the Drizzle command that matches your setup and ensure the documents table exists with the vector index defined in src/lib/db-schema.ts.

Available Scripts

• npm run dev - start the development server
• npm run build - build the production app
• npm run start - run the production server
• npm run lint - run Biome checks
• npm run format - format the codebase with Biome

How It Works

1. A signed-in user uploads a PDF from the upload page.
2. The server action extracts text, splits it into chunks, and generates embeddings with OpenAI.
3. The chunks and embeddings are stored in Neon Postgres.
4. When the user asks a question, the chat route searches the embedded document chunks for relevant context.
5. The model answers using the search results and streams the response back to the UI.

Notes

• The app uses Clerk middleware to protect non-public routes.
• The chat endpoint is intentionally constrained to keep answers concise and grounded in retrieved context.
• Metadata in the root layout can be updated if you want a project-specific title and description.