Enterprise Knowledge Bot — AI-powered document Q&A via Zalo.
- 📚 Knowledge Base: Upload PDF, DOCX → auto-extract & embed
- 🤖 Google Gemini AI: Chat, RAG, summarization (text-embedding-004)
- 🖼️ Image Extraction: Auto-extract images from documents, send with answers
- 📇 Contacts: Admin manages contacts, AI suggests when info not found
- 🗄️ MinIO Storage: S3-compatible file storage with presigned URLs
- 🔍 Semantic Search: pgvector-powered similarity search
- ⚡ arq Worker Queue: Redis-backed async ingestion with real-time progress tracking
┌─────────────┐ ┌───────────┐ ┌───────────┐
│ Next.js 15 │────▶│ FastAPI │────▶│ PostgreSQL│
│ Admin UI │◀────│ API │ │ + pgvector│
└─────────────┘ └─────┬─────┘ └───────────┘
│
┌─────▼─────┐ ┌───────────┐
│ Redis │────▶│ arq Worker│
│ Queue │◀────│ (process) │
└───────────┘ └─────┬─────┘
│
┌─────▼─────┐
│ MinIO │
│ Storage │
└───────────┘
Ingestion Pipeline: API enqueues job → Redis → arq Worker picks up → Extract text → Chunk → Embed → Store → Update progress in DB → Frontend auto-polls progress.
| Layer | Tech |
|---|---|
| API Backend | FastAPI (Python 3.12) |
| Vector Database | PostgreSQL 16 + pgvector |
| Graph Database | Neo4j (Knowledge Graph) |
| AI & Vision | Gemini 3.1 Flash-Lite (Multimodal RAG) + gemini-embedding-2 |
| Task Queue | arq + Redis |
| Storage | MinIO (S3-compatible) |
| Frontend | Next.js 15 + Tailwind CSS + shadcn/ui |
| Deployment | Docker + Docker Compose |
- Python 3.11+
- Node.js 18+
- PostgreSQL 16+ with pgvector extension
- MinIO server
- Redis server
# 1. Copy & edit env file
cp .env.example .env
# → Fill in DATABASE_URL, GOOGLE_API_KEY, NEO4J_URI, NEXT_PUBLIC_API_URL, etc.
# 2. Build and start all services
docker compose up -d --build# 1. Copy & edit env file
cp .env.example .env
# 2. Install backend dependencies & run migrations
python -m venv .venv
source .venv/bin/activate
pip install -e .
alembic upgrade head
# 3. Start API (Terminal 1)
python -m uvicorn app.main:app --reload --port 5055
# 4. Start arq Worker (Terminal 2)
python -m arq app.worker.WorkerSettings
# 5. Start Frontend (Terminal 3)
cd frontend
npm install
npm run dev# --- PostgreSQL ---
DATABASE_URL=postgresql+asyncpg://user:pass@host:5432/dbname
# --- Google AI ---
GOOGLE_API_KEY=your-api-key
# --- MinIO ---
MINIO_ENDPOINT=host:9000
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=your-secret
MINIO_BUCKET=kb-files
MINIO_SECURE=false
# --- Redis (arq worker queue) ---
REDIS_HOST=127.0.0.1
REDIS_PORT=6379
REDIS_PASSWORD=your-password
REDIS_DB=0
# --- Embedding ---
EMBEDDING_MODEL=text-embedding-004
EMBEDDING_DIMENSIONS=768
CHUNK_SIZE=1500
CHUNK_OVERLAP=150bitsbooklm/
├── app/
│ ├── main.py # FastAPI entry
│ ├── config.py # Settings from .env
│ ├── worker.py # arq worker — ingestion tasks
│ ├── database/ # SQLAlchemy + pgvector
│ ├── routers/ # API endpoints
│ ├── services/ # Business logic (KB, MinIO, images)
│ ├── ai/ # Gemini integration
│ └── channels/ # Zalo adapter + session manager
├── alembic/ # DB migrations
├── frontend/ # Next.js 15 admin UI (shadcn/ui)
└── pyproject.toml
The arq worker runs as a separate process consuming jobs from Redis:
ingest_file_task: Upload → Extract text → Extract images → Chunk → Embed → Store → Summarizeingest_url_task: Fetch URL → Extract text → Chunk → Embed → Store → Summarize- Progress tracking: 0–100% with messages, auto-polled by frontend every 2s
- Retry policy: max 3 attempts, 10s delay between retries
- Max concurrent jobs: 3 (configurable via
WORKER_MAX_JOBS)