A full-stack Retrieval-Augmented Generation (RAG) system with per-tenant data isolation, multilingual support, JWT authentication, and a no-build-step browser frontend.
- Features
- Tech stack
- Project layout
- Setup
- Running the app
- Using the frontend
- API reference
- Auth & token flow
- Prompt templates
- CSV ingestion format
- Per-tenant quotas
- Security model
- Management commands
- Troubleshooting
| Feature | Detail |
|---|---|
| Multilingual embeddings | intfloat/multilingual-e5-base — 100+ languages, 768-dim, runs locally (no API key) |
| Multi-tenant isolation | Every query, chunk, and upload is scoped to a tenant. Tenant ID comes from the JWT — never from request input |
| Gemini generation | gemini-2.5-flash for grounded answers; automatic retry with backoff on rate limits |
| 6 prompt templates | qa · summarize · translate · glossary · extract · explain |
| Self-service registration | Email-domain-based automatic tenant assignment |
| Per-tenant quotas | Configurable api_quota; atomic enforcement; monthly reset command |
| Plain HTML/CSS/JS frontend | Zero build step — Django serves the SPA directly |
| JWT auto-refresh | Access token silently refreshed on 401; full logout on second failure |
| Idempotent ingestion | Re-uploading the same CSV filename replaces (not duplicates) previous chunks |
Backend
- Python 3.11+ / Django 6 / Django REST Framework
- PostgreSQL 15 + pgvector (vector similarity search)
sentence-transformers— local multilingual-e5-base embeddingsgoogle-genai— Gemini 2.5 Flash generationdjangorestframework-simplejwt— JWT auth with custom tenant claimsdrf-spectacular— OpenAPI / Swagger docs
Frontend
- Plain HTML5 + CSS + vanilla JavaScript (no framework, no build step)
- Served directly by Django's template + static file system
multilingual-multitenant-rag/
├── core/
│ ├── settings.py # Django config, CORS, REST_FRAMEWORK
│ └── urls.py # Mounts /api/, /admin/, /api/docs/, / (SPA)
├── documents/
│ ├── models.py # Tenant, DocumentChunk, UserProfile
│ ├── auth.py # TenantTokenView — bakes tenant into JWT
│ ├── views.py # QueryView, IngestView, HealthView, UsageView, RegisterView
│ ├── urls.py # API URL table
│ ├── admin.py # Admin registrations
│ ├── services/
│ │ ├── embeddings.py # embed_document, embed_query, embed_documents_batch
│ │ ├── retrieval.py # retrieve_chunks (tenant-filtered cosine search)
│ │ ├── ingestion.py # ingest_csv (validate, idempotent, batch)
│ │ ├── rag.py # rag_query (retrieve → prompt → generate)
│ │ └── prompts.py # 6 named templates + build_prompt()
│ ├── migrations/ # 0001 enable_vector · 0002 initial · 0003 userprofile · 0004 tenant_domain
│ └── management/commands/
│ ├── ingest_csv.py # CLI ingestion
│ └── reset_quotas.py # Monthly quota reset
├── templates/
│ └── index.html # SPA shell served at /
├── static/
│ ├── style.css # Minimal design system
│ └── app.js # All SPA logic (auth, query, upload, usage)
├── .env.example # Environment variable template
├── requirements.txt # Pinned Python dependencies
├── ARCHITECTURE.md # ASCII data-flow diagrams
└── CHANGES.md # Phase 5 change log and trade-off notes
- Python 3.11+
- PostgreSQL 15+ with the
pgvectorextension installed - A Gemini API key
-- run in psql
CREATE DATABASE multilingual_rag;
\c multilingual_rag
CREATE EXTENSION IF NOT EXISTS vector;# Windows PowerShell
python -m venv venv
.\venv\Scripts\Activate.ps1
pip install -r requirements.txtcopy .env.example .envEdit .env and fill in:
SECRET_KEY=<long-random-string>
DEBUG=True
DB_NAME=multilingual_rag
DB_USER=postgres
DB_PASSWORD=<your-db-password>
DB_HOST=localhost
DB_PORT=5432
GEMINI_API_KEY=<your-gemini-key>
CORS_ALLOW_ALL_ORIGINS=Truepython manage.py migratepython manage.py createsuperuser.\venv\Scripts\Activate.ps1
python manage.py runserver| URL | What |
|---|---|
http://localhost:8000/ |
Frontend SPA |
http://localhost:8000/api/docs/ |
Swagger UI |
http://localhost:8000/admin/ |
Django admin |
- Open
http://localhost:8000/ - Click Register to create a new account. Your tenant is assigned automatically based on your email domain — if
alice@acme.comandbob@acme.comregister, they share the sameacme.comtenant. - Or click Login if you already have an account.
Tokens (access + refresh) are stored in localStorage under the keys rag_access and rag_refresh. They persist across page refreshes and browser restarts. Logout clears both keys.
After login you see:
- Header — your tenant name badge and username in the top bar.
- Usage widget — shows how many of your tenant's API calls have been used vs. the quota, with a progress bar.
- Upload panel — pick a
.csvfile, confirm the filename and size, then click Upload. The usage widget refreshes after a successful upload. - Query panel — type a question, pick a prompt template, click Ask. The answer appears with a collapsible Sources section listing the chunks used and their similarity scores.
Every API call that returns 401 automatically:
- Calls
POST /api/auth/refresh/with the stored refresh token. - Saves the new access token to localStorage.
- Retries the original request once.
- If the refresh also fails, clears tokens and redirects to the login screen.
All authenticated endpoints require Authorization: Bearer <access_token>.
Create a user account. Tenant is assigned from the email domain server-side.
Request
{ "username": "alice", "email": "alice@acme.com", "password": "secret123" }Response 201
{
"access": "<jwt>",
"refresh": "<jwt>",
"tenant": "acme.com"
}Errors
| Status | Reason |
|---|---|
| 400 | Missing field / invalid email |
| 400 | Username or email already taken |
Request
{ "username": "alice", "password": "secret123" }Response 200
{ "access": "<jwt>", "refresh": "<jwt>" }Request
{ "refresh": "<refresh_jwt>" }Response 200
{ "access": "<new_access_jwt>" }Request
{
"query": "What is the cancellation policy?",
"template": "qa"
}template is optional and defaults to "qa". See Prompt templates for all values.
Response 200
{
"answer": "Cancellations must be made 48 hours in advance...",
"context": [
{
"content": "Q: What is the cancellation policy?\nA: Cancellations must be made...",
"source": "policies.csv",
"category": "Billing",
"distance": 0.1234
}
]
}Error responses
| Status | Body | Meaning |
|---|---|---|
| 400 | {"error": "query is required"} |
Empty query |
| 400 | {"error": "Unknown template ..."} |
Bad template key |
| 429 | {"error": "Monthly quota exceeded.", "quota": 1000, "used": 1000} |
Tenant quota reached |
Multipart form upload. Field name must be file.
# PowerShell example
$form = @{ file = Get-Item .\data.csv }
Invoke-RestMethod -Uri http://localhost:8000/api/ingest/ `
-Method POST -Form $form `
-Headers @{ Authorization = "Bearer $access" }Response 200
{
"status": "ingested",
"chunks_created": 42,
"rows_skipped": 1,
"tenant": "acme.com"
}Re-uploading the same filename replaces existing chunks for that file (idempotent).
{ "status": "ok", "db": "ok", "chunk_count": 1234 }{
"tenant": "acme.com",
"api_quota": 1000,
"requests_used": 47,
"remaining": 953
}Browser localStorage
rag_access ← access JWT (short-lived, e.g. 5 min)
rag_refresh ← refresh JWT (long-lived, e.g. 1 day)
On every API call
1. Attach Authorization: Bearer <rag_access>
2. If response is 401
→ POST /api/auth/refresh/ with rag_refresh
→ Save new access token to rag_access
→ Retry original request once
3. If retry also 401 → clear both keys → show login screen
On page load
1. Check rag_access exists in localStorage
2. Decode JWT payload (base64) → read exp claim
3. If not expired → enter app directly (no network call)
4. If expired → try refresh silently → enter app or show login
Beyond standard claims, the tokens include:
{ "tenant_id": 7, "tenant_name": "acme.com" }These are baked in at login/register via TenantTokenSerializer and read server-side on every authenticated request. The client can decode them for display but cannot modify them.
All templates enforce the same grounding rules: answer only from the retrieved context; say "I don't know based on the available documents." if the answer is absent; reply in the same language as the question.
| Key | Use case | Example question |
|---|---|---|
qa |
Default grounded Q&A | "What is the return policy?" |
summarize |
Summarise retrieved content | "Summarise the billing section" |
translate |
Translate context to requested language | "Translate the refund terms to French" |
glossary |
Define a term using document terminology | "Define 'chargeback'" |
extract |
Pull a specific fact | "What is the exact deadline for appeals?" |
explain |
Plain-language explanation | "Explain the SLA in simple terms" |
name,category,question,answer
John Smith,Billing,What is the refund window?,Refunds are processed within 14 days.
Jane Doe,Policy,Can I cancel anytime?,You may cancel with 48 hours notice.Required columns: name, category, question, answer
- Rows with missing or empty
question/answerare skipped (counted inrows_skipped). - The header is validated on upload; a missing column raises an error before any rows are processed.
- Re-uploading the same filename deletes and recreates chunks for that file (idempotent).
- All rows are batch-embedded in a single model call for performance.
Each Tenant row has:
api_quota— maximum requests allowed (default1000)requests_used— atomic counter incremented before each generation
When requests_used >= api_quota, POST /api/query/ returns HTTP 429:
{ "error": "Monthly quota exceeded.", "quota": 1000, "used": 1000 }To change a tenant's quota, use Django admin or the shell:
python manage.py shell -c "from documents.models import Tenant; Tenant.objects.filter(name='acme.com').update(api_quota=5000)"To reset all tenants at the start of each month:
python manage.py reset_quotastenant_id is written into the JWT at login by TenantTokenSerializer:
token["tenant_id"] = user.userprofile.tenant_idEvery authenticated view reads it back as:
tenant = request.user.userprofile.tenantNo endpoint accepts a tenant_id parameter in the request body or query string. A user cannot escalate themselves to another tenant by crafting a request.
Every database query for chunks includes the tenant filter:
DocumentChunk.objects.filter(tenant_id=tenant_id)This single line in retrieval.py is the isolation boundary. Removing it would expose all tenants' data to every query.
On POST /api/auth/register/:
- Server extracts
domain = email.split("@")[1] - Looks for an existing
Tenantwithdomain == <that domain> - If found, joins that tenant; if not, creates a new one named after the domain
A user can never pick an arbitrary existing tenant by name or ID through the API.
All secrets (SECRET_KEY, GEMINI_API_KEY, DB credentials) are read from environment variables. Nothing is hardcoded. See .env.example.
Controlled by CORS_ALLOW_ALL_ORIGINS (dev default: True). In production:
CORS_ALLOW_ALL_ORIGINS=False
CORS_ALLOWED_ORIGINS=https://yourdomain.com# Ingest a CSV from the CLI (bypasses the API, runs the same service layer)
python manage.py ingest_csv path\to\data.csv --tenant "Acme Corp"
# Reset requests_used to 0 for ALL tenants (run at the start of each billing period)
python manage.py reset_quotasdjango.db.utils.OperationalError: SSL connection has been closed unexpectedly
The pgvector extension is not installed. Connect to PostgreSQL and run:
CREATE EXTENSION IF NOT EXISTS vector;Embeddings model slow on first request
intfloat/multilingual-e5-base (~500 MB) is downloaded from HuggingFace Hub on first use and cached locally. Subsequent starts reuse the cache.
GEMINI_API_KEY not set error
Make sure .env exists and is filled in. The file must be in the project root (same directory as manage.py).
Frontend shows blank page
Run python manage.py collectstatic if serving from a web server that needs the static files pre-collected, or use python manage.py runserver in development (static files are served automatically).
requests_used keeps growing after a quota reset
The reset_quotas management command sets all tenants to 0. If the value is still climbing, a cron job or scheduled task should call it at the start of each billing period.