Add Supabase authentication and Railway production deployment

.env.example

@@ -6,8 +6,39 @@ DOCULENS_INITIALIZE_DATABASE=true
 DOCULENS_SEED_DEMO_USERS=false
 DOCULENS_SEED_DEMO_WORKSPACE=false
 DOCULENS_SHOWCASE_READ_ONLY=false
+DOCULENS_REQUIRE_AUTH=false
+DOCULENS_SERVE_FRONTEND=false
 DOCULENS_AUTH_SECRET=replace-with-a-long-random-value
+DOCULENS_AUTH_COOKIE_SECURE=false
 DOCULENS_API_KEY=
+SUPABASE_URL=
+SUPABASE_PUBLISHABLE_KEY=
+DOCULENS_SUPABASE_ALLOWED_EMAILS=[]
+DOCULENS_SUPABASE_ALLOWED_DOMAINS=[]
+DOCULENS_ALLOW_PUBLIC_SUPABASE_SIGNIN=false
+DOCULENS_SUPABASE_GOOGLE_ENABLED=false
+DOCULENS_TASK_MODE=celery
+DOCULENS_EXTRACTION_BACKEND=docling
+
+# Adaptive investigation agent
+DOCULENS_AGENT_ENABLED=true
+DOCULENS_AGENT_PROVIDER=openai
+# Optional override; otherwise the selected provider's default model is used.
+DOCULENS_AGENT_MODEL=
+DOCULENS_AGENT_CHECKPOINT_BACKEND=memory
+DOCULENS_AGENT_MAX_STEPS=10
+DOCULENS_AGENT_MAX_EVIDENCE=24
+DOCULENS_AGENT_SEARCH_LIMIT=6
+
+# Document storage (use s3 with Railway Buckets, R2, S3, or compatible storage)
+DOCULENS_STORAGE_BACKEND=local
+DOCULENS_STORAGE_LOCAL_PATH=/workspace/app/data/ingestion
+DOCULENS_S3_ENDPOINT_URL=
+DOCULENS_S3_BUCKET_NAME=
+DOCULENS_S3_ACCESS_KEY_ID=
+DOCULENS_S3_SECRET_ACCESS_KEY=
+DOCULENS_S3_REGION=auto
+DOCULENS_S3_FORCE_PATH_STYLE=false
 
 # AI providers
 OPENAI_API_KEY=
@@ -20,6 +51,7 @@ DOCULENS_EMBEDDING_CACHE_SIZE=1024
 
 # Infrastructure
 PROJECT_NAME=doculens
+DATABASE_URL=
 DATABASE_HOST=doculens_database
 DATABASE_PORT=5432
 DATABASE_NAME=doculens

.github/workflows/backend-ci.yml

@@ -22,7 +22,7 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
-          python-version: "3.11"
+          python-version: "3.12"
           cache: pip
 
       - name: Install dependencies

Makefile

@@ -1,18 +1,21 @@
 SHELL := /bin/bash
 .DEFAULT_GOAL := help
 
-.PHONY: help install dev up down showcase-up showcase-down showcase-logs test lint format typecheck check sample
+.PHONY: help install dev preview up down showcase-up showcase-down showcase-logs test lint format typecheck check sample
 
 help: ## Show available commands
 	@awk 'BEGIN {FS = ":.*## "; printf "DocuLens developer commands\n\n"} /^[a-zA-Z_-]+:.*## / {printf "  %-12s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
 
 install: ## Install backend and frontend development dependencies
-	python3 -m pip install -e '.[dev]'
+	python3 -m pip install -e '.[dev,ocr]'
 	npm --prefix frontend ci
 
 dev: ## Run API with hot reload
 	uvicorn app.main:app --reload --port 8080
 
+preview: ## Start the local product preview without the heavy OCR worker
+	docker compose --env-file .env -f docker/docker-compose.yml up --build -d database redis api frontend
+
 up: ## Start the complete Docker development stack
 	docker compose --env-file .env -f docker/docker-compose.yml up --build -d
 

README.md

@@ -1,16 +1,30 @@
 # DocuLens AI
 
-**Turn unstructured business documents into searchable, cited, operational knowledge.**
+**An evidence-grounded AI agent for investigating business documents and producing auditable decisions.**
 
-DocuLens is an open-source document intelligence system for teams that need more than a chat-with-PDF demo. It accepts documents asynchronously, extracts layout-aware content, creates citation-ready embeddings, and exposes classification, summarization, semantic search, and grounded question answering through a versioned API and an operator console.
+DocuLens is an open-source document intelligence system for teams that need more
+than chat-with-PDF. Give the agent an outcome—such as reviewing vendor renewals
+or comparing security obligations—and it chooses its own bounded research path,
+inspects source passages, repairs unsupported claims, and produces a cited
+decision brief with a complete evidence ledger.
 
 [![Backend CI](https://github.com/codewithmoin/doculens-ai/actions/workflows/backend-ci.yml/badge.svg)](https://github.com/codewithmoin/doculens-ai/actions/workflows/backend-ci.yml)
-[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-3776AB)](https://www.python.org/)
+[![Python 3.12+](https://img.shields.io/badge/python-3.12%2B-3776AB)](https://www.python.org/)
 [![FastAPI](https://img.shields.io/badge/FastAPI-API-009688)](https://fastapi.tiangolo.com/)
 [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+[![Live Demo](https://img.shields.io/badge/live_demo-open-2563EB)](https://doculens-ai.pages.dev/)
+
+**[Explore the live showcase →](https://doculens-ai.pages.dev/)** No sign-in required. The hosted workspace is read-only and uses clearly labelled synthetic data.
 
 ## Why it is technically interesting
 
+- **Adaptive investigation agent:** a typed LangGraph state machine lets the model
+  select, repeat, and stop document tools dynamically while enforcing step,
+  evidence, and cost boundaries.
+- **Evidence ledger and citation gate:** report findings may cite only passages
+  retrieved during the run; invalid references route the graph back into research.
+- **Durable agent state:** PostgreSQL checkpoints preserve every graph step for
+  fault recovery, investigation history, and future human approval workflows.
 - **Layout-aware ingestion:** Docling preserves headings, tables, provenance, and page numbers instead of flattening a document into one string.
 - **Asynchronous AI workflows:** FastAPI accepts work quickly; Celery executes extraction and model calls with bounded task time, late acknowledgement, and worker-loss recovery.
 - **Citation-first RAG:** every vector carries document, chunk, page, title, and token metadata; QA prompts use stable references and require grounded answers.
@@ -45,22 +59,36 @@ The redesigned experience separates the public product story from a focused auth
 
 ```mermaid
 flowchart LR
-    UI["React operator console"] -->|REST /api/v1| API["FastAPI gateway"]
+    UI["React operator console"] -->|REST + SSE /api/v1| API["FastAPI gateway"]
     Client["API clients"] --> API
-    API -->|persist event| DB[("TimescaleDB + PostgreSQL")]
+    API --> Agent["LangGraph investigation agent"]
+    Agent --> Decide{"Choose next action"}
+    Decide --> Tools["Inventory · search · inspect"]
+    Tools --> Evidence["Evidence ledger"]
+    Evidence --> Decide
+    Decide --> Report["Cited decision brief"]
+    Report --> Verify{"Citation gate"}
+    Verify -->|repair| Decide
+    Verify -->|valid| UI
+    API -->|persist event| DB[("PostgreSQL + pgvector")]
     API -->|enqueue id| Redis[("Redis broker")]
     Redis --> Worker["Celery worker"]
     Worker --> Extract["Docling extraction"]
     Extract --> Chunk["Layout-aware chunking"]
     Chunk --> Embed["Batched embeddings"]
-    Embed --> Vector[("pgvector / DiskANN")]
+    Embed --> Vector[("pgvector / HNSW")]
     Worker --> LLM["Structured LLM calls"]
     LLM --> DB
     Vector --> Retrieve["Filtered retrieval"]
     Retrieve --> LLM
+    Agent --> Checkpoints[("PostgreSQL checkpoints")]
 ```
 
-The event record is the durable boundary between HTTP and AI work. A worker validates the stored event, selects a typed pipeline from the registry, runs its nodes, and stores a serializable task context. This design keeps slow or retryable model work away from request threads while retaining an auditable input/output trail.
+The document pipeline and investigation agent intentionally use different
+execution models. Document ingestion remains an asynchronous, retryable pipeline.
+Investigations are adaptive graphs: the model chooses the next read-only tool,
+while application-owned state, budgets, provenance, and validation determine
+what it is allowed to do and when the run can finish.
 
 ## Capabilities
 
@@ -70,8 +98,12 @@ The event record is the durable boundary between HTTP and AI work. A worker vali
 - document summaries with source chunk provenance
 - metadata-filtered semantic and keyword search
 - retrieval-augmented QA with stable citations and confidence
+- nonlinear, cross-document investigations with model-selected tools
+- streamed execution activity without exposing private chain-of-thought
+- per-run evidence ledgers and deterministic citation validation
+- PostgreSQL-backed LangGraph checkpoints and saved investigation history
 - archive, restore, and soft-delete lifecycle operations
-- JWT personas plus optional API-key protection
+- Supabase email-link and Google sign-in, secure session cookies, JWT personas, and optional API keys
 - work queues, notifications, dashboards, and request history
 - OpenAI, Anthropic, OpenRouter, and local OpenAI-compatible model adapters
 
@@ -80,9 +112,10 @@ The event record is the durable boundary between HTTP and AI work. A worker vali
 | Layer | Technology | Responsibility |
 | --- | --- | --- |
 | API | FastAPI, Pydantic, SQLAlchemy | contracts, validation, authentication, persistence |
+| Agent | LangGraph, Instructor, typed tool contracts | adaptive routing, checkpoints, evidence validation |
 | Jobs | Celery, Redis | resilient long-running document and AI processing |
 | Retrieval | Docling, tiktoken, OpenAI embeddings | extraction, bounded chunks, batched embeddings |
-| Data | PostgreSQL, Timescale Vector, pgvector | events, metadata, keyword and vector search |
+| Data | PostgreSQL, pgvector, HNSW | events, metadata, keyword and vector search |
 | Models | Instructor, OpenAI, Anthropic | provider-neutral structured outputs |
 | Web | React 19, TypeScript, Vite, TanStack Query | operator workflows and evidence review |
 | Quality | pytest, Ruff, Pyright, pre-commit, GitHub Actions | repeatable engineering checks |
@@ -93,7 +126,11 @@ The event record is the durable boundary between HTTP and AI work. A worker vali
 
 - Docker with Compose v2
 - an OpenAI API key (required for embeddings; model providers are configurable)
-- Python 3.11+ and Node 20+ only when running services outside Docker
+- Python 3.12 and Node 20+ only when running services outside Docker
+
+`make install` includes the optional Docling OCR stack for full local
+development. The one-service Railway image intentionally installs only the
+lightweight runtime; the Celery worker image installs the `ocr` extra.
 
 ```bash
 git clone https://github.com/codewithmoin/doculens-ai.git
@@ -123,6 +160,14 @@ make showcase-up
 
 See the [showcase deployment runbook](docs/deploy-showcase.md) for DNS, TLS, verification, backups, and rollback. This mode demonstrates the finished AI workflow without accepting public uploads or spending money on visitor model calls.
 
+### Deploy the interactive app on Railway
+
+The repository also includes a one-service Railway profile with Supabase Auth,
+durable S3-compatible uploads, portable pgvector retrieval, and an upgrade path
+to a separate Celery worker. See the
+[Railway deployment runbook](docs/deploy-railway.md) for the Free portfolio
+topology and the full all-Railway topology.
+
 ## API examples
 
 All stable endpoints are under `/api/v1`. Legacy `/events` routes remain available for existing clients.
@@ -148,6 +193,21 @@ curl --fail-with-body http://localhost:8080/api/v1/events/ \
 
 The API returns `202 Accepted` with an event id. Poll the event resource until its task context contains the answer and `chunk_references` used to construct it. Ready-made payloads live in [`requests/events`](requests/events).
 
+Start a streaming investigation:
+
+```bash
+curl --no-buffer --fail-with-body \
+  http://localhost:8080/api/v1/investigations/stream \
+  -H 'Content-Type: application/json' \
+  -H 'Accept: text/event-stream' \
+  -H "X-API-Key: $DOCULENS_API_KEY" \
+  -d '{"goal":"Review vendor agreements for renewal deadlines and contractual risk."}'
+```
+
+The stream contains user-safe activity events, newly discovered evidence,
+citation verification, and the final decision brief. Model reasoning is never
+returned as chain-of-thought.
+
 ## Configuration
 
 Configuration is validated once at startup. See [`.env.example`](.env.example) for the full local template.
@@ -163,6 +223,9 @@ Configuration is validated once at startup. See [`.env.example`](.env.example) f
 | `DOCULENS_PROVIDER_TIMEOUT_SECONDS` | 30 | AI provider network timeout |
 | `DOCULENS_QA_TOP_K` | 5 | default QA retrieval breadth |
 | `DOCULENS_SHOWCASE_READ_ONLY` | false | blocks workspace mutations and enables the public product-tour UX |
+| `DOCULENS_AGENT_CHECKPOINT_BACKEND` | `memory` | use `postgres` for durable production investigations |
+| `DOCULENS_AGENT_MAX_STEPS` | 10 | hard bound on adaptive agent iterations |
+| `DOCULENS_AGENT_MAX_EVIDENCE` | 24 | maximum passages retained in one evidence ledger |
 
 ## Development
 
@@ -178,10 +241,15 @@ Useful commands are discoverable with `make help`. CI runs the same Ruff, Pyrigh
 
 Create labelled `RetrievalExample` cases with known relevant chunk ids and run `evaluate(examples, k=5)`. Track Recall@5 and MRR before changing chunk size, embedding model, filters, or ranking. The utility is intentionally offline and deterministic: it belongs in CI; live provider quality and latency belong in a separate scheduled benchmark.
 
+### Agent evaluation
+
+Create representative investigation goals with expected source documents, required report terms, citation thresholds, and step budgets. `evaluate_investigation` scores the observable graph state for document recall, report coverage, citation coverage, completion, and budget adherence—without using another model as an opaque judge. This gives agent-routing or prompt changes a deterministic regression gate while leaving semantic answer-quality judging to a separately versioned benchmark.
+
 ## Project structure
 
 ```text
 app/
+├── agents/          LangGraph state, model boundary, tools, and citation gates
 ├── api/             HTTP contracts, auth, dependencies, versioned routers
 ├── config/          validated runtime and infrastructure settings
 ├── core/            pipeline primitives and observability
@@ -204,6 +272,12 @@ requests/            executable example payloads
 - **Async jobs, synchronous pipeline nodes:** document work is asynchronous at the system boundary; node code stays easy to reason about because most provider SDKs and extraction libraries are synchronous. Worker concurrency supplies parallelism.
 - **Process-local embedding cache:** avoids repeated provider calls without introducing another consistency-sensitive cache. It resets on deploy and is not intended as durable storage.
 - **Provider-neutral structured outputs:** improves validation and portability, but provider behavior still differs and must be evaluated per model.
+- **Bounded autonomy:** the model selects the investigation path, but tool
+  permissions, workspace scope, step budgets, and citation validation remain
+  deterministic application policy.
+- **One agent, explicit tools:** a single orchestrator is easier to evaluate and
+  operate than a fashionable multi-agent swarm. Parallel document workers can be
+  introduced only when a labelled benchmark demonstrates value.
 - **Compatibility versioning:** `/api/v1` is canonical while legacy routes remain during migration. Removing aliases is a future breaking release.
 - **Product hierarchy over dashboard density:** the public landing page explains the problem and architecture; `/app` is reserved for focused document work. The UI uses route-level code splitting, a paper-and-ink system with cobalt focus and amber evidence, persistent dark mode, and reduced-motion fallbacks instead of a large animation runtime.
 
@@ -215,11 +289,15 @@ More detail is recorded in [`docs/engineering-notes.md`](docs/engineering-notes.
 - The current retrieval path is dense-first; hybrid ranking exists but needs a labelled corpus before tuning.
 - The included deployment manifest is deliberately scoped to a single-node, read-only portfolio showcase—not a multi-tenant customer environment.
 - Authentication is suitable for a single workspace; multi-tenant authorization is not implemented.
+- Free-tier investigations execute through an active SSE request. Checkpoints
+  preserve completed work, but unattended continuation requires the optional worker.
 - Retrieval metrics are utilities, not a bundled benchmark dataset—the project does not claim quality without domain-labelled examples.
 
 ## Roadmap
 
-- persist event lifecycle states and retry diagnostics as first-class columns
+- add resumable human approval before report export
+- evaluate parallel per-document workers for large contract collections
+- add agent task-completion, citation-coverage, latency, and cost benchmarks
 - add a small, redistributable labelled retrieval benchmark
 - evaluate reciprocal-rank fusion for dense + keyword retrieval
 - add OpenTelemetry traces and provider latency/cost dashboards

app/agents/__init__.py

@@ -0,0 +1,5 @@
+"""Evidence-grounded investigation agent."""
+
+from app.agents.graph import InvestigationRuntime, build_investigation_graph
+
+__all__ = ["InvestigationRuntime", "build_investigation_graph"]