Skip to content

DEPLOY.md

Latest commit

 

History

History
109 lines (82 loc) · 4.96 KB

DEPLOY.md

File metadata and controls

109 lines (82 loc) · 4.96 KB
Raw

Deploy ProbabilityRAG to Hugging Face Spaces

A 3-step handoff to a live, $0/month public demo. The Space builds the Docker image server-side (multi-GB — do not build locally), bakes the vector index + model weights at build time, and serves the API + React UI from one container on port 7860.

1. Create the Space

  1. Sign in / create an account at https://huggingface.co.

  2. New → Space. Name it, pick Docker as the SDK, and CPU basic (free) hardware.

  3. HF needs Space metadata in the repo's README.md frontmatter. Add this block to the very top of README.md (before the # 🎲 ProbabilityRAG title):

    ---
title: ProbabilityRAG
emoji: 🎲
colorFrom: indigo
colorTo: purple
sdk: docker
app_port: 7860
pinned: false
---

    sdk: docker tells HF to build from the Dockerfile; app_port: 7860 routes traffic to the port the container's uvicorn listens on.

2. Add the Z.ai API key as a secret

  1. Get an API key from https://z.ai (the demo generates with GLM-Flash via the OpenAI-compatible API).
  2. In the Space: Settings → Variables and secrets → New secret.
  3. Name it ZAI_API_KEY, paste the key. It is injected at runtime — never baked into the image.

3. Push the repo to the Space

The Space is a git remote. From this repo:

# one time — replace <user>/<space> with your Space id
git remote add space https://huggingface.co/spaces/<user>/<space>

# push the deploy branch to the Space's main branch (Spaces build from `main`)
git push space master:main

The build kicks off automatically on push (~10–20 min first time: CPU torch, model download, index build). Watch the Logs tab in the Space.

If HF asks for auth on push, use a write access token as the password (Settings → Access Tokens on huggingface.co), or huggingface-cli login first.

Environment variables

Set as Space secrets (sensitive) or variables (non-sensitive). The Dockerfile already supplies sane defaults for the public demo; only ZAI_API_KEY is required.

Var Required Default (in image) Purpose
ZAI_API_KEY yes (secret) — Z.ai key for GLM-Flash generation. Runtime only.
PROBRAG_PUBLIC no 1 Turns on rate limiting + public hardening.
PROBRAG_MODELS no glm-flash Comma-separated allowlist of enabled models.
PROBRAG_DEFAULT_MODEL no first of PROBRAG_MODELS Which enabled model is the default.
PROBRAG_ORIGINS no * Comma-separated CORS origins. Pin to the Space URL to lock it down.
PROBRAG_IP_LIMIT no 5 Per-IP requests per UTC day (429 after).
PROBRAG_GLOBAL_LIMIT no 200 Global requests per UTC day — protects the API budget.
ZAI_MODEL no glm-4.7-flash Z.ai model id behind the glm-flash name (free tier: glm-4.7-flash, glm-4.5-flash).

Cold start

The Space sleeps after inactivity. The first request after a sleep pays the model-load time (~40 s) while BGE-M3 + the reranker warm up. /health returns {"ok": false} until the lifespan finishes, and the UI shows a "waking up the models…" state on /health failure — so the first visitor sees a wake-up message, not an error. The index and model weights are baked into the image, so cold start is load-only (no indexing, no downloads).

Attribution (GFDL)

The source text is Grinstead & Snell, "Introduction to Probability" — licensed under the GNU Free Documentation License (GFDL), which permits redistribution. The image downloads it from Dartmouth (https://math.dartmouth.edu/~prob/prob/prob.pdf) at build time and serves it at /pdf for the citation viewer. Keep the GFDL attribution visible in the demo footer.

Operational notes (learned in production)

  • Z.ai free tier intermittently returns 429 code 1305 ("service temporarily overloaded") — transient; retrying seconds later succeeds. The UI surfaces it as an error message.
  • Free-tier CPU retrieval is slow under load: concurrent requests queue behind one worker; a single question takes ~30–90 s end-to-end. Fine for a PoC demo.
  • Every git push restarts the Space (≈40 s cold start; code-only pushes rebuild fast thanks to layer ordering — the index only re-bakes when src/, scripts/, or chunks change).

Verify after deploy

Once the Space status is Running, open the Space URL and check:

  1. UI loads — the Space URL serves the React app (StaticFiles mount).
  2. /health → {"ok": true} once warmed (may be false for the first ~40 s after a wake).
  3. /models → lists glm-flash as an enabled model (and the default).
  4. Ask a question — e.g. "What is the moment generating function of a Poisson?" — a grounded, cited answer streams in and a citation opens the PDF at the right page.
  5. Rate limit — send 6 requests from the same IP in one UTC day; the 6th returns HTTP 429 ("Demo limit reached — clone the repo to run it locally").