diff --git a/README.md b/README.md index 666ad7a..d8a906b 100644 --- a/README.md +++ b/README.md @@ -67,14 +67,42 @@ sparse-only modes are exposed too, so the benchmark can compare all three. python -m venv .venv && source .venv/bin/activate pip install -e ".[dev]" -# Run the real benchmark (downloads the embedding models on first run). -tenantq benchmark --embedder fastembed +# Fastest first check (no model downloads; used by CI): +pytest -q +``` + +Expected output (counts may grow as tests are added): -# Fully offline (deterministic hash embedder, no downloads) — used by CI/tests: +``` +........ [100%] +``` + +```bash +# Fully offline benchmark (deterministic hash embedder, no downloads): tenantq benchmark --embedder hash +``` + +Expected shape (absolute numbers vary by machine; modes and columns do not): + +``` +[ingest] 450 points in 0.42s (1071 pts/s) +### ingest throughput: 1071 pts/s + +HNSW m=16 ef_construct=100 hnsw_ef=64 multitenant=True payload_m=16 docs=450 queries=30 + +| mode | recall@5 | recall@10 | index_recall@10 | p50 ms | p95 ms | p99 ms | qps | +| --- | --- | --- | --- | --- | --- | --- | --- | +| dense | 0.xxx | 0.xxx | 0.xxx | ... | ... | ... | ... | +| sparse | 0.xxx | 0.xxx | n/a | ... | ... | ... | ... | +| hybrid | 0.xxx | 0.xxx | n/a | ... | ... | ... | ... | +``` + +```bash +# Full embedder (downloads models on first run): +tenantq benchmark --embedder fastembed # One-off tenant-scoped search against an ingested collection: -tenantq search "vector similarity ranking" --tenant acme --mode hybrid +tenantq search "neural networks" --tenant acme --mode hybrid --limit 5 --embedder hash ``` By default everything runs against an in-process Qdrant (`:memory:`), which @@ -82,6 +110,7 @@ supports sparse vectors and Query API fusion — so the benchmark produces genui numbers with zero infrastructure. Point at a real server by setting `QDRANT_URL` (see Docker below). + ## Configuration Everything is env-overridable (`Settings.from_env()`) or a CLI flag: