From 3c8a1c6f98716917095c18a1d81ec4b56b3bfb57 Mon Sep 17 00:00:00 2001 From: AshSgDe29071999 Date: Mon, 10 Aug 2026 09:43:05 +0530 Subject: [PATCH 1/2] docs: lead Quickstart with pytest path and expected output First-run should not require embedding downloads. Add pytest -q and the expected green outcome before the full benchmark commands. Fixes #1 --- README.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 666ad7a..b35de2f 100644 --- a/README.md +++ b/README.md @@ -67,20 +67,25 @@ sparse-only modes are exposed too, so the benchmark can compare all three. python -m venv .venv && source .venv/bin/activate pip install -e ".[dev]" -# Run the real benchmark (downloads the embedding models on first run). -tenantq benchmark --embedder fastembed +# Fastest first check (no model downloads; used by CI): +pytest -q +``` -# Fully offline (deterministic hash embedder, no downloads) — used by CI/tests: +Expected: a short green summary ending with `passed` (and no failures). + +```bash +# Fully offline benchmark (deterministic hash embedder, no downloads): tenantq benchmark --embedder hash +# Full embedder (downloads models on first run): +tenantq benchmark --embedder fastembed + # One-off tenant-scoped search against an ingested collection: -tenantq search "vector similarity ranking" --tenant acme --mode hybrid +tenantq search "neural networks" --tenant acme --mode hybrid --limit 5 --embedder hash ``` -By default everything runs against an in-process Qdrant (`:memory:`), which -supports sparse vectors and Query API fusion — so the benchmark produces genuine -numbers with zero infrastructure. Point at a real server by setting `QDRANT_URL` -(see Docker below). +`tenant_id` / `--tenant` is mandatory. Empty tenant is refused (not treated as "zero hits"). + ## Configuration From f9fabf9170a4bdf08cac5e1a05a138087dcf792e Mon Sep 17 00:00:00 2001 From: Ashay Date: Mon, 10 Aug 2026 09:47:40 +0530 Subject: [PATCH 2/2] docs: show concrete expected output in Quickstart Add pytest dots output and a shaped benchmark table so first-run readers know what success looks like without downloading models. Fixes #1 --- README.md | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index b35de2f..d8a906b 100644 --- a/README.md +++ b/README.md @@ -71,12 +71,33 @@ pip install -e ".[dev]" pytest -q ``` -Expected: a short green summary ending with `passed` (and no failures). +Expected output (counts may grow as tests are added): + +``` +........ [100%] +``` ```bash # Fully offline benchmark (deterministic hash embedder, no downloads): tenantq benchmark --embedder hash +``` + +Expected shape (absolute numbers vary by machine; modes and columns do not): +``` +[ingest] 450 points in 0.42s (1071 pts/s) +### ingest throughput: 1071 pts/s + +HNSW m=16 ef_construct=100 hnsw_ef=64 multitenant=True payload_m=16 docs=450 queries=30 + +| mode | recall@5 | recall@10 | index_recall@10 | p50 ms | p95 ms | p99 ms | qps | +| --- | --- | --- | --- | --- | --- | --- | --- | +| dense | 0.xxx | 0.xxx | 0.xxx | ... | ... | ... | ... | +| sparse | 0.xxx | 0.xxx | n/a | ... | ... | ... | ... | +| hybrid | 0.xxx | 0.xxx | n/a | ... | ... | ... | ... | +``` + +```bash # Full embedder (downloads models on first run): tenantq benchmark --embedder fastembed @@ -84,7 +105,10 @@ tenantq benchmark --embedder fastembed tenantq search "neural networks" --tenant acme --mode hybrid --limit 5 --embedder hash ``` -`tenant_id` / `--tenant` is mandatory. Empty tenant is refused (not treated as "zero hits"). +By default everything runs against an in-process Qdrant (`:memory:`), which +supports sparse vectors and Query API fusion — so the benchmark produces genuine +numbers with zero infrastructure. Point at a real server by setting `QDRANT_URL` +(see Docker below). ## Configuration