Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 16 additions & 6 deletions docs/about/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

---

## [Unreleased]

### Removed
- **BREAKING CHANGE: Vector store** — removed the ChromaDB vector store backend (`ChromaConfig`, `ChromaVectorStore`) and the `chromadb` dependency, due to known security vulnerabilities in the `chromadb` package. `ai4rag.rag.vector_store` no longer exports `ChromaConfig`; the `"chroma"` value for `vector_store_type` is no longer accepted (only `"milvus"`, `"milvus_lite"`, and `"pgvector"` are supported)
## [0.17.0](https://github.com/IBM/ai4rag/releases/tag/v0.17.0)

### Added
- **Vector store** — Milvus Lite, the embedded, zero-server mode of the Milvus backend, is now the recommended local/zero-config replacement for the removed Chroma store: use the new `MilvusLiteConfig(db_path="./ai4rag.db")` (or `MilvusLiteConfig()` for the default path). Unlike Chroma, Milvus Lite supports hybrid (dense + BM25) search. `pymilvus[milvus-lite]` is now a core dependency, so no extra installation step is needed
- **Vector store** — the Milvus config was split into `MilvusConfig` (remote server / Zilliz Cloud only, which now validates that `uri` is an `http(s)://` URL and raises `ValueError` otherwise) and a new `MilvusLiteConfig` (embedded, local only, configured via `db_path`, which conversely rejects `http(s)://` values). Previously, a single `MilvusConfig` selected between a remote server and embedded Milvus Lite based on whether `uri` looked like a URL or a local file path; a mistyped or unreachable `MILVUS_URI` could therefore be silently interpreted as a local path and create an unintended throwaway local database. That silent fallback is no longer possible — a misconfigured server URI now fails loudly instead. `ai4rag.rag.vector_store` now also exports `MilvusLiteConfig`, and the `vector_store_type` search-space parameter accepts `"milvus_lite"` in addition to `"milvus"` and `"pgvector"`
- **Vector store** — Milvus Lite, the embedded, zero-server mode of the Milvus backend, is now the recommended local/zero-config replacement for the removed Chroma store: use the new `MilvusLiteConfig(db_path="./ai4rag.db")` (or `MilvusLiteConfig()` for the default path). Unlike Chroma, Milvus Lite supports hybrid (dense + BM25) search, though it computes BM25 statistics segment-locally rather than corpus-wide, so hybrid-search ranking fidelity may not transfer exactly to a production Milvus server. `pymilvus[milvus-lite]` is now a core dependency, so no extra installation step is needed
- **Vector store** — new `temporary_milvus_lite_store()` helper (`ai4rag.rag.vector_store.local_store`) providing a disposable, file-backed Milvus Lite store for internal throwaway indexes (model pre-selection, judge calibration), replacing the previous ephemeral in-memory Chroma store used for the same purpose
- **Notebooks** — the MaaS indexing notebook template gains a "Prerequisites for Disconnected Clusters" section (validation cells, configuration examples, download instructions) for pre-downloading Docling and HuggingFace artifacts before running on an air-gapped cluster; the README documents the pre-download/transfer workflow (`DOCLING_ARTIFACTS_PATH`, `HF_HOME`, `HF_HUB_OFFLINE`)

### Changed
- **Vector store** — the Milvus config was split into `MilvusConfig` (remote server / Zilliz Cloud only, which now validates that `uri` is an `http(s)://` URL and raises `ValueError` otherwise) and a new `MilvusLiteConfig` (embedded, local only, configured via `db_path`, which conversely rejects `http(s)://` values). Previously, a single `MilvusConfig` selected between a remote server and embedded Milvus Lite based on whether `uri` looked like a URL or a local file path; a mistyped or unreachable `MILVUS_URI` could therefore be silently interpreted as a local path and create an unintended throwaway local database. That silent fallback is no longer possible — a misconfigured server URI now fails loudly instead. `ai4rag.rag.vector_store` now also exports `MilvusLiteConfig`, and the `vector_store_type` search-space parameter accepts `"milvus_lite"` in addition to `"milvus"` and `"pgvector"`. **Breaking:** `get_vector_store()` now requires `MilvusLiteConfig` (not `MilvusConfig`) when `config.provider == "milvus_lite"`
- **Search space** — hybrid search parameters (`search_mode`, `ranker_strategy`, `ranker_k`, `ranker_alpha`) are now unconditionally part of the default search space, since every remaining vector store backend supports hybrid search. **Breaking:** `AI4RAGSearchSpace`, `prepare_search_space_with_maas()`, and `get_default_ai4rag_search_space_parameters()` no longer accept a `vector_store_type` parameter
- **Event handler** — `VectorStoreSettings` payload fields `provider_id` / `vector_store_id` replaced by `provider_type` / `collection_name`. **Breaking:** code reading `on_pattern_creation()` payloads must switch to the new field names
- **Dependencies** — refreshed `uv.lock` (transitive dependency updates)

### Fixed
- **CI** — the `publish-pypi` maintainer check now reads the actor's `role_name` from the GitHub API instead of the removed `permission` field

### Removed
- **BREAKING CHANGE: Vector store** — removed the ChromaDB vector store backend (`ChromaConfig`, `ChromaVectorStore`), the `chromadb` dependency, and the now-unused `ai4rag.utils.compat.ensure_sqlite3()` compatibility shim, due to known security vulnerabilities in the `chromadb` package. `ai4rag.rag.vector_store` no longer exports `ChromaConfig`; the `"chroma"` value for `vector_store_type` is no longer accepted (only `"milvus"`, `"milvus_lite"`, and `"pgvector"` are supported)

---

Expand Down
3 changes: 1 addition & 2 deletions docs/architecture/core-components.md
Original file line number Diff line number Diff line change
Expand Up @@ -429,11 +429,10 @@ class AI4RAGSearchSpace(SearchSpace):
self,
params: list[Parameter] | None = None,
rules: list[RuleFunction] | None = None,
vector_store_type: str = "milvus",
):
```

`vector_store_type` selects the target backend for search-space defaults; supported values are `"milvus"` (remote server, `MilvusConfig`), `"milvus_lite"` (embedded, local file, `MilvusLiteConfig`), and `"pgvector"` — an unsupported value raises `ValueError`. All three backends support hybrid search, so the same validation rules and defaults apply across them.
All supported vector store backends (`MilvusConfig`, `MilvusLiteConfig`, `PGVectorConfig`) support hybrid search, so the same validation rules and defaults apply regardless of which backend is used at runtime.

**Built-in Validation Rules:**

Expand Down
1 change: 0 additions & 1 deletion docs/development/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -367,7 +367,6 @@ class TestExperimentMilvusLite:
self, client, documents, benchmark_data, foundation_model, embedding_model, tmp_path
):
search_space = AI4RAGSearchSpace(
vector_store_type="milvus_lite",
params=[
Parameter(name="foundation_model", param_type="C", values=[foundation_model]),
Parameter(name="embedding_model", param_type="C", values=[embedding_model]),
Expand Down
2 changes: 1 addition & 1 deletion docs/user-guide/event-handlers.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ def on_pattern_creation(
"embedding": {"model_id": "...", "embedding_params": {"embedding_dimension": 768}},
"retrieval": {"method": "simple", "number_of_chunks": 5, "search_mode": "vector"},
"generation": {"model_id": "...", ...},
"vector_store_binding": {"provider_id": "local_milvus", "vector_store_id": "..."},
"vector_store_binding": {"provider_type": "local_milvus", "collection_name": "..."},
},
}
```
Expand Down
3 changes: 1 addition & 2 deletions docs/user-guide/search-space.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ Parameter(

## Default Parameters

If you don't specify certain parameters, `AI4RAGSearchSpace` uses sensible defaults. The `vector_store_type` parameter defaults to `"milvus"` and accepts `"milvus"` (remote server, `MilvusConfig`), `"milvus_lite"` (embedded local, `MilvusLiteConfig`), or `"pgvector"` — an unsupported value raises `ValueError`. All three support dense and hybrid search (Milvus and Milvus Lite via server-side/embedded BM25, PGVector via PostgreSQL full-text search), so they share the same default parameter set.
If you don't specify certain parameters, `AI4RAGSearchSpace` uses sensible defaults. All supported vector store backends (`MilvusConfig`, `MilvusLiteConfig`, `PGVectorConfig`) support dense and hybrid search (Milvus and Milvus Lite via server-side/embedded BM25, PGVector via PostgreSQL full-text search), so hybrid search parameters are always included in the default parameter set.

### Default Values

Expand Down Expand Up @@ -628,7 +628,6 @@ search_space = AI4RAGSearchSpace(
Parameter(name="ranker_k", param_type="C", values=[0, 30, 60, 100]),
Parameter(name="ranker_alpha", param_type="C", values=[1, 0.3, 0.5, 0.7]),
],
vector_store_type="milvus", # Required for hybrid search
)
```

Expand Down
Loading