diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index ff38f228..16e44700 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -56,7 +56,7 @@ body: - ai4rag version: - Python version: - openai SDK version: - - Vector Store (Milvus/ChromaDB): + - Vector Store (Milvus/pgvector): - Operating System: render: markdown validations: diff --git a/README.md b/README.md index b53ca2ae..601eb58e 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ It accepts a variety of RAG Templates and a search space definition, then return > [!IMPORTANT] > `ai4rag` is **provider-agnostic**. It reaches foundation and embedding models through the stock [`openai`](https://github.com/openai/openai-python) SDK, so any **OpenAI-compatible endpoint** works — a hosted API, a self-managed server (vLLM, TGI, Ollama, …), or an [OpenShift AI Models-as-a-Service (MaaS)](https://www.redhat.com/en/products/ai) deployment, the integration `ai4rag` ships helpers for out of the box. You can also plug in your **own** foundation model, embedding model, or vector store by implementing the matching `Base*` interface. -> To run an experiment you'll need one foundation model and one embedding model (from any of the above), plus a vector store (Chroma, Milvus, or PostgreSQL/pgvector) connected directly via `ai4rag.rag.vector_store`. +> To run an experiment you'll need one foundation model and one embedding model (from any of the above), plus a vector store (remote Milvus, embedded Milvus Lite, or PostgreSQL/pgvector) connected directly via `ai4rag.rag.vector_store`. ## Model providers @@ -48,21 +48,24 @@ When using the MaaS backend, ai4rag relies on: - **Embeddings** — Text embeddings via the `embeddings` endpoint (e.g. for indexing and query encoding). Because `models.list()` carries no metadata, embedding dimension and context length are auto-detected at construction (or supplied via `params`). - **Chat / completions** — Foundation model integration for answer generation when evaluating RAG patterns. -Vector storage is independent of MaaS: `ai4rag` connects directly to Chroma, Milvus, or PostgreSQL/pgvector via the config classes in `ai4rag.rag.vector_store` (see [Vector stores](#vector-stores) below). +Vector storage is independent of MaaS: `ai4rag` connects directly to remote Milvus, embedded Milvus Lite, or PostgreSQL/pgvector via the config classes in `ai4rag.rag.vector_store` (see [Vector stores](#vector-stores) below). ## Vector stores ai4RAG talks to the vector store directly through provider-specific clients — no MaaS deployment is required for this part. Pick a provider and pass its config to `AI4RAGExperiment` as `vector_store_config`: -- **`ChromaConfig`** — Chroma. Ephemeral in-memory by default; persistent (via `persist_directory`) or client/server (via `host`/`port`) modes are also supported. Vector-only search. -- **`MilvusConfig`** — Milvus. Requires a `uri`; supports TLS (`https://` scheme) and self-signed CAs via `server_cert`. Hybrid search (dense + BM25). +- **`MilvusConfig`** — remote Milvus server or Zilliz Cloud only. `uri` must be a `http(s)://` URL (TLS and self-signed CAs via `server_cert`); anything else (a bare host, a file path, an empty string) raises `ValueError`. This is a deliberate safety check: a mistyped or unreachable `MILVUS_URI` now fails loudly instead of silently falling back to a throwaway local database. Supports hybrid search (dense + BM25). +- **`MilvusLiteConfig`** — the embedded, zero-server **Milvus Lite** engine, backed by a local `db_path` file (default `"./ai4rag_milvus_lite.db"`) — no setup required, ideal for local development and small-scale workloads. Also supports hybrid search (dense + BM25); rejects `http(s)://` values (use `MilvusConfig` for those). - **`PGVectorConfig`** — PostgreSQL with the `pgvector` extension. Hybrid search (dense + `tsvector` full-text). -Each config is a frozen dataclass with a `.from_env()` constructor and an `env_vars` attribute listing the environment variables it reads (e.g. `MILVUS_URI`, `PGVECTOR_HOST`). +Each config is a frozen dataclass with a `.from_env()` constructor and an `env_vars` attribute listing the environment variables it reads (e.g. `MILVUS_URI` for `MilvusConfig`, `MILVUS_LITE_DB_PATH` for `MilvusLiteConfig`, `PGVECTOR_HOST` for `PGVectorConfig`). + +> [!note] +> Milvus Lite is intended for local development, tests, and small-scale workloads (prototyping, up to roughly 1M vectors) — not production serving. For production or large corpora, use a remote Milvus server (`MilvusConfig`), Zilliz Cloud, or pgvector. ## Document processing -ai4RAG uses [`docling-core`](https://github.com/docling-project/docling-core) for document representation and chunking. Documents are represented as `DoclingDocument` instances, and the `DoclingChunker` leverages docling's `HybridChunker` for structure-aware, token-aware chunking. `docling-core`, `openai`, and the vector store clients (`chromadb`, `pymilvus`, `pgvector`, `asyncpg`) are all installed automatically with `ai4rag`. +ai4RAG uses [`docling-core`](https://github.com/docling-project/docling-core) for document representation and chunking. Documents are represented as `DoclingDocument` instances, and the `DoclingChunker` leverages docling's `HybridChunker` for structure-aware, token-aware chunking. `docling-core`, `openai`, and the vector store clients (`pymilvus` with Milvus Lite, `pgvector`, `asyncpg`) are all installed automatically with `ai4rag`. ## Quick start @@ -232,8 +235,8 @@ Using the information from the previous steps, create an experiment and run the > [!note] > Select the vector store by passing a `vector_store_config` to `AI4RAGExperiment`: -> `ChromaConfig()` for a zero-config in-memory store (vector-only search), or -> `MilvusConfig.from_env()` / `PGVectorConfig.from_env()` for a server-backed store with hybrid (dense + keyword) search. +> `MilvusLiteConfig()` (or `MilvusLiteConfig(db_path="./ai4rag.db")`) for a zero-config, local Milvus Lite store, or +> `MilvusConfig.from_env()` / `PGVectorConfig.from_env()` for a server-backed store. All support hybrid (dense + keyword) search. ```python from ai4rag.core.experiment.experiment import AI4RAGExperiment diff --git a/ai4rag/core/experiment/experiment.py b/ai4rag/core/experiment/experiment.py index 0131ee86..64613a75 100644 --- a/ai4rag/core/experiment/experiment.py +++ b/ai4rag/core/experiment/experiment.py @@ -425,11 +425,6 @@ def run_single_evaluation(self, rag_params: RAGParamsType) -> float: number_of_chunks = retrieval_params[AI4RAGParamNames.NUMBER_OF_CHUNKS] search_mode = retrieval_params.get(AI4RAGParamNames.SEARCH_MODE, "vector") - if search_mode != "vector" and self.vector_store_config.provider == "chroma": - raise RAGExperimentError( - f"Search mode '{search_mode}' is not supported with chroma vector store. " - "Only 'vector' mode is supported for chroma." - ) context_template_text = foundation_model.context_template_text system_message_text = foundation_model.system_message_text @@ -500,11 +495,6 @@ def run_single_evaluation(self, rag_params: RAGParamsType) -> float: collection_name = vector_store.collection_name - # The store's connection/client is only needed for indexing and retrieval, - # both of which finish before scoring; closing it deterministically here - # (rather than waiting on garbage collection) keeps a long HPO search from - # accumulating one open connection per evaluated pattern, including on - # trials that fail and get caught by search()'s objective_function. with vector_store: if not self._collection_exists(collection_name=collection_name): chunking_method = chunking_params.get(AI4RAGParamNames.CHUNKING_METHOD) @@ -903,9 +893,10 @@ def _evaluate_response( def _collection_exists(self, collection_name: str) -> bool: """ This method checks if a collection with a given name already exists. - The trick comes with chromadb. We always need to assume that collection - does not exist, as we create new instance of chroma in memory per each - run. + Existence is tracked by this run's own bookkeeping + (``self.results.collection_names``) rather than by querying the backend, + so the check is backend-agnostic and reflects only collections this + experiment created and can safely reuse. Parameters ---------- diff --git a/ai4rag/core/experiment/mps.py b/ai4rag/core/experiment/mps.py index c52ef5fe..0aa2b06d 100644 --- a/ai4rag/core/experiment/mps.py +++ b/ai4rag/core/experiment/mps.py @@ -26,7 +26,7 @@ from ai4rag.rag.retrieval.retriever import Retriever from ai4rag.rag.template.simple_rag_template import SimpleRAG from ai4rag.rag.vector_store.base_vector_store import BaseVectorStore -from ai4rag.rag.vector_store.chroma import ChromaVectorStore +from ai4rag.rag.vector_store.local_store import temporary_milvus_lite_store from ai4rag.utils.constants import AI4RAGParamNames, PreSelectorConstants __all__ = ["PreSelectorError", "ModelsPreSelector"] @@ -140,8 +140,9 @@ def evaluate_patterns(self): If knowledge base references were provided, the retriever is created once and reused. Otherwise, a separate vector store is built for each embedding model, and the best-performing ones are selected. - For evaluation only sample of the documents is used, embedded and added - to chroma vector store. + For evaluation only a sample of the documents is used, embedded and added + to a throwaway, file-backed Milvus Lite vector store that is torn down as + soon as the embedding model's evaluation finishes. This method does not return anything, but in the end changes attributes of the instance: self.evaluation_results is a mapping holding results @@ -156,17 +157,16 @@ def evaluate_patterns(self): chunked_documents = self._chunk_documents(documents) for i, embedding_model in enumerate(self.embedding_models): + collection_name = f"ai4rag_mps_collection_{i}" try: - collection_name = f"ai4rag_mps_collection_{i}" - try: - vector_store = self._create_vector_store( - embedding_model, chunked_documents, collection_name=collection_name - ) - except Exception as exc: - raise IndexingError(exc, collection_name, embedding_model.model_id) from exc + with temporary_milvus_lite_store(embedding_model, collection_name=collection_name) as vector_store: + try: + self._index_documents(vector_store, chunked_documents) + except Exception as exc: + raise IndexingError(exc, collection_name, embedding_model.model_id) from exc - retriever = Retriever(vector_store, **self.retrieval_params) - self._evaluate_foundation_models(retriever=retriever, embedding_model=embedding_model) + retriever = Retriever(vector_store, **self.retrieval_params) + self._evaluate_foundation_models(retriever=retriever, embedding_model=embedding_model) except IndexingError as exc: self._exception_handler.handle_exception(exc) @@ -221,56 +221,35 @@ def _evaluate_foundation_models(self, retriever: Retriever, embedding_model: Bas continue @staticmethod - def _create_vector_store( - embedding_model: BaseEmbeddingModel, - chunked_documents: list[AI4RAGChunk], - collection_name: str, - ) -> BaseVectorStore: - """ - Create instance of vector store with given chunked documents and embedding model. + def _index_documents(vector_store: BaseVectorStore, chunked_documents: list[AI4RAGChunk]) -> None: + """Embed and add the chunked documents to *vector_store*, retrying once. + + A single retry absorbs transient embedding-service hiccups. If the retry + also fails, the exception propagates so the caller can wrap it in an + :class:`IndexingError` and skip this embedding model. Parameters ---------- - embedding_model : BaseEmbeddingModel - Embedding model used for collection creation. + vector_store : BaseVectorStore + Store the chunks are embedded into. chunked_documents : list[AI4RAGChunk] Chunked documents for the embedding process. - collection_name : str - Name of the collection in the chroma vector database. - - Returns - ------- - VectorStore - Instance for communication with properly created index in the - vector database. - Raises ------ - PreSelectorError - When 2 attempts of embedding documents are failing + Exception + Propagates the second failure when both the initial embedding attempt + and the retry fail. """ - logger.info("Building index for pre-evaluation using embedding model: '%s'.", embedding_model.model_id) - - vector_store = ChromaVectorStore( - embedding_model=embedding_model, - collection_name=collection_name, - ) - logger.debug("MPS: Embedding documents ...") try: vector_store.add_documents(chunked_documents) except Exception as err: # pylint: disable=broad-exception-caught - logger.warning("Failed to create in-memory vector index due to: %s.", repr(err), exc_info=True) - try: - vector_store.add_documents(chunked_documents) - except Exception as exc: # pylint: disable=broad-exception-caught - raise PreSelectorError(f"Failed to create in-memory vector index due to: {repr(exc)}.") from exc + logger.warning("Failed to build the vector index due to: %s. Retrying once.", repr(err), exc_info=True) + vector_store.add_documents(chunked_documents) logger.debug("MPS: Embedding documents finished!") - return vector_store - def _evaluate_single_pattern( self, foundation_model: BaseFoundationModel, retriever: Retriever ) -> EvaluationMetricsResult: diff --git a/ai4rag/core/experiment/utils.py b/ai4rag/core/experiment/utils.py index dee594d3..cb7e5748 100644 --- a/ai4rag/core/experiment/utils.py +++ b/ai4rag/core/experiment/utils.py @@ -31,7 +31,7 @@ _semantic_chunker_cache = {} -VectorStoreType: TypeAlias = Literal["milvus", "chroma"] +VectorStoreType: TypeAlias = Literal["milvus", "milvus_lite", "pgvector"] class RAGExperimentError(Exception): diff --git a/ai4rag/evaluator/judge_selection.py b/ai4rag/evaluator/judge_selection.py index 9edbe7bd..b8899a18 100644 --- a/ai4rag/evaluator/judge_selection.py +++ b/ai4rag/evaluator/judge_selection.py @@ -194,16 +194,16 @@ def _run_reference_rag( list List of :class:`EvaluationData` instances ready for judge scoring. """ - from ai4rag.rag.vector_store.chroma import ChromaVectorStore + from ai4rag.rag.vector_store.local_store import temporary_milvus_lite_store chunker = LangChainChunker(chunk_size=512, method="recursive", chunk_overlap=128) chunks = chunker.split_documents(documents) - vector_store = ChromaVectorStore(embedding_model=embedding_model, collection_name="ai4rag_judge_calibration") - vector_store.add_documents(chunks) - retriever = Retriever(vector_store=vector_store, number_of_chunks=3, method="simple", search_mode="vector") - rag = SimpleRAG(foundation_model=foundation_model, retriever=retriever) - inference_response = query_rag(rag=rag, questions=list(benchmark_data.questions), max_threads=max_threads) - return build_evaluation_data(benchmark_data=benchmark_data, inference_response=inference_response) + with temporary_milvus_lite_store(embedding_model, collection_name="ai4rag_judge_calibration") as vector_store: + vector_store.add_documents(chunks) + retriever = Retriever(vector_store=vector_store, number_of_chunks=3, method="simple", search_mode="vector") + rag = SimpleRAG(foundation_model=foundation_model, retriever=retriever) + inference_response = query_rag(rag=rag, questions=list(benchmark_data.questions), max_threads=max_threads) + return build_evaluation_data(benchmark_data=benchmark_data, inference_response=inference_response) def _ordered_question_scores(evaluation_result: EvaluationMetricsResult, metric_name: str) -> list[float | None]: diff --git a/ai4rag/rag/vector_store/__init__.py b/ai4rag/rag/vector_store/__init__.py index 3ae97ce5..1a21bd09 100644 --- a/ai4rag/rag/vector_store/__init__.py +++ b/ai4rag/rag/vector_store/__init__.py @@ -4,8 +4,8 @@ # ----------------------------------------------------------------------------- from ai4rag.rag.vector_store.base_vector_store import BaseVectorStore from ai4rag.rag.vector_store.config import ( - ChromaConfig, MilvusConfig, + MilvusLiteConfig, PGVectorConfig, get_vector_store_config, get_vector_store_env_vars, @@ -14,8 +14,8 @@ __all__ = [ "BaseVectorStore", - "ChromaConfig", "MilvusConfig", + "MilvusLiteConfig", "PGVectorConfig", "get_vector_store", "get_vector_store_config", diff --git a/ai4rag/rag/vector_store/chroma.py b/ai4rag/rag/vector_store/chroma.py deleted file mode 100644 index 1ec02cbf..00000000 --- a/ai4rag/rag/vector_store/chroma.py +++ /dev/null @@ -1,412 +0,0 @@ -# ----------------------------------------------------------------------------- -# Copyright IBM Corp. 2025-2026 -# SPDX-License-Identifier: Apache-2.0 -# ----------------------------------------------------------------------------- -from typing import Any, cast - -import chromadb -from chromadb.api import ClientAPI - -from ai4rag.rag.chunking.chunk import AI4RAGChunk - -from ..embedding.base_model import BaseEmbeddingModel -from .base_vector_store import BaseVectorStore -from .config import ChromaConfig -from .utils import merge_window_into_a_document - -__all__ = ["ChromaVectorStore"] - - -class ChromaVectorStore(BaseVectorStore): - """Vector store backed by ChromaDB via the native ``chromadb`` client. - - Parameters - ---------- - embedding_model : BaseEmbeddingModel - Model used to embed documents and queries. - config : ChromaConfig | None, default=None - Connection parameters selecting the Chroma running mode (ephemeral, - persistent, or client/server). Defaults to an ephemeral in-memory - instance. - distance_metric : str, default="cosine" - Metric used to calculate similarity between vectors. One of - ``"cosine"`` or ``"l2"``. - collection_name : str | None, default=None - Existing collection to reuse; must start with the ``ai4rag`` prefix. When - omitted, a new compliant name is generated (see - :func:`ai4rag.rag.vector_store.utils.resolve_collection_name`). - """ - - _supported_distance_metrics = ("cosine", "l2") - _BATCH_SIZE = 2048 - DOCUMENT_NAME_FIELD = "document_id" - SEQUENCE_NUMBER_FIELD = "sequence_number" - - def __init__( - self, - embedding_model: BaseEmbeddingModel, - config: ChromaConfig | None = None, - distance_metric: str = "cosine", - collection_name: str | None = None, - ) -> None: - """Initialize the store and open (or create) the backing collection. - - Parameters - ---------- - embedding_model : BaseEmbeddingModel - Model used to embed documents and queries. - config : ChromaConfig | None, default=None - Connection parameters selecting the Chroma running mode (ephemeral, - persistent, or client/server). Defaults to an ephemeral in-memory - instance. - distance_metric : str, default="cosine" - Metric used to measure similarity between vectors. One of - ``"cosine"`` or ``"l2"``. - collection_name : str | None, default=None - Existing collection to reuse; must start with the ``ai4rag`` prefix. - When omitted, a new compliant name is generated. - """ - # Resolve the config once so both the base class and the client builder - # see the same instance; the default ephemeral config must not leak - # ``None`` into ``_build_client``. - config = config or ChromaConfig() - super().__init__(embedding_model, config, distance_metric, collection_name) - - # Ephemeral mode (neither host nor persist_directory) is backed by a - # single process-wide in-memory chromadb ``System`` shared across every - # ``EphemeralClient`` (keyed by the constant "ephemeral" identifier). - # This flag lets ``close()`` skip tearing that System down — see there - # for why closing an ephemeral client is actively harmful. - self._is_ephemeral = not config.host and not config.persist_directory - self._client = self._build_client(config) - self._collection = self._client.get_or_create_collection( - name=self._collection_name, - metadata={"hnsw:space": self.distance_metric}, - ) - - @staticmethod - def _build_client(config: ChromaConfig) -> ClientAPI: - """Create a Chroma client for the mode implied by *config*. - - ``host`` selects a remote client/server connection and takes precedence; - otherwise ``persist_directory`` selects an on-disk persistent client; - with neither set an ephemeral in-memory client is used. - - Parameters - ---------- - config : ChromaConfig - Connection parameters selecting the Chroma running mode. - - Returns - ------- - ClientAPI - A configured Chroma client for the selected running mode. - """ - if config.host: - return chromadb.HttpClient(host=config.host, port=config.port) - if config.persist_directory: - return chromadb.PersistentClient(path=config.persist_directory) - return chromadb.EphemeralClient() - - @property - def distance_metric(self) -> str: - """Distance metric currently used for similarity search. - - Returns - ------- - str - The active distance metric (``"cosine"`` or ``"l2"``). - """ - return self._distance_metric - - @distance_metric.setter - def distance_metric(self, value: str) -> None: - """Set the distance metric used for similarity search. - - Parameters - ---------- - value : str - Distance metric to use. One of ``"cosine"`` or ``"l2"``. - - Raises - ------ - ValueError - If the distance metric is not supported. - """ - if value not in self._supported_distance_metrics: - raise ValueError(f"Invalid distance metric: {value}. Use one of: {self._supported_distance_metrics}.") - self._distance_metric = value - - def _distance_to_similarity(self, distance: float) -> float: - """Convert a Chroma distance into a "higher = more relevant" score. - - Keeps ``include_scores`` semantics consistent with the Milvus and - PGVector stores. For ``cosine`` the returned value is the true cosine - similarity (``1 - distance``); for ``l2`` a monotonically decreasing - ``1 / (1 + distance)`` maps the unbounded distance into ``(0, 1]``. - - Parameters - ---------- - distance : float - Raw distance returned by Chroma for a matched vector. - - Returns - ------- - float - Similarity score where larger values indicate greater relevance. - """ - if self._distance_metric == "cosine": - return 1.0 - distance - return 1.0 / (1.0 + distance) - - def clear(self) -> None: - """Delete all entries while keeping the collection in place.""" - all_ids = self._collection.get()["ids"] - if all_ids: - self._collection.delete(ids=all_ids) - - def count(self) -> int: - """Count the number of entries in the collection. - - Returns - ------- - int - Number of stored chunks. - """ - return self._collection.count() - - def add_documents(self, documents: list[AI4RAGChunk], **kwargs: Any) -> list[str]: - """Embed, deduplicate, and upsert chunks into the collection. - - Parameters - ---------- - documents : list[AI4RAGChunk] - Chunks to be embedded and stored. - **kwargs : Any - Optional overrides. ``max_batch_size`` (int) sets the upsert batch - size (default :attr:`_BATCH_SIZE`). - - Returns - ------- - list[str] - IDs of the stored chunks (deduplicated by ``chunk_id``). - """ - if not documents: - return [] - - # Deduplicate by chunk_id, keeping first-seen order and last-seen content. - unique_chunks: dict[str, AI4RAGChunk] = {} - for chunk in documents: - unique_chunks[chunk.chunk_id] = chunk - - ids = list(unique_chunks.keys()) - chunks = list(unique_chunks.values()) - texts = [chunk.text for chunk in chunks] - embeddings = self.embedding_model.embed_documents(texts) - # Chroma rejects empty-dict metadata; represent "no metadata" as None. - metadatas = [chunk.metadata if chunk.metadata else None for chunk in chunks] - - batch_size = kwargs.get("max_batch_size", self._BATCH_SIZE) - for start in range(0, len(ids), batch_size): - end = start + batch_size - self._collection.upsert( - ids=ids[start:end], - documents=texts[start:end], - embeddings=embeddings[start:end], - metadatas=metadatas[start:end], # type: ignore[arg-type] - ) - return ids - - def search( - self, - query: str, - k: int = 5, - include_scores: bool = False, - **kwargs: Any, - ) -> list[AI4RAGChunk] | list[tuple[AI4RAGChunk, float]]: - """Search for chunks most similar to *query*. - - Chroma supports pure vector search only; hybrid-search keyword arguments - (``search_mode``, ``ranker_*``) forwarded by the retriever are ignored. - A metadata filter may be supplied via ``where`` (or ``filter``). - - Parameters - ---------- - query : str - Query for which grounding documents will be searched for. - k : int, default=5 - Number of documents to retrieve. - include_scores : bool, default=False - Whether to return similarity scores. Scores follow the - "higher = more relevant" convention shared across ai4rag stores. - - Returns - ------- - list[AI4RAGChunk] | list[tuple[AI4RAGChunk, float]] - Found chunks with or without scores. - """ - where = kwargs.get("where") or kwargs.get("filter") - embedding = self.embedding_model.embed_query(query) - result = self._collection.query( - query_embeddings=[embedding], - n_results=k, - where=where, - include=["documents", "metadatas", "distances"], - ) - - documents = result["documents"][0] if result["documents"] else [] - metadatas = result["metadatas"][0] if result["metadatas"] else [] - distances = result["distances"][0] if result["distances"] else [] - - chunks = [ - AI4RAGChunk(text=text, metadata=dict(metadata) if metadata else {}) - for text, metadata in zip(documents, metadatas) - ] - if include_scores: - return [(chunk, self._distance_to_similarity(distance)) for chunk, distance in zip(chunks, distances)] - return chunks - - def window_search( - self, - query: str, - k: int = 5, - include_scores: bool = False, - window_size: int = 2, - **kwargs: Any, - ) -> list[AI4RAGChunk] | list[tuple[AI4RAGChunk, float]]: - """Search for similar chunks and expand each with its neighbouring chunks. - - Each matched chunk is extended with up to ``window_size`` adjacent chunks - on either side from the same source document, then merged into a single - chunk (overlapping text is de-duplicated). - - Parameters - ---------- - query : str - Query for which grounding documents will be searched for. - k : int, default=5 - Number of documents to retrieve. - include_scores : bool, default=False - Whether similarity scores of found documents should be returned. - window_size : int, default=2 - Number of chunks from the right and left side of the original chunk. - - Returns - ------- - list[AI4RAGChunk] | list[tuple[AI4RAGChunk, float]] - Found chunks with or without scores. - """ - results = self.search(query, k, include_scores, **kwargs) - if window_size <= 0: - return results - - if not include_scores: - chunks = cast(list[AI4RAGChunk], results) - return [self._window_extend_and_merge(chunk, window_size) for chunk in chunks] - - chunks_and_scores = cast(list[tuple[AI4RAGChunk, float]], results) - chunks = [t[0] for t in chunks_and_scores] - scores = [t[1] for t in chunks_and_scores] - extended = [self._window_extend_and_merge(chunk, window_size) for chunk in chunks] - return list(zip(extended, scores)) - - def delete(self, ids: list[str], **kwargs: Any) -> None: - """Delete stored chunks by ID. - - Parameters - ---------- - ids : list[str] - IDs of the chunks to delete. - """ - self._collection.delete(ids=ids, **kwargs) - - def clean_collection(self) -> None: - """Drop the underlying Chroma collection.""" - self._client.delete_collection(self._collection_name) - - def close(self) -> None: - """Release the underlying Chroma client's operating-system resources. - - No-op for an ephemeral client. Its data lives in a process-wide, in-memory - ``System`` that chromadb shares across every ``EphemeralClient`` and - reference-counts; ``client.close()`` decrements that count and, once it - reaches zero, stops the ``System`` and discards all in-memory data — - destroying collections a later store (e.g. a subsequent HPO trial reusing - the same ``collection_name``) still depends on. An ephemeral client holds - no OS resource to release, so skipping the close leaks nothing durable: - the shared ``System`` is reclaimed when the interpreter exits. - - For a persistent client this releases the SQLite file lock (so the store - can be reopened); for an HTTP client it releases the client-side sockets. - In both cases the actual data survives on disk / on the server, so reuse - across store instances is unaffected. - """ - if self._is_ephemeral: - return - self._client.close() - - def _get_window_documents(self, doc_id: str, seq_nums_window: list[int]) -> list[AI4RAGChunk]: - """Fetch chunks of a document within a contiguous sequence-number range. - - Parameters - ---------- - doc_id : str - ID of the source document. - seq_nums_window : list[int] - Ordered sequence numbers bounding the window (first and last used). - - Returns - ------- - list[AI4RAGChunk] - Chunks of ``doc_id`` whose sequence number falls within the window. - """ - expr = { - "$and": [ - {self.DOCUMENT_NAME_FIELD: {"$eq": doc_id}}, - {self.SEQUENCE_NUMBER_FIELD: {"$gte": seq_nums_window[0]}}, - {self.SEQUENCE_NUMBER_FIELD: {"$lte": seq_nums_window[-1]}}, - ] - } - result = self._collection.get(where=expr, include=["documents", "metadatas"]) # type: ignore[arg-type] - texts, metadatas = result["documents"] or [], result["metadatas"] or [] - return [ - AI4RAGChunk(text=text, metadata=dict(metadata) if metadata else {}) - for text, metadata in zip(texts, metadatas) - ] - - def _window_extend_and_merge(self, chunk: AI4RAGChunk, window_size: int) -> AI4RAGChunk: - """Extend a chunk with its neighbours and merge them into one chunk. - - Retrieves the adjacent chunks (if any) from the same source document, - orders them by sequence number, and merges them while de-duplicating any - overlapping text. - - Parameters - ---------- - chunk : AI4RAGChunk - Chunk to be extended to its window and merged. - window_size : int - Number of adjacent chunks to retrieve before and after the center. - - Returns - ------- - AI4RAGChunk - Chunk after extending and merging. - - Raises - ------ - ValueError - If the chunk metadata lacks ``document_id`` or ``sequence_number``. - """ - if self.DOCUMENT_NAME_FIELD not in chunk.metadata: - raise ValueError(f'chunk must have "{self.DOCUMENT_NAME_FIELD}" in its metadata') - if self.SEQUENCE_NUMBER_FIELD not in chunk.metadata: - raise ValueError(f'chunk must have "{self.SEQUENCE_NUMBER_FIELD}" in its metadata') - doc_id = chunk.metadata[self.DOCUMENT_NAME_FIELD] - seq_num = chunk.metadata[self.SEQUENCE_NUMBER_FIELD] - seq_nums_window = [seq_num + i for i in range(-window_size, window_size + 1, 1)] - - window_chunks = self._get_window_documents(doc_id, seq_nums_window) - window_chunks.sort(key=lambda c: c.metadata[self.SEQUENCE_NUMBER_FIELD]) - - return merge_window_into_a_document(window_chunks) diff --git a/ai4rag/rag/vector_store/config.py b/ai4rag/rag/vector_store/config.py index 3d1dbc7f..e101c20d 100644 --- a/ai4rag/rag/vector_store/config.py +++ b/ai4rag/rag/vector_store/config.py @@ -9,14 +9,29 @@ from typing import ClassVar __all__ = [ + "SUPPORTED_PROVIDERS", "BaseVectorStoreConfig", "MilvusConfig", + "MilvusLiteConfig", "PGVectorConfig", - "ChromaConfig", "get_vector_store_config", "get_vector_store_env_vars", ] +#: Default on-disk location for an embedded Milvus Lite database when the caller +#: does not specify one. A relative path lands in the current working directory. +DEFAULT_MILVUS_LITE_DB_PATH = "./ai4rag_milvus_lite.db" + + +def _is_server_url(value: object) -> bool: + """Return whether *value* is a Milvus server URL (``http://``/``https://``). + + Shared by :meth:`MilvusConfig.__post_init__` and + :meth:`MilvusLiteConfig.__post_init__` so the two mirror-image guards can + never drift apart on which schemes count as a "server" endpoint. + """ + return isinstance(value, str) and value.startswith(("http://", "https://")) + @dataclass(frozen=True, kw_only=True) class BaseVectorStoreConfig(ABC): @@ -25,7 +40,7 @@ class BaseVectorStoreConfig(ABC): Attributes ---------- provider : str - Backend discriminator (e.g. ``"chroma"``, ``"milvus"``, ``"pgvector"``) + Backend discriminator (``"milvus"``, ``"milvus_lite"``, or ``"pgvector"``) used by :func:`ai4rag.rag.vector_store.get_vector_store.get_vector_store` to select the concrete store class. """ @@ -39,89 +54,31 @@ def from_env(cls) -> "BaseVectorStoreConfig": """Create config from environment variables.""" -@dataclass(frozen=True, kw_only=True) -class ChromaConfig(BaseVectorStoreConfig): - """Connection parameters for a Chroma instance. - - The running mode is inferred from which fields are set, so the same config - class drives all three Chroma deployment styles: - - * **Ephemeral (default)** — fully in-memory, nothing persisted, when both - ``persist_directory`` and ``host`` are ``None``. - * **Persistent** — local on-disk storage when ``persist_directory`` is set. - * **Client/server** — connect to a remote Chroma server when ``host`` is - set (``host`` takes precedence over ``persist_directory``). - - Parameters - ---------- - persist_directory : str | None, default=None - Filesystem path backing a local persistent client. ``None`` selects an - ephemeral in-memory client. - host : str | None, default=None - Hostname of a remote Chroma server. ``None`` keeps operation local - (ephemeral or persistent). - port : int, default=8000 - Port of the remote Chroma server. Used only when ``host`` is set. - provider : str, default="chroma" - Name of the provider used in the system. - - Attributes - ---------- - env_vars : ClassVar[tuple[tuple[str, str], ...]] - ``(name, description)`` pairs for the environment variables consulted by - :meth:`from_env`. Exposed for documentation and notebook generation. - """ - - env_vars: ClassVar[tuple[tuple[str, str], ...]] = ( - ("CHROMA_HOST", "Hostname of a remote Chroma server. Leave unset to run locally."), - ("CHROMA_PORT", "Port of the remote Chroma server (used with CHROMA_HOST; default 8000)."), - ( - "CHROMA_PERSIST_DIR", - "Filesystem path for a local persistent store. Unset uses an ephemeral in-memory store.", - ), - ) - - persist_directory: str | None = None - host: str | None = None - port: int = 8000 - provider: str = "chroma" - - @classmethod - def from_env(cls) -> "ChromaConfig": - """Build config from ``CHROMA_*`` environment variables. - - Reads ``CHROMA_PERSIST_DIR``, ``CHROMA_HOST`` and ``CHROMA_PORT``. - Unset variables fall back to the ephemeral in-memory defaults. - - Returns - ------- - ChromaConfig - Config populated from the ``CHROMA_*`` environment variables. - """ - return cls( - persist_directory=os.environ.get("CHROMA_PERSIST_DIR"), - host=os.environ.get("CHROMA_HOST"), - port=int(os.environ.get("CHROMA_PORT", "8000")), - ) - - @dataclass(frozen=True, kw_only=True) class MilvusConfig(BaseVectorStoreConfig): - """Connection parameters for a Milvus instance. - - TLS is driven entirely by the ``uri`` scheme, matching the ``MilvusClient`` - contract: an ``https://`` URI opens a secure gRPC channel, an ``http://`` URI - stays plaintext. When the endpoint presents a certificate signed by a - self-signed or private CA, pass the CA/server certificate as PEM text via - ``server_cert``; :class:`~ai4rag.rag.vector_store.milvus.MilvusVectorStore` - materializes it to a temporary file for pymilvus to verify against. Endpoints - with publicly trusted certificates need no ``server_cert``. + """Connection parameters for a **remote** Milvus server. + + This config targets a running Milvus (or Zilliz Cloud) instance reached over + gRPC. For an embedded, local, zero-server database use + :class:`MilvusLiteConfig` instead — the two are deliberately separate so that + a mistyped or unreachable server ``uri`` fails loudly rather than silently + spinning up a throwaway local database (a dangerous surprise in production). + + To enforce that, ``uri`` **must** be an ``http://`` or ``https://`` URL; + anything else (a bare host, a file path, an empty string) is rejected at + construction. TLS is driven by the scheme: ``https://`` opens a secure gRPC + channel, ``http://`` stays plaintext. When a remote endpoint presents a + certificate signed by a self-signed or private CA, pass the CA/server + certificate as PEM text via ``server_cert``; + :class:`~ai4rag.rag.vector_store.milvus.MilvusVectorStore` materializes it to + a temporary file for pymilvus to verify against. Endpoints with publicly + trusted certificates need no ``server_cert``. Parameters ---------- uri : str - Milvus server URI. Use ``https://host:port`` for TLS, - ``http://host:port`` for plaintext. + Milvus server endpoint. Must start with ``http://`` (plaintext) or + ``https://`` (TLS), e.g. ``https://host:19530``. token : str | None Authentication token (``"user:password"``). ``None`` for unauthenticated. server_cert : str | None @@ -136,12 +93,18 @@ class MilvusConfig(BaseVectorStoreConfig): env_vars : ClassVar[tuple[tuple[str, str], ...]] ``(name, description)`` pairs for the environment variables consulted by :meth:`from_env`. Exposed for documentation and notebook generation. + + Raises + ------ + ValueError + If ``uri`` is not an ``http://`` or ``https://`` URL. """ env_vars: ClassVar[tuple[tuple[str, str], ...]] = ( ( "MILVUS_URI", - "Milvus server URI. Use https://host:port for TLS or http://host:port for plaintext. (required)", + "Milvus server endpoint URL: https://host:port (TLS) or http://host:port (plaintext). " + "For a local embedded database, use the milvus_lite provider instead. (required)", ), ("MILVUS_TOKEN", "Authentication token in 'user:password' form. (optional)"), ("MILVUS_SERVER_CERT", "PEM-encoded CA/server certificate for self-signed TLS endpoints. (optional)"), @@ -152,6 +115,22 @@ class MilvusConfig(BaseVectorStoreConfig): server_cert: str | None = None provider: str = "milvus" + def __post_init__(self) -> None: + """Reject any ``uri`` that is not an explicit Milvus server URL. + + Guards against the footgun where an incorrect ``uri`` (a typo, a bare + hostname, or a stray path) is silently interpreted by ``MilvusClient`` as + a local Milvus Lite database file, creating a throwaway store instead of + connecting to the intended server. Local, embedded use must go through + :class:`MilvusLiteConfig`. + """ + if not _is_server_url(self.uri): + raise ValueError( + f"MilvusConfig.uri must be a Milvus server URL starting with 'http://' or 'https://', " + f"got {self.uri!r}. For a local, embedded database use MilvusLiteConfig(db_path=...) " + "(provider 'milvus_lite') instead." + ) + @classmethod def from_env(cls) -> "MilvusConfig": """Build config from ``MILVUS_*`` environment variables. @@ -169,6 +148,8 @@ def from_env(cls) -> "MilvusConfig": ------ KeyError If the required ``MILVUS_URI`` variable is not set. + ValueError + If ``MILVUS_URI`` is not an ``http://``/``https://`` URL. """ return cls( uri=os.environ["MILVUS_URI"], @@ -177,6 +158,86 @@ def from_env(cls) -> "MilvusConfig": ) +@dataclass(frozen=True, kw_only=True) +class MilvusLiteConfig(BaseVectorStoreConfig): + """Connection parameters for an **embedded, local** Milvus Lite database. + + Milvus Lite is the zero-server Milvus engine bundled with + ``pymilvus[milvus-lite]``; it stores everything in a single local file and is + the recommended lightweight option for local development, tests, and + small-scale workloads (prototyping, up to roughly one million vectors) — not + production serving. For a remote server use :class:`MilvusConfig`. + + Choosing the embedded engine is explicit: it happens only when this config is + used (provider ``"milvus_lite"``), never as a silent fallback from a + misconfigured :class:`MilvusConfig`. + + Parameters + ---------- + db_path : str, default=:data:`DEFAULT_MILVUS_LITE_DB_PATH` + Local filesystem path to the Milvus Lite database file. Created on first + use; a relative path resolves against the current working directory. + provider : str, default="milvus_lite" + Name of the provider used in the system. + + Attributes + ---------- + env_vars : ClassVar[tuple[tuple[str, str], ...]] + ``(name, description)`` pairs for the environment variables consulted by + :meth:`from_env`. Exposed for documentation and notebook generation. + + Raises + ------ + ValueError + If ``db_path`` is empty/blank, or looks like a server URL + (``http://``/``https://``). + """ + + env_vars: ClassVar[tuple[tuple[str, str], ...]] = ( + ( + "MILVUS_LITE_DB_PATH", + f"Local file path for the embedded Milvus Lite database " + f"(default {DEFAULT_MILVUS_LITE_DB_PATH}). (optional)", + ), + ) + + db_path: str = DEFAULT_MILVUS_LITE_DB_PATH + provider: str = "milvus_lite" + + def __post_init__(self) -> None: + """Reject a ``db_path`` that is blank or is actually a server URL. + + The symmetric guard to :meth:`MilvusConfig.__post_init__`: a value like + ``https://host:19530`` is a server endpoint, not a local database file, + and belongs in :class:`MilvusConfig`. An empty or whitespace-only path + is rejected here too, rather than being handed to ``MilvusClient`` where + it would surface as an opaque, hard-to-trace pymilvus error. + """ + if not isinstance(self.db_path, str) or not self.db_path.strip(): + raise ValueError( + f"MilvusLiteConfig.db_path must be a non-empty local filesystem path, got {self.db_path!r}." + ) + if _is_server_url(self.db_path): + raise ValueError( + f"MilvusLiteConfig.db_path must be a local filesystem path, not a server URL, " + f"got {self.db_path!r}. For a remote Milvus server use MilvusConfig(uri=...) " + "(provider 'milvus') instead." + ) + + @classmethod + def from_env(cls) -> "MilvusLiteConfig": + """Build config from the ``MILVUS_LITE_DB_PATH`` environment variable. + + An unset variable falls back to :data:`DEFAULT_MILVUS_LITE_DB_PATH`. + + Returns + ------- + MilvusLiteConfig + Config populated from ``MILVUS_LITE_DB_PATH`` (or the default path). + """ + return cls(db_path=os.environ.get("MILVUS_LITE_DB_PATH", DEFAULT_MILVUS_LITE_DB_PATH)) + + @dataclass(frozen=True, kw_only=True) class PGVectorConfig(BaseVectorStoreConfig): """Connection parameters for a PostgreSQL + pgvector instance. @@ -253,13 +314,27 @@ def from_env(cls) -> "PGVectorConfig": # class's ``provider`` default so the provider string has a single source of truth. # Wrapped in a read-only view so importers cannot mutate the shared mapping. _CONFIG_BY_PROVIDER: MappingProxyType[str, type[BaseVectorStoreConfig]] = MappingProxyType( - {config_cls.provider: config_cls for config_cls in (ChromaConfig, MilvusConfig, PGVectorConfig)} + {config_cls.provider: config_cls for config_cls in (MilvusConfig, MilvusLiteConfig, PGVectorConfig)} ) +#: Provider discriminators accepted by :func:`get_vector_store_config` and +#: :func:`ai4rag.rag.vector_store.get_vector_store.get_vector_store`, derived from +#: the registry above so callers outside this package (e.g. the search space +#: defaults) never have to hand-maintain a second copy of this list. +SUPPORTED_PROVIDERS: tuple[str, ...] = tuple(sorted(_CONFIG_BY_PROVIDER)) + def _resolve_config_cls(provider: str) -> type[BaseVectorStoreConfig]: """Return the config class registered for *provider*. + The single source of truth for which config class a provider discriminator + maps to; used both to build a config from the environment (below) and by + :func:`ai4rag.rag.vector_store.get_vector_store.get_vector_store` to check + that a caller-supplied config matches its declared ``provider``. Not part of + the package's public API (see ``ai4rag/rag/vector_store/__init__.py``) — + both call sites live inside this package, so a leading-underscore, directly + imported helper is enough without widening the public surface. + Raises ------ ValueError @@ -268,8 +343,9 @@ def _resolve_config_cls(provider: str) -> type[BaseVectorStoreConfig]: try: return _CONFIG_BY_PROVIDER[provider] except KeyError as exc: - supported = ", ".join(sorted(_CONFIG_BY_PROVIDER)) - raise ValueError(f"Vector store provider '{provider}' is not supported. Choose one of: {supported}.") from exc + raise ValueError( + f"Vector store provider '{provider}' is not supported. Choose one of: {', '.join(SUPPORTED_PROVIDERS)}." + ) from exc def get_vector_store_config(provider: str) -> BaseVectorStoreConfig: @@ -284,7 +360,8 @@ def get_vector_store_config(provider: str) -> BaseVectorStoreConfig: Parameters ---------- provider : str - Backend discriminator, one of ``"chroma"``, ``"milvus"`` or ``"pgvector"``. + Backend discriminator, one of ``"milvus"``, ``"milvus_lite"`` or + ``"pgvector"``. Returns ------- @@ -314,7 +391,8 @@ def get_vector_store_env_vars(provider: str) -> tuple[tuple[str, str], ...]: Parameters ---------- provider : str - Backend discriminator, one of ``"chroma"``, ``"milvus"`` or ``"pgvector"``. + Backend discriminator, one of ``"milvus"``, ``"milvus_lite"`` or + ``"pgvector"``. Returns ------- diff --git a/ai4rag/rag/vector_store/get_vector_store.py b/ai4rag/rag/vector_store/get_vector_store.py index 1bafdf97..906f394e 100644 --- a/ai4rag/rag/vector_store/get_vector_store.py +++ b/ai4rag/rag/vector_store/get_vector_store.py @@ -4,7 +4,7 @@ # ----------------------------------------------------------------------------- from ..embedding.base_model import BaseEmbeddingModel from .base_vector_store import BaseVectorStore -from .config import BaseVectorStoreConfig, ChromaConfig, MilvusConfig, PGVectorConfig +from .config import BaseVectorStoreConfig, MilvusConfig, MilvusLiteConfig, PGVectorConfig def get_vector_store( @@ -23,8 +23,10 @@ def get_vector_store( embedding_model : BaseEmbeddingModel Embedding model used for embeddings creation. - config : ChromaConfig | MilvusConfig | PGVectorConfig - Connection config for the chosen backend. + config : MilvusConfig | MilvusLiteConfig | PGVectorConfig + Connection config for the chosen backend. :class:`MilvusConfig` targets a + remote Milvus server; :class:`MilvusLiteConfig` selects the embedded, + local Milvus Lite engine. collection_name : str | None, default=None Name of an existing collection to reuse. When omitted, a new name @@ -46,21 +48,21 @@ def get_vector_store( """ match config.provider: - case "chroma": - if not isinstance(config, ChromaConfig): - raise TypeError("ChromaConfig is required when provider='chroma'.") + case "milvus": + if not isinstance(config, MilvusConfig): + raise TypeError("MilvusConfig is required when provider='milvus'.") - from .chroma import ChromaVectorStore + from .milvus import MilvusVectorStore - return ChromaVectorStore( + return MilvusVectorStore( embedding_model=embedding_model, config=config, collection_name=collection_name, ) - case "milvus": - if not isinstance(config, MilvusConfig): - raise TypeError("MilvusConfig is required when provider='milvus'.") + case "milvus_lite": + if not isinstance(config, MilvusLiteConfig): + raise TypeError("MilvusLiteConfig is required when provider='milvus_lite'.") from .milvus import MilvusVectorStore diff --git a/ai4rag/rag/vector_store/local_store.py b/ai4rag/rag/vector_store/local_store.py new file mode 100644 index 00000000..c0f802fe --- /dev/null +++ b/ai4rag/rag/vector_store/local_store.py @@ -0,0 +1,83 @@ +# ----------------------------------------------------------------------------- +# Copyright IBM Corp. 2026 +# SPDX-License-Identifier: Apache-2.0 +# ----------------------------------------------------------------------------- +"""Helpers for throwaway, file-backed Milvus Lite vector stores. + +Milvus Lite is the embedded, zero-server Milvus engine, selected simply by +pointing a :class:`~ai4rag.rag.vector_store.config.MilvusConfig` at a local file +path. This module wraps the common "disposable local index" pattern used by +model pre-selection and judge calibration, where a vector store must live only +for the duration of one evaluation and leave nothing behind on disk. It is the +local, zero-server replacement for the previously used ephemeral in-memory +Chroma store. +""" + +import tempfile +from collections.abc import Iterator +from contextlib import contextmanager +from pathlib import Path + +from ai4rag import logger +from ai4rag.rag.embedding.base_model import BaseEmbeddingModel +from ai4rag.rag.vector_store.base_vector_store import BaseVectorStore +from ai4rag.rag.vector_store.config import MilvusLiteConfig +from ai4rag.rag.vector_store.get_vector_store import get_vector_store + +__all__ = ["temporary_milvus_lite_store"] + +#: Database file name created inside each throwaway store's private directory. +_DB_FILENAME = "store.db" + + +@contextmanager +def temporary_milvus_lite_store( + embedding_model: BaseEmbeddingModel, + collection_name: str | None = None, +) -> Iterator[BaseVectorStore]: + """Yield a disposable, file-backed Milvus Lite vector store. + + Creates a private temporary directory, opens an embedded Milvus Lite store + inside it (a :class:`~ai4rag.rag.vector_store.config.MilvusLiteConfig` whose + ``db_path`` is a file in that directory), and yields the store. On exit — + normal or exceptional — the client is closed on a best-effort basis (a + failure is logged, never raised, so it cannot mask an exception from the + caller or skip cleanup) and the whole directory (database file plus any + auxiliary files Milvus Lite created) is unconditionally removed, so nothing + is left behind. + + A fresh store bound to its own file keeps at most one Milvus Lite database + open per caller at a time, matching the engine's single-writer model. + + Parameters + ---------- + embedding_model : BaseEmbeddingModel + Model used to embed documents and queries. + collection_name : str | None, default=None + Existing collection to reuse; must start with the ``ai4rag`` prefix. When + omitted, a new compliant name is generated (see + :func:`ai4rag.rag.vector_store.utils.resolve_collection_name`). + + Yields + ------ + BaseVectorStore + A Milvus Lite store bound to a temporary database file. + """ + with tempfile.TemporaryDirectory(prefix="ai4rag-milvus-lite-") as tmp_dir: + store: BaseVectorStore | None = None + try: + store = get_vector_store( + embedding_model=embedding_model, + config=MilvusLiteConfig(db_path=str(Path(tmp_dir) / _DB_FILENAME)), + collection_name=collection_name, + ) + yield store + finally: + if store is not None: + try: + store.close() + except Exception: + # Never let a close() failure skip the directory removal below + # (handled by TemporaryDirectory's own __exit__), or mask a + # real exception raised from inside the `with` block above. + logger.warning("Failed to close temporary Milvus Lite store at '%s'.", tmp_dir, exc_info=True) diff --git a/ai4rag/rag/vector_store/milvus.py b/ai4rag/rag/vector_store/milvus.py index 0c2c254a..3e005205 100644 --- a/ai4rag/rag/vector_store/milvus.py +++ b/ai4rag/rag/vector_store/milvus.py @@ -22,7 +22,7 @@ from ai4rag.rag.chunking.chunk import AI4RAGChunk from ai4rag.rag.embedding.base_model import BaseEmbeddingModel from ai4rag.rag.vector_store.base_vector_store import BaseVectorStore -from ai4rag.rag.vector_store.config import MilvusConfig +from ai4rag.rag.vector_store.config import MilvusConfig, MilvusLiteConfig from ai4rag.rag.vector_store.utils import iter_unique_chunks, resolve_embedding_dimension, validate_search_params __all__ = ["MilvusVectorStore"] @@ -84,20 +84,25 @@ def _cleanup_server_certs() -> None: class MilvusVectorStore(BaseVectorStore): - """Vector store backed by a remote Milvus instance via ``pymilvus``. + """Vector store backed by Milvus via ``pymilvus`` (remote server or Milvus Lite). - Supports both pure vector search and hybrid search (dense + BM25 sparse) - with RRF or weighted reranking, using Milvus native server-side fusion. + A single store class serves both deployment styles, selected by the config + type: :class:`~ai4rag.rag.vector_store.config.MilvusConfig` connects to a + remote server, while :class:`~ai4rag.rag.vector_store.config.MilvusLiteConfig` + opens the embedded, local Milvus Lite engine. Both support pure vector search + and hybrid search (dense + BM25 sparse) with RRF or weighted reranking, using + Milvus native server-side fusion. Parameters ---------- embedding_model : BaseEmbeddingModel Model used to embed documents and queries. - config : MilvusConfig - Connection parameters for the Milvus server. TLS is enabled by an - ``https://`` URI; when ``config.server_cert`` is set, its PEM text is - written to a temporary file and passed to ``MilvusClient`` as - ``server_pem_path`` for certificate verification. + config : MilvusConfig | MilvusLiteConfig + Connection parameters. A :class:`MilvusConfig` connects to a remote server + (TLS via an ``https://`` URI; ``config.server_cert`` supplies a self-signed + CA certificate, materialized to a temporary file and passed to + ``MilvusClient`` as ``server_pem_path``). A :class:`MilvusLiteConfig` opens + the embedded engine backed by its local ``db_path``. distance_metric : str Distance metric for vector similarity (default ``"cosine"``). collection_name : str | None @@ -111,24 +116,27 @@ class MilvusVectorStore(BaseVectorStore): def __init__( self, embedding_model: BaseEmbeddingModel, - config: MilvusConfig, + config: MilvusConfig | MilvusLiteConfig, distance_metric: str = "cosine", collection_name: str | None = None, ): """Initialize the store, open a client, and ensure the collection exists. - A ``MilvusClient`` is built from *config*; when ``config.server_cert`` is - set, its PEM text is materialized to a temporary file (see - :func:`_materialize_server_cert`) and passed as ``server_pem_path`` for - TLS verification. The target collection — with its dense, sparse/BM25, and - JSON fields — is created only when it does not already exist. + The ``MilvusClient`` is built according to the config type: a + :class:`MilvusLiteConfig` opens the embedded engine at its local + ``db_path``; a :class:`MilvusConfig` connects to a remote server and, when + ``config.server_cert`` is set, materializes its PEM text to a temporary + file (see :func:`_materialize_server_cert`) passed as ``server_pem_path`` + for TLS verification. The target collection — with its dense, sparse/BM25, + and JSON fields — is created only when it does not already exist. Parameters ---------- embedding_model : BaseEmbeddingModel Model used to embed documents and queries. - config : MilvusConfig - Connection parameters for the Milvus server. + config : MilvusConfig | MilvusLiteConfig + Connection parameters for a remote Milvus server or the embedded + Milvus Lite engine. distance_metric : str, default="cosine" Distance metric used for dense vector similarity. collection_name : str | None, default=None @@ -138,15 +146,40 @@ def __init__( super().__init__(embedding_model, config, distance_metric, collection_name) self._embedding_dimension = resolve_embedding_dimension(self.embedding_model) + self._client = MilvusClient(**self._build_connect_kwargs(config)) + + if not self._client.has_collection(self._collection_name): + self._create_collection() + + @staticmethod + def _build_connect_kwargs(config: MilvusConfig | MilvusLiteConfig) -> dict[str, Any]: + """Return the ``MilvusClient`` keyword arguments for *config*. + + For :class:`MilvusLiteConfig` the ``uri`` is the local database file path + (the embedded engine needs no auth or TLS). For :class:`MilvusConfig` the + ``uri`` is the server URL, with an optional ``token`` and, when a + self-signed ``server_cert`` is supplied, a ``server_pem_path`` pointing at + the materialized certificate file. + + Parameters + ---------- + config : MilvusConfig | MilvusLiteConfig + Connection parameters for a remote server or the embedded engine. + + Returns + ------- + dict[str, Any] + Keyword arguments to pass to ``MilvusClient``. + """ + if isinstance(config, MilvusLiteConfig): + return {"uri": config.db_path} + connect_kwargs: dict[str, Any] = {"uri": config.uri} if config.token: connect_kwargs["token"] = config.token if config.server_cert: connect_kwargs["server_pem_path"] = _materialize_server_cert(config.server_cert) - self._client = MilvusClient(**connect_kwargs) - - if not self._client.has_collection(self._collection_name): - self._create_collection() + return connect_kwargs def _create_collection(self) -> None: """Create the Milvus collection with its schema, indexes, and BM25 function. diff --git a/ai4rag/rag/vector_store/utils.py b/ai4rag/rag/vector_store/utils.py index 16ab995f..5f9459e8 100644 --- a/ai4rag/rag/vector_store/utils.py +++ b/ai4rag/rag/vector_store/utils.py @@ -23,9 +23,9 @@ #: Maximum collection name length, bounded by the tightest identifier limit #: across supported backends: PostgreSQL truncates identifiers at 63 bytes -#: (``NAMEDATALEN - 1``) and Chroma caps collection names at 63 characters. -#: Enforcing it up front turns a silent, collision-inducing truncation into an -#: explicit error. +#: (``NAMEDATALEN - 1``); Milvus allows longer names, so this is the binding +#: constraint. Enforcing it up front turns a silent, collision-inducing +#: truncation into an explicit error. _MAX_COLLECTION_NAME_LENGTH = 63 _COLLECTION_NAME_SUFFIX_ALPHABET = string.ascii_lowercase + string.digits diff --git a/ai4rag/search_space/prepare/prepare_search_space.py b/ai4rag/search_space/prepare/prepare_search_space.py index 19404b11..2834de16 100644 --- a/ai4rag/search_space/prepare/prepare_search_space.py +++ b/ai4rag/search_space/prepare/prepare_search_space.py @@ -90,7 +90,6 @@ def _apply_language_detection(foundation_models: list, benchmark_data: pd.DataFr def prepare_search_space_with_maas( payload: dict[str, Any], client: OpenAI, - vector_store_type: str = "milvus", benchmark_data: pd.DataFrame | None = None, ) -> AI4RAGSearchSpace: """Prepare an AI4RAGSearchSpace using OpenShift MaaS for model validation. @@ -124,11 +123,6 @@ def prepare_search_space_with_maas( client : OpenAI General MaaS client used for model discovery and validation. - vector_store_type : str, default="milvus" - Type of vector store. When "chroma", hybrid search parameters are - excluded from the default search space since ChromaDB does not support - hybrid search. - benchmark_data : pd.DataFrame | None, default=None Benchmark data used for language detection. If not given, models will use automatic language detection per session. @@ -176,5 +170,4 @@ def prepare_search_space_with_maas( return AI4RAGSearchSpace( params=[fms_param, ems_param, *extra_params], - vector_store_type=vector_store_type, ) diff --git a/ai4rag/search_space/src/default_search_space.py b/ai4rag/search_space/src/default_search_space.py index 07c1225c..c11838b3 100644 --- a/ai4rag/search_space/src/default_search_space.py +++ b/ai4rag/search_space/src/default_search_space.py @@ -15,8 +15,6 @@ _default_chunk_overlaps = (0, 128, 256) _default_retrieval_methods = ("simple",) _default_window_sizes = (0,) -_default_chroma_retrieval_methods = ("simple",) -_default_chroma_window_sizes = (0, 1, 3, 5) _default_numbers_of_chunks = (3, 5, 10) _default_search_modes = ("vector", "hybrid") _default_ranker_strategies = ("", "rrf", "weighted") @@ -24,50 +22,25 @@ _default_ranker_alpha = (1, 0.5) -def get_default_ai4rag_search_space_parameters(vector_store_type: str = "milvus") -> list[Parameter]: +def get_default_ai4rag_search_space_parameters() -> list[Parameter]: """Return the default search space parameters for an AI4RAG experiment. - Parameters - ---------- - vector_store_type : str, default="milvus" - Type of vector store. Supported values: ``"milvus"``, ``"pgvector"``, - and ``"chroma"``. When ``"chroma"``, hybrid search parameters are - excluded since ChromaDB does not support hybrid search. - Returns ------- list[Parameter] Parameters that will be used for creating AI4RAGSearchSpace. """ - - if vector_store_type == "chroma": - retrieval_methods = _default_chroma_retrieval_methods - window_sizes = _default_chroma_window_sizes - else: - retrieval_methods = _default_retrieval_methods - window_sizes = _default_window_sizes - default_search_space_parameters = [ Parameter(name=AI4RAGParamNames.CHUNKING_METHOD, values=_default_chunking_methods), Parameter(name=AI4RAGParamNames.CHUNK_SIZE, values=_default_chunk_sizes), Parameter(name=AI4RAGParamNames.CHUNK_OVERLAP, values=_default_chunk_overlaps), - Parameter(name=AI4RAGParamNames.RETRIEVAL_METHOD, values=retrieval_methods), - Parameter(name=AI4RAGParamNames.WINDOW_SIZE, values=window_sizes), + Parameter(name=AI4RAGParamNames.RETRIEVAL_METHOD, values=_default_retrieval_methods), + Parameter(name=AI4RAGParamNames.WINDOW_SIZE, values=_default_window_sizes), Parameter(name=AI4RAGParamNames.NUMBER_OF_CHUNKS, values=_default_numbers_of_chunks), + Parameter(name=AI4RAGParamNames.SEARCH_MODE, values=_default_search_modes), + Parameter(name=AI4RAGParamNames.RANKER_STRATEGY, values=_default_ranker_strategies), + Parameter(name=AI4RAGParamNames.RANKER_K, values=_default_ranker_k), + Parameter(name=AI4RAGParamNames.RANKER_ALPHA, values=_default_ranker_alpha), ] - if vector_store_type == "chroma": - default_search_space_parameters.append( - Parameter(name=AI4RAGParamNames.SEARCH_MODE, values=("vector",)), - ) - else: - default_search_space_parameters.extend( - [ - Parameter(name=AI4RAGParamNames.SEARCH_MODE, values=_default_search_modes), - Parameter(name=AI4RAGParamNames.RANKER_STRATEGY, values=_default_ranker_strategies), - Parameter(name=AI4RAGParamNames.RANKER_K, values=_default_ranker_k), - Parameter(name=AI4RAGParamNames.RANKER_ALPHA, values=_default_ranker_alpha), - ] - ) - return default_search_space_parameters diff --git a/ai4rag/search_space/src/search_space.py b/ai4rag/search_space/src/search_space.py index 90aacf0f..8c3c54de 100644 --- a/ai4rag/search_space/src/search_space.py +++ b/ai4rag/search_space/src/search_space.py @@ -345,12 +345,6 @@ class AI4RAGSearchSpace(SearchSpace): rules : list[RuleFunction] List of functions - called "rules" - that will be applied on each combination in the search space. - - vector_store_type : str, default="milvus" - Type of vector store. Supported values: ``"milvus"``, ``"pgvector"``, - and ``"chroma"``. When ``"chroma"``, hybrid search parameters are - excluded from the default search space since ChromaDB does not - support hybrid search. """ _base_rules = ( @@ -370,9 +364,8 @@ def __init__( self, params: list[Parameter] | None = None, rules: list[RuleFunction] | None = None, - vector_store_type: str = "milvus", ): - default_search_space_parameters = get_default_ai4rag_search_space_parameters(vector_store_type) + default_search_space_parameters = get_default_ai4rag_search_space_parameters() params = params or [] self._validate_user_params(params) @@ -380,7 +373,7 @@ def __init__( params, default_search_space_parameters ) - builtin_rules = self._base_rules + self._hybrid_rules if vector_store_type != "chroma" else self._base_rules + builtin_rules = self._base_rules + self._hybrid_rules _summed_rules = builtin_rules + rules if rules else builtin_rules super().__init__(params, _summed_rules) diff --git a/ai4rag/utils/compat.py b/ai4rag/utils/compat.py deleted file mode 100644 index f854b02b..00000000 --- a/ai4rag/utils/compat.py +++ /dev/null @@ -1,24 +0,0 @@ -# ----------------------------------------------------------------------------- -# Copyright IBM Corp. 2026 -# SPDX-License-Identifier: Apache-2.0 -# ----------------------------------------------------------------------------- -import sys - - -def ensure_sqlite3() -> None: - """Patch stdlib ``sqlite3`` with ``pysqlite3-binary`` if available. - - ChromaDB requires sqlite3 >= 3.35. On platforms with an older system - sqlite (e.g. RHEL 9), this function swaps the stdlib module with the - ``pysqlite3`` wheel so that ChromaDB (and LangChain-Chroma) can work. - - Safe to call multiple times — the patch is idempotent. - """ - if getattr(sys.modules.get("sqlite3"), "__name__", None) == "pysqlite3": - return - try: - import pysqlite3 # type: ignore[import-untyped] - - sys.modules["sqlite3"] = pysqlite3 - except ImportError: - pass diff --git a/ai4rag/utils/event_handler/base_event_handler.py b/ai4rag/utils/event_handler/base_event_handler.py index 8984fd66..80127833 100644 --- a/ai4rag/utils/event_handler/base_event_handler.py +++ b/ai4rag/utils/event_handler/base_event_handler.py @@ -45,12 +45,11 @@ class AggregateMetricPayload(TypedDict): optimization_metric: NotRequired[bool] -class VectorStoreSettings(TypedDict, total=False): +class VectorStoreSettings(TypedDict): """Vector store configuration used by a RAG pattern.""" - provider_id: str - vector_store_id: str provider_type: str + collection_name: str class ChunkingSettings(TypedDict): @@ -214,7 +213,9 @@ def on_pattern_creation( }, 'duration_seconds': 42, 'settings': { - 'vector_store_binding': {'provider_id': 'local_chroma', 'vector_store_id': 'ai4rag_20260317092550'}, + 'vector_store_binding': { + 'provider_type': 'local_milvus', 'collection_name': 'ai4rag_20260317092550' + }, 'chunking': {'method': 'recursive', 'chunk_size': 1024, 'chunk_overlap': 256}, 'embedding': { 'model_id': 'mock-em-1', diff --git a/dev_utils/run_experiment.py b/dev_utils/run_experiment.py index 6cedbc7a..9abe2528 100644 --- a/dev_utils/run_experiment.py +++ b/dev_utils/run_experiment.py @@ -38,7 +38,6 @@ # Edit configurations of search space search_space = AI4RAGSearchSpace( - vector_store_type="milvus", params=[ Parameter( name="foundation_model", diff --git a/docs/about/changelog.md b/docs/about/changelog.md index fde1704e..04e49210 100644 --- a/docs/about/changelog.md +++ b/docs/about/changelog.md @@ -7,6 +7,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 --- +## [Unreleased] + +### Removed +- **BREAKING CHANGE: Vector store** — removed the ChromaDB vector store backend (`ChromaConfig`, `ChromaVectorStore`) and the `chromadb` dependency, due to known security vulnerabilities in the `chromadb` package. `ai4rag.rag.vector_store` no longer exports `ChromaConfig`; the `"chroma"` value for `vector_store_type` is no longer accepted (only `"milvus"`, `"milvus_lite"`, and `"pgvector"` are supported) + +### Added +- **Vector store** — Milvus Lite, the embedded, zero-server mode of the Milvus backend, is now the recommended local/zero-config replacement for the removed Chroma store: use the new `MilvusLiteConfig(db_path="./ai4rag.db")` (or `MilvusLiteConfig()` for the default path). Unlike Chroma, Milvus Lite supports hybrid (dense + BM25) search. `pymilvus[milvus-lite]` is now a core dependency, so no extra installation step is needed +- **Vector store** — the Milvus config was split into `MilvusConfig` (remote server / Zilliz Cloud only, which now validates that `uri` is an `http(s)://` URL and raises `ValueError` otherwise) and a new `MilvusLiteConfig` (embedded, local only, configured via `db_path`, which conversely rejects `http(s)://` values). Previously, a single `MilvusConfig` selected between a remote server and embedded Milvus Lite based on whether `uri` looked like a URL or a local file path; a mistyped or unreachable `MILVUS_URI` could therefore be silently interpreted as a local path and create an unintended throwaway local database. That silent fallback is no longer possible — a misconfigured server URI now fails loudly instead. `ai4rag.rag.vector_store` now also exports `MilvusLiteConfig`, and the `vector_store_type` search-space parameter accepts `"milvus_lite"` in addition to `"milvus"` and `"pgvector"` + +--- + ## [0.16.0](https://github.com/IBM/ai4rag/releases/tag/v0.16.0) ### Added diff --git a/docs/api-reference/rag/vector-stores.md b/docs/api-reference/rag/vector-stores.md index c71bcbaa..0f82b8e7 100644 --- a/docs/api-reference/rag/vector-stores.md +++ b/docs/api-reference/rag/vector-stores.md @@ -6,11 +6,34 @@ for a single backend, and [`get_vector_store`](#store-selection) instantiates th matching store — the backend is chosen entirely from `config.provider`, so no separate type string is needed. Three backends are supported today: -| Backend | Config | Store | Hybrid search | -|---------|--------|-------|---------------| -| Chroma | `ChromaConfig` | `ChromaVectorStore` | ❌ vector only | -| Milvus | `MilvusConfig` | `MilvusVectorStore` | ✅ server-side dense + BM25 | -| PostgreSQL + pgvector | `PGVectorConfig` | `PGVectorStore` | ✅ dense + full-text | +| Backend | Config | Provider | Store | Hybrid search | +|---------|--------|----------|-------|---------------| +| Milvus (remote server only) | `MilvusConfig` | `"milvus"` | `MilvusVectorStore` | ✅ server-side dense + BM25 | +| Milvus Lite (embedded, local file only) | `MilvusLiteConfig` | `"milvus_lite"` | `MilvusVectorStore` | ✅ embedded dense + BM25 | +| PostgreSQL + pgvector | `PGVectorConfig` | `"pgvector"` | `PGVectorStore` | ✅ dense + full-text | + +`MilvusConfig` and `MilvusLiteConfig` both construct a `MilvusVectorStore`, but they are separate, +mutually-exclusive config classes rather than two modes of one config: + +!!! note "Why `MilvusConfig` and `MilvusLiteConfig` are separate — and why that matters" + `MilvusConfig.uri` is validated to be an `http(s)://` URL and **raises `ValueError`** for anything else + (a bare host, a local file path, an empty string). This is a deliberate safety fix: previously, a + mistyped or unreachable `MILVUS_URI` could be silently interpreted as a local file path, creating an + unintended local Milvus Lite database instead of failing — dangerous in production, where it could mask a + misconfigured deployment. That silent fallback is no longer possible: a bad `MILVUS_URI` now fails loudly + at construction time. + + To use the embedded engine, opt in explicitly with **`MilvusLiteConfig(db_path="./ai4rag.db")`** (or + `MilvusLiteConfig()` for the default path, `DEFAULT_MILVUS_LITE_DB_PATH` = `"./ai4rag_milvus_lite.db"`). + `MilvusLiteConfig` validates the inverse — it rejects `http(s)://` values in `db_path`, since those belong + in `MilvusConfig`. + +!!! warning "Milvus Lite limitations" + Milvus Lite is intended for local development, tests, and small-scale workloads, not production. It + computes BM25 statistics segment-locally rather than corpus-wide, so hybrid-search ranking fidelity (and + any benchmark/HPO scores measured against it) may not transfer exactly to a production Milvus server; and + it serializes writes, so only one process should open a given `.db` file at a time. For production or + large corpora, use a remote Milvus server (`MilvusConfig`), Zilliz Cloud, or pgvector. Every config is a frozen dataclass exposing a `from_env()` classmethod, so connection details (and secrets) can be sourced from environment variables and @@ -37,13 +60,6 @@ never embedded in generated artefacts. show_root_heading: true show_source: true -## Chroma - -::: ai4rag.rag.vector_store.chroma - options: - show_root_heading: true - show_source: true - ## Milvus ::: ai4rag.rag.vector_store.milvus diff --git a/docs/architecture/core-components.md b/docs/architecture/core-components.md index 19371afc..989ab72e 100644 --- a/docs/architecture/core-components.md +++ b/docs/architecture/core-components.md @@ -50,7 +50,7 @@ The `AI4RAGExperiment` class is the central orchestrator for the entire optimiza ```python from ai4rag.core.experiment.experiment import AI4RAGExperiment -from ai4rag.rag.vector_store import MilvusConfig # or ChromaConfig, PGVectorConfig +from ai4rag.rag.vector_store import MilvusConfig # or MilvusLiteConfig / PGVectorConfig experiment = AI4RAGExperiment( documents=documents, @@ -433,7 +433,7 @@ class AI4RAGSearchSpace(SearchSpace): ): ``` -`vector_store_type` selects which backend's hybrid-search rules apply during validation; supported values are `"milvus"`, `"pgvector"`, and `"chroma"`. +`vector_store_type` selects the target backend for search-space defaults; supported values are `"milvus"` (remote server, `MilvusConfig`), `"milvus_lite"` (embedded, local file, `MilvusLiteConfig`), and `"pgvector"` — an unsupported value raises `ValueError`. All three backends support hybrid search, so the same validation rules and defaults apply across them. **Built-in Validation Rules:** @@ -454,7 +454,7 @@ class AI4RAGSearchSpace(SearchSpace): - Verifies `chunk_size <= context_length * 0.9` (both in tokens) - 10% safety margin accounts for tokenizer divergence -**Hybrid Search Rules (only for `vector_store_type != "chroma"`):** +**Hybrid Search Rules (always applied — both `"milvus"` and `"pgvector"` support hybrid search):** 5. **Search mode ↔ ranker parameter consistency** - When `search_mode == "vector"`: all ranker params must be sentinels (`""`, `0`, `1`) diff --git a/docs/architecture/data-flow.md b/docs/architecture/data-flow.md index 6f524094..571c0f12 100644 --- a/docs/architecture/data-flow.md +++ b/docs/architecture/data-flow.md @@ -26,7 +26,7 @@ sequenceDiagram participant EM as OpenAIEmbeddingModel participant VS as VectorStore participant MaaSClient as MaaS Client - participant DB as Vector DB (Milvus/Chroma/PGVector) + participant DB as Vector DB (Milvus, incl. Milvus Lite/PGVector) Exp->>Exp: check if collection exists alt Collection exists @@ -50,7 +50,7 @@ sequenceDiagram EM-->>VS: all embeddings deactivate EM - loop Batches (backend-specific size: Milvus/Chroma 2048, PGVector 1024) + loop Batches (backend-specific size: Milvus 2048, PGVector 1024) VS->>DB: upsert(chunks + embeddings) DB-->>VS: success end @@ -155,7 +155,7 @@ def embed_documents(texts: list[str]) -> list[list[float]]: ### Vector Store Insertion -The backend is selected by `vector_store_config` (a `ChromaConfig`, `MilvusConfig`, or `PGVectorConfig`) passed to `AI4RAGExperiment`. The experiment resolves the concrete store once via `get_vector_store`, which talks **directly** to the configured backend (Milvus, Chroma, or PostgreSQL/pgvector) — there is no intermediary API server between ai4rag and the vector database. +The backend is selected by `vector_store_config` (a `MilvusConfig`, `MilvusLiteConfig`, or `PGVectorConfig`) passed to `AI4RAGExperiment`. The experiment resolves the concrete store once via `get_vector_store`, which talks **directly** to the configured backend (a remote Milvus server, embedded Milvus Lite, or PostgreSQL/pgvector) — there is no intermediary API server between ai4rag and the vector database. **Vector Store Selection:** @@ -172,7 +172,7 @@ collection_name = vector_store.collection_name # Resolved, ai4rag-prefixed name **Batch Insertion (Milvus example):** -Chunks are embedded once, then upserted directly into the backend in batches (Milvus/Chroma: 2048, PGVector: 1024): +Chunks are embedded once, then upserted directly into the backend in batches (Milvus: 2048, PGVector: 1024): ```python embeddings = embedding_model.embed_documents([chunk.text for chunk in chunks]) @@ -238,7 +238,7 @@ sequenceDiagram participant EM as OpenAIEmbeddingModel participant FM as OpenAIFoundationModel participant MaaSClient as MaaS Client - participant DB as Vector DB (Milvus/Chroma/PGVector) + participant DB as Vector DB (Milvus, incl. Milvus Lite/PGVector) Note over QR: Parallel execution (ThreadPoolExecutor) par Question 1 @@ -355,7 +355,7 @@ reference_documents = vector_store.search( ) ``` -Combines the dense vector search with a backend-native sparse/keyword search — Milvus's server-side BM25 field fused via `RRFRanker`/`WeightedRanker`, or PGVector's `tsvector` full-text search fused in-memory via `WeightedInMemoryAggregator` — then returns the fused top-k chunks. Chroma does not support `search_mode="hybrid"`. +Combines the dense vector search with a backend-native sparse/keyword search — Milvus's server-side BM25 field fused via `RRFRanker`/`WeightedRanker` (available on both a remote server and the embedded Milvus Lite engine, though Milvus Lite computes BM25 IDF statistics segment-locally rather than corpus-wide), or PGVector's `tsvector` full-text search fused in-memory via `WeightedInMemoryAggregator` — then returns the fused top-k chunks. ### Context Formatting @@ -933,8 +933,8 @@ DoclingDocument(name="doc1", ...) ```python [{"content": "Chunk 1", "embedding": [0.1, ...], "metadata": {...}}, ...] -↓ (MilvusVectorStore.add_documents / ChromaVectorStore.add_documents / PGVectorStore.add_documents) -Collection "xyz" in the configured backend (Milvus, Chroma, or PGVector) +↓ (MilvusVectorStore.add_documents / PGVectorStore.add_documents) +Collection "xyz" in the configured backend (Milvus — server or embedded Milvus Lite — or PGVector) ``` **Question → Retrieved Chunks:** @@ -942,8 +942,8 @@ Collection "xyz" in the configured backend (Milvus, Chroma, or PGVector) ```python "What is the capital of France?" ↓ (MilvusVectorStore.search — embeds via OpenAIEmbeddingModel.embed_query internally, - then queries Milvus directly; ChromaVectorStore/PGVectorStore follow the same - embed-then-query pattern against their own backend) + then queries Milvus directly (server or embedded Milvus Lite); PGVectorStore + follows the same embed-then-query pattern against its own backend) [ AI4RAGChunk(text="Paris is the capital...", metadata={...}), AI4RAGChunk(text="France's capital city...", metadata={...}), diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 7de8b2c9..27fce080 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -11,7 +11,7 @@ ai4rag is designed as a modular RAG optimization engine with clear separation of ai4rag is **LLM and Vector Database provider agnostic**. It integrates with various backends through: - **OpenAI-compatible model endpoints**: Foundation and embedding models are reached through the stock `openai` SDK, so any OpenAI-compatible endpoint works (OpenShift MaaS is the built-in integration) -- **Direct Vector Store Clients**: Chroma, Milvus, and PGVector are integrated directly +- **Direct Vector Store Clients**: Milvus (remote server, via `MilvusConfig`), Milvus Lite (embedded, local file, via `MilvusLiteConfig`), and PGVector are integrated directly - **Pluggable Components**: Foundation models, embeddings, and vector stores are all defined by abstract base classes — implement one to plug in your own provider ### Template-Based Approach @@ -119,7 +119,7 @@ graph TB **Vector Stores** (`ai4rag/rag/vector_store/`) - Stores and retrieves document embeddings -- Supports Milvus, PostgreSQL/pgvector, and ChromaDB via direct clients +- Supports Milvus (remote server, `MilvusConfig`), Milvus Lite (embedded, local file, `MilvusLiteConfig`), and PostgreSQL/pgvector via direct clients - Provides similarity search capabilities **Retrieval** (`ai4rag/rag/retrieval/`) @@ -201,7 +201,7 @@ Chunking (chunk_size, chunk_overlap) ↓ Embedding Model (embedding_model) ↓ -Vector Store (Milvus / PGVector / Chroma via vector_store_config) +Vector Store (Milvus / Milvus Lite / PGVector via vector_store_config) ``` ### Query Phase (Per Configuration) diff --git a/docs/architecture/rag-components.md b/docs/architecture/rag-components.md index 5b4be600..403899c6 100644 --- a/docs/architecture/rag-components.md +++ b/docs/architecture/rag-components.md @@ -52,12 +52,6 @@ classDiagram +add_documents(AI4RAGChunk[])* void } - class ChromaVectorStore { - +search(query, k, include_scores) AI4RAGChunk[] - +window_search(query, k, window_size) AI4RAGChunk[] - +add_documents(AI4RAGChunk[]) void - } - class MilvusVectorStore { +search(query, k, search_mode, ranker_*) AI4RAGChunk[] +add_documents(AI4RAGChunk[]) void @@ -117,7 +111,6 @@ classDiagram BaseFoundationModel <|-- OpenAIFoundationModel BaseEmbeddingModel <|-- OpenAIEmbeddingModel - BaseVectorStore <|-- ChromaVectorStore BaseVectorStore <|-- MilvusVectorStore BaseVectorStore <|-- PGVectorStore BaseChunker <|-- DoclingChunker @@ -445,7 +438,7 @@ class BaseVectorStore(ABC): **Configuration:** -Every concrete store is constructed from a typed, frozen `config` dataclass (`ChromaConfig`, `MilvusConfig`, or `PGVectorConfig`) that carries the backend's connection parameters and a `provider` discriminator (`"chroma"`, `"milvus"`, `"pgvector"`). Each config class exposes a `from_env()` classmethod that reads its own `*_ENV` variables, so connection details never need to be hardcoded in application code or generated artifacts (e.g. pattern notebooks). +Every concrete store is constructed from a typed, frozen `config` dataclass (`MilvusConfig`, `MilvusLiteConfig`, or `PGVectorConfig`) that carries the backend's connection parameters and a `provider` discriminator (`"milvus"`, `"milvus_lite"`, `"pgvector"`). `MilvusConfig` and `MilvusLiteConfig` are separate, validated classes rather than two modes of a single config: `MilvusConfig.uri` must be an `http(s)://` URL (remote server or Zilliz Cloud) and raises `ValueError` otherwise, while `MilvusLiteConfig.db_path` is a local file path and raises `ValueError` if given an `http(s)://` value. Both are served by the same `MilvusVectorStore` implementation. Each config class exposes a `from_env()` classmethod that reads its own `*_ENV` variables, so connection details never need to be hardcoded in application code or generated artifacts (e.g. pattern notebooks). **Collection naming (shared across all backends):** @@ -463,8 +456,8 @@ behaves identically: silently coerced. - **Identifier safety** — the name is sanitized into a valid identifier (non-alphanumeric characters become underscores) and bounded to 63 characters - (the tightest limit across PostgreSQL and Chroma), so it is usable verbatim as - a backend collection name *and* as a physical SQL table name. + (PostgreSQL's identifier limit), so it is usable verbatim as a backend + collection name *and* as a physical SQL table name. **Interface Methods:** @@ -506,7 +499,7 @@ vector_store = get_vector_store( ```python def get_vector_store( embedding_model: BaseEmbeddingModel, - config: ChromaConfig | MilvusConfig | PGVectorConfig, + config: MilvusConfig | MilvusLiteConfig | PGVectorConfig, collection_name: str | None = None, ) -> BaseVectorStore: """Backend selected by ``config.provider``; raises TypeError on a @@ -517,128 +510,41 @@ def get_vector_store( | Config | `provider` | Key Fields | Env Vars | |--------|------------|------------|----------| -| `ChromaConfig` | `"chroma"` | `persist_directory`, `host`, `port` | `CHROMA_HOST`, `CHROMA_PORT`, `CHROMA_PERSIST_DIR` | -| `MilvusConfig` | `"milvus"` | `uri` (required), `token`, `server_cert` | `MILVUS_URI` (required), `MILVUS_TOKEN`, `MILVUS_SERVER_CERT` | +| `MilvusConfig` | `"milvus"` | `uri` (required, must be an `http(s)://` URL — a remote server or Zilliz Cloud; raises `ValueError` otherwise), `token`, `server_cert` | `MILVUS_URI` (required, must be `http(s)://`), `MILVUS_TOKEN`, `MILVUS_SERVER_CERT` | +| `MilvusLiteConfig` | `"milvus_lite"` | `db_path` (a local file path, default `"./ai4rag_milvus_lite.db"`; raises `ValueError` if given an `http(s)://` value) | `MILVUS_LITE_DB_PATH` (optional) | | `PGVectorConfig` | `"pgvector"` | `host`, `port`, `dbname`, `user`, `password` | `PGVECTOR_HOST`, `PGVECTOR_PORT`, `PGVECTOR_DB`, `PGVECTOR_USER`, `PGVECTOR_PASSWORD` | +!!! note "Why `MilvusConfig` and `MilvusLiteConfig` are separate" + Previously, a single `MilvusConfig` selected between a remote server and embedded Milvus Lite purely from + the shape of `uri` (a server URL vs. a local file path). That meant a mistyped or unreachable `MILVUS_URI` + could be silently reinterpreted as a local path, creating an unintended throwaway local database instead + of failing — a real risk in production. `MilvusConfig` now validates `uri` and raises `ValueError` for + anything that is not an `http(s)://` URL, so a bad `MILVUS_URI` fails loudly. `MilvusLiteConfig` is the + explicit, separate opt-in for the embedded engine. + `get_vector_store_config(provider)` and `get_vector_store_env_vars(provider)` complement `get_vector_store` when only a provider string is available (e.g. when building a config from the `vector_store_type` selected on the search space): ```python from ai4rag.rag.vector_store import get_vector_store_config, get_vector_store_env_vars -config = get_vector_store_config("milvus") # MilvusConfig.from_env() +config = get_vector_store_config("milvus") # MilvusConfig.from_env() +config = get_vector_store_config("milvus_lite") # MilvusLiteConfig.from_env() env_vars = get_vector_store_env_vars("milvus") # (("MILVUS_URI", "..."), ...) ``` -### ChromaVectorStore - -In-memory ChromaDB implementation for development and testing. Chroma is **vector-only** — it does not support hybrid (dense + keyword) search: - -```python -class ChromaVectorStore(BaseVectorStore): - def __init__( - self, - embedding_model: BaseEmbeddingModel, - config: ChromaConfig | None = None, - distance_metric: str = "cosine", - collection_name: str | None = None, - **kwargs - ): -``` - -**Supported Distance Metrics:** - -- `"cosine"`: Cosine similarity (default) -- `"l2"`: Euclidean distance - -**Search Methods:** - -**1. Standard Search:** - -```python -def search( - self, - query: str, - k: int = 5, - include_scores: bool = False, - **kwargs -) -> list[AI4RAGChunk] | list[tuple[AI4RAGChunk, float]]: - """Vector similarity search.""" -``` - -**2. Window Search:** - -```python -def window_search( - self, - query: str, - k: int = 5, - window_size: int = 2, - include_scores: bool = False, - **kwargs -) -> list[AI4RAGChunk]: - """Retrieve chunks + adjacent chunks (window) from same document.""" -``` - -**Window Search Details:** - -For each retrieved chunk: -1. Extract `document_id` and `sequence_number` from metadata -2. Query vector store for chunks with: - - Same `document_id` - - `sequence_number` in `[seq - window_size, seq + window_size]` -3. Sort by `sequence_number` -4. Merge into single chunk (concatenate text) - -**Example:** - -```python -# Retrieved chunk: document_id="doc1", sequence_number=5 -# window_size=2 -# Fetches chunks with sequence_number in [3, 4, 5, 6, 7] -# Returns merged document with all 5 chunks concatenated -``` - -**Batch Document Addition:** - -```python -def add_documents(self, documents: list[AI4RAGChunk], max_batch_size: int = 2048) -> list[str]: - """Add chunks in batches of max_batch_size.""" - for batch_start in range(0, len(docs), max_batch_size): - batch = docs[batch_start : batch_start + max_batch_size] - self._vector_store.add_documents(batch, ids=ids) -``` - -**Usage:** - -```python -vector_store = ChromaVectorStore( - embedding_model=embedding_model, - distance_metric="cosine" -) - -# Index documents -vector_store.add_documents(chunked_documents) - -# Search -results = vector_store.search(query="What is X?", k=5) -# Returns: [AI4RAGChunk(...), AI4RAGChunk(...), ...] - -# Window search -results = vector_store.window_search(query="What is X?", k=5, window_size=2) -# Returns: [merged_chunk_1, merged_chunk_2, ...] -``` - ### MilvusVectorStore -Vector store backed by a remote Milvus instance via `pymilvus`, supporting both pure dense vector search and hybrid search (dense + BM25 sparse) with **server-side** fusion: +Vector store backed by `pymilvus`, supporting both pure dense vector search and hybrid search (dense + BM25 sparse) with **server-side** fusion. The same class serves two deployment modes, each configured through its own dedicated config class: + +- **Remote Milvus server** (or Zilliz Cloud) — configured via `MilvusConfig`, whose `uri` must be a `http(s)://host:port` URL. +- **Milvus Lite** — configured via `MilvusLiteConfig`, whose `db_path` (e.g. `"./ai4rag.db"`) starts the embedded, zero-server Milvus Lite engine backed by that local file. This is the local, zero-setup replacement for the previously used in-memory Chroma store: recommended for local development, tests, and small-scale workloads (prototyping, up to roughly 1M vectors), not production. Milvus Lite computes BM25 IDF statistics segment-locally rather than corpus-wide, so hybrid-search ranking fidelity — and any benchmark/HPO scores measured against it — may not transfer exactly to a production server; it also serializes writes, so only one process should open a given `.db` file at a time. ```python class MilvusVectorStore(BaseVectorStore): def __init__( self, embedding_model: BaseEmbeddingModel, - config: MilvusConfig, + config: MilvusConfig | MilvusLiteConfig, distance_metric: str = "cosine", collection_name: str | None = None, ): @@ -646,16 +552,19 @@ class MilvusVectorStore(BaseVectorStore): **Connection Configuration:** -TLS is driven entirely by the `uri` scheme: `https://` opens a secure channel, `http://` stays plaintext. For endpoints with a self-signed or private-CA certificate, pass the PEM text via `server_cert`. +For `MilvusConfig`, TLS is driven entirely by the `uri` scheme: `https://` opens a secure channel, `http://` stays plaintext. For endpoints with a self-signed or private-CA certificate, pass the PEM text via `server_cert`. `MilvusLiteConfig` has no network/TLS concerns — it only takes a local `db_path`. ```python -from ai4rag.rag.vector_store import MilvusConfig +from ai4rag.rag.vector_store import MilvusConfig, MilvusLiteConfig -# From environment: MILVUS_URI (required), MILVUS_TOKEN, MILVUS_SERVER_CERT +# Remote server, from environment: MILVUS_URI (required, http(s)://), MILVUS_TOKEN, MILVUS_SERVER_CERT config = MilvusConfig.from_env() -# Or explicit +# Remote server, explicit config = MilvusConfig(uri="https://localhost:19530", token="user:pass") + +# Embedded Milvus Lite, explicit local file (or MilvusLiteConfig() for the default path) +config = MilvusLiteConfig(db_path="./ai4rag.db") ``` **Collection Schema:** @@ -1110,7 +1019,7 @@ class Retriever: - **number_of_chunks**: Top-k parameter (how many chunks to retrieve) - **method**: Retrieval method - `"simple"`: Return top-k chunks as-is - - `"window"`: Expand each chunk to include adjacent chunks (ChromaDB only) + - `"window"`: Reserved for expanding each chunk with adjacent chunks; not distinctly implemented by the current backends (see below) - **search_mode**: Search type - `"vector"`: Dense semantic search only - `"hybrid"`: Dense + sparse (keyword) search @@ -1137,12 +1046,7 @@ def retrieve(self, query: str, **kwargs) -> list[AI4RAGChunk]: **Simple vs Window Retrieval:** -The `method` parameter determines retrieval strategy but actual implementation depends on vector store: - -- **MilvusVectorStore** / **PGVectorStore**: Always return simple chunks (no window expansion) -- **ChromaVectorStore**: - - `method="simple"`: Returns top-k chunks - - `method="window"`: Returns top-k chunks expanded with adjacent chunks +Both current backends — **MilvusVectorStore** and **PGVectorStore** — always return simple top-k chunks; neither expands a retrieved chunk with its adjacent chunks, so `method="window"` currently behaves the same as `method="simple"`. **Usage:** @@ -1158,7 +1062,7 @@ retriever = Retriever( docs = retriever.retrieve("What is X?") # Returns: [AI4RAGChunk(...), AI4RAGChunk(...), ...] (5 chunks) -# Hybrid retrieval with RRF (Milvus or PGVector; Chroma is vector-only) +# Hybrid retrieval with RRF (Milvus, incl. Milvus Lite, or PGVector) retriever = Retriever( vector_store=milvus_vector_store, number_of_chunks=5, @@ -1366,7 +1270,8 @@ embedding_model = OpenAIEmbeddingModel( ) # 4. Create vector store — a direct-client store selected by config.provider -# (swap MilvusConfig for ChromaConfig/PGVectorConfig to change backend) +# (swap MilvusConfig for MilvusLiteConfig(db_path=...) for embedded local +# storage, or PGVectorConfig for PostgreSQL/pgvector) vector_store = get_vector_store( embedding_model=embedding_model, config=MilvusConfig.from_env(), @@ -1483,9 +1388,9 @@ class CustomRAG(BaseRAGTemplate): **Vector Stores:** -1. **Use Milvus or PGVector for production** hybrid search (server-side fusion for Milvus, in-memory fusion for PGVector); Chroma is vector-only -2. **Use ChromaVectorStore** for development/testing (in-memory, simpler setup) -3. **Enable hybrid search** for keyword-heavy domains (technical docs, legal, medical) — not supported on Chroma +1. **Use a remote Milvus server or PGVector for production** hybrid search (server-side fusion for Milvus, in-memory fusion for PGVector) +2. **Use Milvus Lite** (`MilvusLiteConfig` with a local `db_path`) for development/testing (embedded, zero-server, simpler setup) +3. **Enable hybrid search** for keyword-heavy domains (technical docs, legal, medical) — supported by both backends, including Milvus Lite 4. **Tune ranker parameters** (ranker_k, ranker_alpha) via optimization **Chunking:** diff --git a/docs/development/testing.md b/docs/development/testing.md index 978d6c1b..c294a44a 100644 --- a/docs/development/testing.md +++ b/docs/development/testing.md @@ -357,17 +357,17 @@ def benchmark_data(): ### Example Functional Test ```python -from ai4rag.rag.vector_store import ChromaConfig +from ai4rag.rag.vector_store import MilvusLiteConfig -class TestExperimentChroma: - """Run experiment with chroma vector store and MaaS models.""" +class TestExperimentMilvusLite: + """Run experiment with an embedded Milvus Lite vector store and MaaS models.""" - def test_experiment_chroma_maas_models( - self, client, documents, benchmark_data, foundation_model, embedding_model + def test_experiment_milvus_lite_maas_models( + self, client, documents, benchmark_data, foundation_model, embedding_model, tmp_path ): search_space = AI4RAGSearchSpace( - vector_store_type="chroma", + vector_store_type="milvus_lite", params=[ Parameter(name="foundation_model", param_type="C", values=[foundation_model]), Parameter(name="embedding_model", param_type="C", values=[embedding_model]), @@ -382,7 +382,7 @@ class TestExperimentChroma: search_space=search_space, optimizer_settings=optimizer_settings, event_handler=LocalEventHandler(), - vector_store_config=ChromaConfig(), + vector_store_config=MilvusLiteConfig(db_path=str(tmp_path / "ai4rag.db")), # embedded Milvus Lite ) experiment.search(skip_mps=True) diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index a9b12496..c89dc8ae 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -5,14 +5,14 @@ - **Python**: 3.12 or 3.13 (strictly required) - **Operating System**: macOS or Linux - **A model provider**: a foundation model and an embedding model reachable over any OpenAI-compatible endpoint (a hosted API, a self-managed vLLM/TGI/Ollama server, or an OpenShift MaaS deployment), accessed through the `openai` SDK — or your own `BaseFoundationModel` / `BaseEmbeddingModel` implementation -- **A vector store**: Chroma (in-memory by default, no setup required), or a running Milvus/PostgreSQL (pgvector) instance for hybrid retrieval +- **A vector store**: Milvus Lite (embedded, local-file, no setup required), or a running Milvus server/PostgreSQL (pgvector) instance for server-backed retrieval — all support hybrid search !!! note "External models and vector store integration" `ai4rag` is designed to be provider-agnostic. It means you can use any model from any source as long as it satisfies `BaseFoundationModel` interface. The same rule applies to embedding model. - Vector stores are selected via a typed `vector_store_config` (`ChromaConfig`, `MilvusConfig`, or `PGVectorConfig`) passed directly to the experiment. + Vector stores are selected via a typed `vector_store_config` (`MilvusConfig` for a remote server, `MilvusLiteConfig` for embedded local storage, or `PGVectorConfig`) passed directly to the experiment. A custom vector store can also be plugged in by delivering your own `BaseVectorStore` implementation. --- @@ -29,8 +29,8 @@ This installs the core package with all required dependencies. Using `"@main"` will download and install latest version of `ai4rag`. If you want to use specific version, please use e.g. `"@v0.1.1"` -Vector store clients — `chromadb`, `pymilvus`, `pgvector`, and `asyncpg` — are core dependencies and install automatically. -No extra step is needed to use Chroma, Milvus, or PostgreSQL/pgvector as a vector store. +Vector store clients — `pymilvus` (with the `milvus-lite` extra), `pgvector`, and `asyncpg` — are core dependencies and install automatically. There is no separate vector-store extra to install. +No extra step is needed to use a remote Milvus server, embedded Milvus Lite, or PostgreSQL/pgvector as a vector store. !!! note "OCR and audio ingestion" Text extraction from born-digital documents (PDF, DOCX, Markdown, HTML, …) works out of the box. @@ -100,7 +100,7 @@ self-managed server (vLLM, TGI, Ollama, …), or an The steps below use MaaS, the provider `ai4rag` ships helpers for; to use a different endpoint, point the same `openai` client at its URL (or supply your own `BaseFoundationModel` / `BaseEmbeddingModel` implementation). The vector store is configured independently, via direct -clients (Chroma, Milvus, or PGVector) — see [Vector Store Setup](#vector-store-setup) below. +clients (remote Milvus, embedded Milvus Lite, or PGVector) — see [Vector Store Setup](#vector-store-setup) below. ### 1. Get Access to a MaaS Deployment @@ -129,22 +129,39 @@ Pick a provider and pass its config to `AI4RAGExperiment` as `vector_store_confi | Provider | Config | Hybrid search (dense + keyword) | Setup | |----------|--------|:---:|-------| -| Chroma | `ChromaConfig` | :material-close: vector-only | None — defaults to an ephemeral in-memory client | -| Milvus | `MilvusConfig` | :material-check: dense + BM25 | Requires a reachable Milvus instance | +| Milvus Lite (embedded, local file) | `MilvusLiteConfig(db_path="./ai4rag.db")` | :material-check: dense + BM25 | None — zero-server, backed by a local file | +| Milvus (server) | `MilvusConfig(uri="http(s)://host:19530")` | :material-check: dense + BM25 | Requires a reachable Milvus (or Zilliz Cloud) instance | | PGVector | `PGVectorConfig` | :material-check: dense + tsvector full-text | Requires a reachable PostgreSQL instance with the `pgvector` extension | ```python -from ai4rag.rag.vector_store import ChromaConfig, MilvusConfig, PGVectorConfig +from ai4rag.rag.vector_store import MilvusConfig, MilvusLiteConfig, PGVectorConfig -# Zero-config, in-memory (great for local experimentation) -vector_store_config = ChromaConfig() +# Zero-config, embedded Milvus Lite backed by a local file (great for local experimentation) +vector_store_config = MilvusLiteConfig(db_path="./ai4rag.db") -# Or build a config from environment variables -vector_store_config = MilvusConfig.from_env() # reads MILVUS_URI, MILVUS_TOKEN, MILVUS_SERVER_CERT +# Or build configs from environment variables +vector_store_config = MilvusConfig.from_env() # reads MILVUS_URI (must be http(s)://), MILVUS_TOKEN, MILVUS_SERVER_CERT +vector_store_config = MilvusLiteConfig.from_env() # reads MILVUS_LITE_DB_PATH (optional; defaults to "./ai4rag_milvus_lite.db") vector_store_config = PGVectorConfig.from_env() # reads PGVECTOR_HOST, PGVECTOR_PORT, PGVECTOR_DB, PGVECTOR_USER, PGVECTOR_PASSWORD ``` -Each config class exposes the environment variables it reads via its `env_vars` attribute, and can be constructed explicitly instead of from the environment, e.g. `MilvusConfig(uri="https://localhost:19530")`. +Each config class exposes the environment variables it reads via its `env_vars` attribute, and can be constructed explicitly instead of from the environment, e.g. `MilvusConfig(uri="https://localhost:19530")` for a remote server or `MilvusLiteConfig(db_path="./ai4rag.db")` for embedded local storage. + +!!! note "Why `MilvusConfig` and `MilvusLiteConfig` are separate" + `MilvusConfig` (remote Milvus server / Zilliz Cloud) validates that `uri` is an `http(s)://` URL and + **raises `ValueError`** for anything else — a bare host, a local file path, or an empty string. This is a + deliberate safety check: a mistyped or unreachable `MILVUS_URI` now fails loudly at construction time + instead of silently being interpreted as a local file path and creating a throwaway Milvus Lite database. + To opt into the embedded local engine explicitly, use `MilvusLiteConfig(db_path=...)` instead, which in + turn rejects `http(s)://` values. + +!!! warning "Milvus Lite is not a production store" + `MilvusLiteConfig` is intended for local development, tests, and small-scale workloads + (prototyping, up to roughly 1M vectors) — not production serving. For production or large corpora, use a + remote Milvus server or Zilliz Cloud via `MilvusConfig`, or pgvector. Milvus Lite + also computes BM25 statistics segment-locally rather than corpus-wide, so hybrid-search ranking fidelity — + and any benchmark/HPO scores measured on it — may not transfer exactly to a production server; and it + serializes writes, so only one process should open a given `.db` file at a time. --- diff --git a/docs/getting-started/quick-start.md b/docs/getting-started/quick-start.md index 2bdac608..d6376d97 100644 --- a/docs/getting-started/quick-start.md +++ b/docs/getting-started/quick-start.md @@ -138,7 +138,7 @@ Create a `benchmark_data.json` file with questions and ground truth answers. Eac { "question": "Which vector databases are supported?", "correct_answers": [ - "Milvus and ChromaDB are supported." + "Milvus and PostgreSQL/pgvector are supported." ], "correct_answer_document_keys": ["overview.md", "reference/stores.md"] } @@ -246,8 +246,8 @@ optimizer_settings = GAMOptSettings( !!! note "Choosing a Vector Store" The vector store is selected by passing a `vector_store_config` to `AI4RAGExperiment`: - - `ChromaConfig()` — zero-config, in-memory. Vector-only search (no hybrid/BM25). - - `MilvusConfig.from_env()` or `MilvusConfig(uri=...)` — a running Milvus server. Supports hybrid search (dense + BM25). + - `MilvusLiteConfig(db_path="./ai4rag.db")` (or `MilvusLiteConfig()`) — zero-config, embedded **Milvus Lite** backed by a local file. Supports hybrid search (dense + BM25); intended for local development, tests, and small-scale workloads, not production. + - `MilvusConfig.from_env()` or `MilvusConfig(uri="http(s)://host:19530")` — a running Milvus server (or Zilliz Cloud). `uri` must be an `http(s)://` URL — anything else (a bare host, a local file path, an empty string) raises `ValueError` rather than silently falling back to a local database. Supports hybrid search (dense + BM25). - `PGVectorConfig.from_env()` or `PGVectorConfig(host=...)` — a running PostgreSQL instance with `pgvector`. Supports hybrid search (dense + full-text). All three classes live in `ai4rag.rag.vector_store` and can be built explicitly or from environment variables via `.from_env()`. diff --git a/docs/index.md b/docs/index.md index 9e4f3986..743c443e 100644 --- a/docs/index.md +++ b/docs/index.md @@ -38,7 +38,7 @@ It accepts benchmark data, search space definition, optimizer configuration then ```python from ai4rag.core.experiment.experiment import AI4RAGExperiment from ai4rag.core.hpo.gam_opt import GAMOptSettings -from ai4rag.rag.vector_store import ChromaConfig +from ai4rag.rag.vector_store import MilvusLiteConfig from ai4rag.search_space.src.search_space import AI4RAGSearchSpace from ai4rag.utils.event_handler import LocalEventHandler from pathlib import Path @@ -54,7 +54,7 @@ experiment = AI4RAGExperiment( documents=documents, benchmark_data=benchmark_data, search_space=search_space, - vector_store_config=ChromaConfig(), + vector_store_config=MilvusLiteConfig(db_path="./ai4rag.db"), # local Milvus Lite, zero setup optimizer_settings=optimizer_settings, event_handler=LocalEventHandler( output_path=Path(__file__).parent / "ai4rag_results" @@ -128,7 +128,7 @@ graph TB - **Foundation Model**: any OpenAI-compatible chat endpoint via `OpenAIFoundationModel` — or bring your own via the `BaseFoundationModel` interface - **Embedding Model**: any OpenAI-compatible embedding endpoint via `OpenAIEmbeddingModel` — or bring your own via the `BaseEmbeddingModel` interface -- **Vector Store**: Milvus, PostgreSQL/pgvector, or Chroma via direct clients — or bring your own via the `BaseVectorStore` interface +- **Vector Store**: remote Milvus, embedded Milvus Lite, or PostgreSQL/pgvector via direct clients — or bring your own via the `BaseVectorStore` interface - **Chunking**: document splitting into smaller chunks - **Retrieval**: simple and window-based retrieval strategies - **Templates**: complete RAG implementations defined as a `RAGTemplate` @@ -142,7 +142,7 @@ To run an optimization you need a **foundation model** (for text generation) and !!! tip "Bring your own models" Not using an OpenAI-compatible endpoint? Provide your own model classes instead: any implementation of `BaseFoundationModel` / `BaseEmbeddingModel` plugs straight into an experiment. See [Provider-Agnostic Design](user-guide/provider-agnostic.md). -The vector store is independent of the model provider: connect directly to Chroma, Milvus, or PostgreSQL/pgvector via `ChromaConfig`, `MilvusConfig`, or `PGVectorConfig`. +The vector store is independent of the model provider: connect directly to a remote Milvus server via `MilvusConfig`, embedded Milvus Lite via `MilvusLiteConfig`, or PostgreSQL/pgvector via `PGVectorConfig`. --- diff --git a/docs/user-guide/evaluation.md b/docs/user-guide/evaluation.md index 82c6651b..7aa0f5b8 100644 --- a/docs/user-guide/evaluation.md +++ b/docs/user-guide/evaluation.md @@ -60,8 +60,8 @@ Faithfulness: High (answer is fully grounded in the context) ``` Question: "What vector databases does ai4rag support?" -Ground truth: ["ChromaDB and Milvus", "Milvus and ChromaDB"] -Answer: "ai4rag supports ChromaDB and Milvus." +Ground truth: ["Milvus and PGVector", "PGVector and Milvus"] +Answer: "ai4rag supports Milvus and PGVector." Answer Correctness: High (matches ground truth) ``` @@ -337,7 +337,7 @@ Your `benchmark_data.json` must follow this schema: { "question": "Which vector databases are supported?", "correct_answers": [ - "ChromaDB and Milvus" + "Milvus and PGVector" ], "correct_answer_document_keys": ["vector_stores.md", "quick_start.md"] } @@ -377,9 +377,9 @@ Provide alternative phrasings for the same correct answer: { "question": "What vector databases does ai4rag support?", "correct_answers": [ - "ChromaDB and Milvus", - "Milvus and ChromaDB", - "ChromaDB (in-memory) and Milvus" + "Milvus and PGVector", + "PGVector and Milvus", + "Milvus (including embedded Milvus Lite) and PGVector" ] } ``` diff --git a/docs/user-guide/event-handlers.md b/docs/user-guide/event-handlers.md index a143c1ea..806a84d9 100644 --- a/docs/user-guide/event-handlers.md +++ b/docs/user-guide/event-handlers.md @@ -89,7 +89,7 @@ def on_pattern_creation( "embedding": {"model_id": "...", "embedding_params": {"embedding_dimension": 768}}, "retrieval": {"method": "simple", "number_of_chunks": 5, "search_mode": "vector"}, "generation": {"model_id": "...", ...}, - "vector_store_binding": {"provider_id": "local_chroma", "vector_store_id": "..."}, + "vector_store_binding": {"provider_id": "local_milvus", "vector_store_id": "..."}, }, } ``` diff --git a/docs/user-guide/hybrid-search.md b/docs/user-guide/hybrid-search.md index 5bf88ffe..c35da1e0 100644 --- a/docs/user-guide/hybrid-search.md +++ b/docs/user-guide/hybrid-search.md @@ -39,20 +39,27 @@ Consider enabling hybrid search when: ## Prerequisites -!!! warning "Vector Store Requirement" - Hybrid search is **supported with Milvus** (`MilvusConfig`) **and PGVector** (`PGVectorConfig`). It is **NOT available with Chroma**, which is vector-only. +!!! note "Vector Store Requirement" + Hybrid search is supported by **all** built-in backends: **Milvus** (`MilvusConfig` — remote server only), + **Milvus Lite** (`MilvusLiteConfig` — embedded, local file only), and **PGVector** (`PGVectorConfig`). -Ensure your experiment is configured with: +Ensure your experiment is configured with one of these backends: ```python from ai4rag.rag.vector_store import MilvusConfig experiment = AI4RAGExperiment( - vector_store_config=MilvusConfig.from_env(), # Required for hybrid search; PGVectorConfig also works + vector_store_config=MilvusConfig.from_env(), # or MilvusLiteConfig(db_path="./ai4rag.db") for Milvus Lite; PGVectorConfig also works # ... other parameters ) ``` +!!! warning "Milvus Lite hybrid-ranking fidelity" + Milvus Lite (`MilvusLiteConfig`, backed by a local file) computes BM25 IDF statistics segment-locally rather + than corpus-wide, so hybrid-search ranking fidelity is lower than on a full Milvus server — benchmark or HPO + scores obtained against Milvus Lite may not transfer exactly to a production deployment. It is intended for + local development, tests, and small-scale workloads, not production serving. + --- ## Configuration @@ -533,23 +540,6 @@ print("Hybrid avg score:", hybrid_results["objective_value"].mean()) ## Troubleshooting -### Error: "Search mode ... is not supported with chroma vector store" - -**Cause**: Your `vector_store_config` is a `ChromaConfig` (Chroma is vector-only). - -**Solution**: Switch to Milvus or PGVector: - -```python -from ai4rag.rag.vector_store import MilvusConfig - -experiment = AI4RAGExperiment( - vector_store_config=MilvusConfig.from_env(), # or PGVectorConfig.from_env() - # ... -) -``` - ---- - ### Error: "Invalid parameter combination" **Cause**: Validation rules are rejecting your configuration. @@ -592,7 +582,7 @@ experiment = AI4RAGExperiment( Hybrid search in `ai4rag` combines the best of semantic and keyword-based retrieval: -- **Use `search_mode="hybrid"`** to enable hybrid search (requires Milvus or PGVector, i.e., `vector_store_config=MilvusConfig(...)` or `PGVectorConfig(...)`) +- **Use `search_mode="hybrid"`** to enable hybrid search (requires Milvus, Milvus Lite, or PGVector, i.e., `vector_store_config=MilvusConfig(...)`, `MilvusLiteConfig(...)`, or `PGVectorConfig(...)`) - **Choose a ranker strategy**: `"rrf"` (general-purpose), `"weighted"` (fine control), or `"normalized"` - **Configure strategy parameters**: `ranker_k` for RRF, `ranker_alpha` for weighted - **Let the optimizer explore**: Include both vector and hybrid modes to find the best approach diff --git a/docs/user-guide/provider-agnostic.md b/docs/user-guide/provider-agnostic.md index dad5ba62..8f739c24 100644 --- a/docs/user-guide/provider-agnostic.md +++ b/docs/user-guide/provider-agnostic.md @@ -13,13 +13,13 @@ Rather than locking you into a specific vendor or technology stack, `ai4rag` def 2. **Embedding Models** (for document and query embeddings) 3. **Vector Stores** (for storing and retrieving document chunks) -Concrete implementations for different providers — an OpenAI-compatible endpoint (OpenShift MaaS out of the box, accessed through the OpenAI SDK) for foundation and embedding models; Chroma, Milvus, and PGVector for vector stores — all adhere to these interfaces, making them **interchangeable** within the optimization framework. +Concrete implementations for different providers — an OpenAI-compatible endpoint (OpenShift MaaS out of the box, accessed through the OpenAI SDK) for foundation and embedding models; Milvus (remote server), Milvus Lite (embedded, local), and PGVector for vector stores — all adhere to these interfaces, making them **interchangeable** within the optimization framework. --- ## Supported Providers -For **models**, `ai4rag` speaks the OpenAI API: any OpenAI-compatible endpoint works — a hosted service, a self-managed server (vLLM, TGI, Ollama, …), or OpenShift MaaS (the integration shipped out of the box, detailed below). Not OpenAI-compatible? Implement `BaseFoundationModel` / `BaseEmbeddingModel` (see [Extending with Custom Providers](#extending-with-custom-providers)). For **vector stores**, pick from the built-in Chroma / Milvus / PGVector backends or add your own via `BaseVectorStore`. +For **models**, `ai4rag` speaks the OpenAI API: any OpenAI-compatible endpoint works — a hosted service, a self-managed server (vLLM, TGI, Ollama, …), or OpenShift MaaS (the integration shipped out of the box, detailed below). Not OpenAI-compatible? Implement `BaseFoundationModel` / `BaseEmbeddingModel` (see [Extending with Custom Providers](#extending-with-custom-providers)). For **vector stores**, pick from the built-in Milvus (remote server, via `MilvusConfig`), Milvus Lite (embedded, local, via `MilvusLiteConfig`), or PGVector backends, or add your own via `BaseVectorStore`. ### OpenShift MaaS Integration @@ -64,7 +64,7 @@ embedding_model = OpenAIEmbeddingModel( # Vector store: chosen independently of the model clients via a typed config from ai4rag.rag.vector_store import MilvusConfig -vector_store_config = MilvusConfig.from_env() +vector_store_config = MilvusConfig.from_env() # remote Milvus server; reads MILVUS_URI (http(s)://), MILVUS_TOKEN, MILVUS_SERVER_CERT ``` !!! tip "Discovering models automatically" @@ -72,34 +72,41 @@ vector_store_config = MilvusConfig.from_env() --- -### ChromaDB (In-Memory) +### Milvus Lite (Embedded, Local File) -**What it is**: An in-memory vector database perfect for development, testing, and small-scale deployments. +**What it is**: The embedded, zero-server counterpart to the Milvus backend, served by the same `MilvusVectorStore` implementation but configured through its own dedicated **`MilvusLiteConfig`** class (provider `"milvus_lite"`), distinct from `MilvusConfig`. It is the local, zero-setup option for development, testing, and small-scale deployments. **What ai4rag supports**: -- **Vector Store**: ChromaDB for document storage and retrieval +- **Vector Store**: Milvus Lite (via `MilvusLiteConfig(db_path="./ai4rag.db")`) for document storage and retrieval, including hybrid (dense + BM25) search -**Key advantage**: No external services required. Great for quick experimentation. +**Key advantage**: No external services required — data is persisted to the local file at `db_path` (default `"./ai4rag_milvus_lite.db"`, exported as `DEFAULT_MILVUS_LITE_DB_PATH`). Great for quick experimentation. + +!!! note "Why a separate config class" + `MilvusConfig` and `MilvusLiteConfig` are kept separate on purpose. `MilvusConfig` validates that `uri` is + an `http(s)://` URL and raises `ValueError` otherwise, so a mistyped or unreachable `MILVUS_URI` fails + loudly instead of silently being treated as a local file path and creating a throwaway local database. + `MilvusLiteConfig` is the explicit opt-in for the embedded engine — it validates that `db_path` is *not* + an `http(s)://` URL, rejecting that case in favor of `MilvusConfig`. **Limitations**: -- **No hybrid search**: ChromaDB doesn't support sparse embeddings or hybrid retrieval -- **In-memory by default**: Data isn't persisted between runs unless you set `persist_directory` on `ChromaConfig` -- **Not for production**: Suitable for development, not large-scale deployments +- **Not for production**: Suitable for local development and small-scale workloads (prototyping, up to roughly 1M vectors), not large-scale or production deployments — use a remote Milvus server (`MilvusConfig`), Zilliz Cloud, or pgvector instead +- **Lower hybrid-ranking fidelity**: BM25 statistics are computed segment-locally rather than corpus-wide, so hybrid search scores (and benchmark/HPO results measured against them) may not transfer exactly to a production Milvus server +- **Single writer**: Writes are serialized — only one process should open a given `.db` file at a time **Usage**: ```python # Can use with any foundation/embedding models -from ai4rag.rag.vector_store import ChromaConfig +from ai4rag.rag.vector_store import MilvusLiteConfig from ai4rag.utils.event_handler import LocalEventHandler experiment = AI4RAGExperiment( documents=documents, benchmark_data=benchmark_data, search_space=search_space, - vector_store_config=ChromaConfig(), # In-memory vector store + vector_store_config=MilvusLiteConfig(db_path="./ai4rag.db"), # embedded Milvus Lite, backed by a local file optimizer_settings=optimizer_settings, event_handler=LocalEventHandler(output_path="./output"), # required ) @@ -211,8 +218,7 @@ class BaseVectorStore(ABC): **Current implementations**: -- `ChromaVectorStore`: ChromaDB (vector-only) -- `MilvusVectorStore`: Milvus (hybrid: server-side dense + BM25) +- `MilvusVectorStore`: Milvus — serves both a remote server (`MilvusConfig`) and the embedded, local-file Milvus Lite engine (`MilvusLiteConfig`) (hybrid: dense + BM25) - `PGVectorStore`: PostgreSQL + pgvector (hybrid: dense + tsvector full-text) --- @@ -250,12 +256,12 @@ experiment = AI4RAGExperiment( --- -### Example 2: MaaS Models with ChromaDB +### Example 2: MaaS Models with Milvus Lite -Use MaaS for models, but ChromaDB for quick local development: +Use MaaS for models, but embedded Milvus Lite for quick local development: ```python -from ai4rag.rag.vector_store import ChromaConfig +from ai4rag.rag.vector_store import MilvusLiteConfig from ai4rag.core.experiment.experiment import AI4RAGExperiment from ai4rag.utils.event_handler import LocalEventHandler @@ -269,29 +275,32 @@ experiment = AI4RAGExperiment( # ... other params ] ), - vector_store_config=ChromaConfig(), # In-memory ChromaDB + vector_store_config=MilvusLiteConfig(db_path="./ai4rag.db"), # embedded Milvus Lite, local file optimizer_settings=optimizer_settings, event_handler=LocalEventHandler(output_path="./output"), # required ) ``` -!!! warning "No Hybrid Search with ChromaDB" - Remember that ChromaDB doesn't support hybrid search. If your search space includes `search_mode="hybrid"`, use `MilvusConfig` or `PGVectorConfig` instead (Chroma is vector-only). +!!! warning "Milvus Lite Is Not for Production" + Milvus Lite fully supports `search_mode="hybrid"`, but its BM25 statistics are computed segment-locally + rather than corpus-wide, so hybrid-search ranking fidelity (and any scores measured against it) may not + transfer exactly to a production server. Use a remote Milvus server, Zilliz Cloud, or `PGVectorConfig` for + production workloads. --- -## ChromaDB for Development +## Milvus Lite for Development -ChromaDB is the fastest way to get started with ai4rag without setting up external services. +Milvus Lite is the fastest way to get started with ai4rag without setting up external services — it's the embedded, local-file counterpart to a full Milvus deployment, configured via its own `MilvusLiteConfig` class rather than `MilvusConfig` (which is reserved for a remote server and validates its `uri` accordingly). ### Quick Setup -No configuration needed - just pass `vector_store_config=ChromaConfig()`: +No configuration needed beyond a local file path — just pass `vector_store_config=MilvusLiteConfig(db_path="./ai4rag.db")` (or `MilvusLiteConfig()` for the default path): ```python from pathlib import Path from ai4rag.core.experiment.experiment import AI4RAGExperiment -from ai4rag.rag.vector_store import ChromaConfig +from ai4rag.rag.vector_store import MilvusLiteConfig from ai4rag.utils.event_handler import LocalEventHandler from dev_utils.file_store import FileStore @@ -301,12 +310,12 @@ from dev_utils.utils import read_benchmark_from_json documents = FileStore(Path("./docs")).load_as_documents() benchmark_data = read_benchmark_from_json(Path("./benchmark.json")) -# Run experiment with ChromaDB (no vector database setup needed!) +# Run experiment with Milvus Lite (no external vector database setup needed!) experiment = AI4RAGExperiment( documents=documents, benchmark_data=benchmark_data, search_space=search_space, - vector_store_config=ChromaConfig(), # In-memory, zero config + vector_store_config=MilvusLiteConfig(db_path="./ai4rag.db"), # embedded, zero-server, local file optimizer_settings=optimizer_settings, event_handler=LocalEventHandler(output_path="./output"), # required ) @@ -314,19 +323,19 @@ experiment = AI4RAGExperiment( best_pattern = experiment.search() ``` -**When to use ChromaDB**: +**When to use Milvus Lite**: - Local development and testing -- Prototyping RAG configurations -- Small document sets (<1000 documents) +- Prototyping RAG configurations, including hybrid search +- Small-to-medium document sets (up to roughly 1M vectors) - Quick experiments without infrastructure setup -**When NOT to use ChromaDB**: +**When NOT to use Milvus Lite**: -- Production deployments -- Large document collections (>10,000 documents) -- Hybrid search requirements -- Persistent storage requirements +- Production deployments — use a remote Milvus server, Zilliz Cloud, or pgvector instead +- Large-scale corpora beyond the embedded engine's target range +- Workloads needing corpus-global BM25 fidelity for hybrid search +- Concurrent writers to the same store (Milvus Lite serializes writes to its `.db` file) --- @@ -453,19 +462,21 @@ class MyCustomVectorStore(BaseVectorStore): | Config class | Provider | Key connection params | Env vars (`.from_env()`) | |---|---|---|---| -| `ChromaConfig` | ChromaDB (vector-only) | `persist_directory`, `host`, `port` | `CHROMA_PERSIST_DIR`, `CHROMA_HOST`, `CHROMA_PORT` | -| `MilvusConfig` | Milvus (hybrid: dense + BM25) | `uri` (required), `token`, `server_cert` | `MILVUS_URI` (required), `MILVUS_TOKEN`, `MILVUS_SERVER_CERT` | +| `MilvusConfig` | Milvus — remote server or Zilliz Cloud only (hybrid: dense + BM25) | `uri` (required, must be `http(s)://`; raises `ValueError` otherwise), `token`, `server_cert` | `MILVUS_URI` (required, must be `http(s)://`), `MILVUS_TOKEN`, `MILVUS_SERVER_CERT` | +| `MilvusLiteConfig` | Milvus Lite — embedded, local file only (hybrid: dense + BM25) | `db_path` (local file path; defaults to `"./ai4rag_milvus_lite.db"`; raises `ValueError` if given a `http(s)://` value) | `MILVUS_LITE_DB_PATH` (optional) | | `PGVectorConfig` | PostgreSQL + pgvector (hybrid: dense + full-text) | `host`, `port`, `dbname`, `user`, `password` | `PGVECTOR_HOST`, `PGVECTOR_PORT`, `PGVECTOR_DB`, `PGVECTOR_USER`, `PGVECTOR_PASSWORD` | +Both `MilvusConfig` and `MilvusLiteConfig` are served by the same `MilvusVectorStore` implementation; they only differ in where the data lives (remote server vs. local file) and are validated to prevent mixing the two up (see the note under [Milvus Lite (Embedded, Local File)](#milvus-lite-embedded-local-file)). + Each config class is a frozen, keyword-only dataclass with a `.from_env()` classmethod that builds an instance from the environment variables above: ```python -from ai4rag.rag.vector_store import ChromaConfig, MilvusConfig, PGVectorConfig +from ai4rag.rag.vector_store import MilvusConfig, MilvusLiteConfig, PGVectorConfig -# Ephemeral in-memory Chroma (default) — no env vars required -chroma_config = ChromaConfig() +# Embedded Milvus Lite backed by a local file — no external service, no env vars required +milvus_lite_config = MilvusLiteConfig(db_path="./ai4rag.db") -# Milvus, reading MILVUS_URI / MILVUS_TOKEN / MILVUS_SERVER_CERT from the environment +# Remote Milvus server (or Zilliz Cloud), reading MILVUS_URI / MILVUS_TOKEN / MILVUS_SERVER_CERT milvus_config = MilvusConfig.from_env() # PGVector, reading PGVECTOR_HOST / PGVECTOR_PORT / PGVECTOR_DB / PGVECTOR_USER / PGVECTOR_PASSWORD @@ -478,15 +489,15 @@ Pass the resulting config as `vector_store_config` to `AI4RAGExperiment`, or bui ## Provider Comparison -| Feature | OpenShift MaaS | ChromaDB | Milvus | PGVector | +| Feature | OpenShift MaaS | Milvus Lite (embedded) | Milvus (server) | PGVector | |---------|------------|----------|--------|----------| | **Foundation Models** | Yes (any deployed chat model) | N/A | N/A | N/A | | **Embedding Models** | Yes (any deployed embedding model) | N/A | N/A | N/A | -| **Vector Store** | No (models only) | Yes (in-memory) | Yes | Yes | -| **Hybrid Search** | N/A | No | Yes (dense + BM25) | Yes (dense + full-text) | +| **Vector Store** | No (models only) | Yes (local file) | Yes | Yes | +| **Hybrid Search** | N/A | Yes (dense + BM25, segment-local IDF) | Yes (dense + BM25) | Yes (dense + full-text) | | **Setup Complexity** | Medium (MaaS deployment required) | None | Medium (server required) | Medium (server required) | | **Cost** | Self-hosted (infra cost) | Free | Self-hosted (infra cost) | Self-hosted (infra cost) | -| **Best For** | On-prem, self-hosted OpenAI-compatible models | Local dev, testing | Production, hybrid search | Production, hybrid search, existing Postgres infra | +| **Best For** | On-prem, self-hosted OpenAI-compatible models | Local dev, testing, prototyping | Production, hybrid search | Production, hybrid search, existing Postgres infra | --- @@ -497,6 +508,6 @@ Pass the resulting config as `vector_store_config` to `AI4RAGExperiment`, or bui - **Abstract base classes**: `BaseFoundationModel`, `BaseEmbeddingModel`, `BaseVectorStore` - **Extensible**: Add support for new providers by implementing base classes - **OpenShift MaaS**: OpenAI-SDK access to any deployed foundation and embedding model -- **Direct-client vector stores**: `ChromaConfig`/`ChromaVectorStore` for zero-config local development, `MilvusConfig`/`MilvusVectorStore` and `PGVectorConfig`/`PGVectorStore` for production deployments with hybrid search +- **Direct-client vector stores**: `MilvusVectorStore`, backed by either `MilvusLiteConfig` (embedded Milvus Lite for zero-config local development) or `MilvusConfig` (remote server/Zilliz Cloud for production), plus `PGVectorConfig`/`PGVectorStore` for production deployments — all with hybrid search The choice of provider doesn't affect the optimization process - ai4rag works the same regardless of which model you're using. Focus on finding the best RAG configuration for your use case, not your infrastructure. diff --git a/docs/user-guide/search-space.md b/docs/user-guide/search-space.md index 00d74a0b..49b74a71 100644 --- a/docs/user-guide/search-space.md +++ b/docs/user-guide/search-space.md @@ -251,27 +251,28 @@ Parameter( ## Default Parameters -If you don't specify certain parameters, `AI4RAGSearchSpace` uses sensible defaults. The `vector_store_type` parameter defaults to `"milvus"` and accepts `"milvus"`, `"pgvector"`, or `"chroma"`. These defaults differ slightly between Chroma (vector-only) and the hybrid-capable stores, Milvus and PGVector. +If you don't specify certain parameters, `AI4RAGSearchSpace` uses sensible defaults. The `vector_store_type` parameter defaults to `"milvus"` and accepts `"milvus"` (remote server, `MilvusConfig`), `"milvus_lite"` (embedded local, `MilvusLiteConfig`), or `"pgvector"` — an unsupported value raises `ValueError`. All three support dense and hybrid search (Milvus and Milvus Lite via server-side/embedded BM25, PGVector via PostgreSQL full-text search), so they share the same default parameter set. ### Default Values -| Parameter | Default (Milvus / PGVector) | Default (Chroma) | Type | -|-----------|----------------------|-------------------|------| -| `chunking_method` | `("recursive", "hybrid")` | `("recursive", "hybrid")` | Categorical | -| `chunk_size` | `(512, 1024, 2048)` | `(512, 1024, 2048)` | Categorical | -| `chunk_overlap` | `(0, 128, 256)` | `(0, 128, 256)` | Categorical | -| `retrieval_method` | `("simple",)` | `("simple",)` | Categorical | -| `window_size` | `(0,)` | `(0, 1, 3, 5)` | Categorical | -| `number_of_chunks` | `(3, 5, 10)` | `(3, 5, 10)` | Categorical | -| `search_mode` | `("vector", "hybrid")` | `("vector",)` | Categorical | -| `ranker_strategy` | `("", "rrf", "weighted")` | N/A | Categorical | -| `ranker_k` | `(0, 60)` | N/A | Categorical | -| `ranker_alpha` | `(1, 0.5)` | N/A | Categorical | - -!!! note "Why Different Defaults?" - - **Chroma** doesn't support hybrid search, so `search_mode` is fixed to `"vector"` and ranker parameters are excluded - - **Chroma** defaults explore a wider range of `window_size` values (`(0, 1, 3, 5)` vs `(0,)`) since it's an in-memory store (faster experimentation) - - **Milvus** and **PGVector** defaults focus on simple retrieval but include hybrid search exploration +| Parameter | Default (Milvus / Milvus Lite / PGVector) | Type | +|-----------|----------------------|------| +| `chunking_method` | `("recursive", "hybrid")` | Categorical | +| `chunk_size` | `(512, 1024, 2048)` | Categorical | +| `chunk_overlap` | `(0, 128, 256)` | Categorical | +| `retrieval_method` | `("simple",)` | Categorical | +| `window_size` | `(0,)` | Categorical | +| `number_of_chunks` | `(3, 5, 10)` | Categorical | +| `search_mode` | `("vector", "hybrid")` | Categorical | +| `ranker_strategy` | `("", "rrf", "weighted")` | Categorical | +| `ranker_k` | `(0, 60)` | Categorical | +| `ranker_alpha` | `(1, 0.5)` | Categorical | + +!!! note "Uniform Defaults Across Backends" + All three supported vector store backends (`milvus`, `milvus_lite`, and `pgvector`) support hybrid search, + so the default search space is identical across them: `search_mode` explores `"vector"` and `"hybrid"`, and + the ranker parameters (`ranker_strategy`, `ranker_k`, `ranker_alpha`) are always included. `get_vector_store_config("milvus_lite")` + returns a `MilvusLiteConfig`; see the note on Milvus Lite's hybrid-ranking fidelity in [Hybrid Search](hybrid-search.md). --- diff --git a/pyproject.toml b/pyproject.toml index 5317bf21..9b98b05f 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -30,7 +30,6 @@ dynamic = ["version"] dependencies = [ "boto3>=1.28", "docling-slim[feat-chunking]~=2.121.0", - "chromadb>=1.5,<2", "langchain-text-splitters~=1.1.0", "multiprocess>=0.70", "openai~=2.53.0", @@ -39,7 +38,7 @@ dependencies = [ "pgvector~=0.5.0", "pydantic==2.11.*", "pygam~=0.12.0", - "pymilvus~=3.0.1", + "pymilvus[milvus-lite]~=3.0.1", "ragas>=0.3,<0.5", "langchain-community<0.4.2", "scikit-learn==1.8.*", @@ -102,6 +101,10 @@ namespaces = false python_files = ["tests/**/*.py"] log_cli = false log_level = "INFO" +markers = [ + "milvus: tests that target the Milvus backend (server or embedded Milvus Lite).", + "pgvector: tests that target the PostgreSQL + pgvector backend.", +] [tool.coverage.run] # Skipping orchestration modules that require integration tests (S3, multiprocessing, full experiment runs) diff --git a/tests/functional/test_experiment.py b/tests/functional/test_experiment.py index bc2d6bb6..c9baac11 100644 --- a/tests/functional/test_experiment.py +++ b/tests/functional/test_experiment.py @@ -12,7 +12,7 @@ from ai4rag.core.experiment.experiment import AI4RAGExperiment from ai4rag.core.hpo.gam_opt import GAMOptSettings -from ai4rag.rag.vector_store.config import ChromaConfig, MilvusConfig, PGVectorConfig +from ai4rag.rag.vector_store.config import MilvusConfig, MilvusLiteConfig, PGVectorConfig from ai4rag.search_space.src.parameter import Parameter from ai4rag.search_space.src.search_space import AI4RAGSearchSpace from ai4rag.utils.event_handler import LocalEventHandler @@ -77,13 +77,14 @@ def _make_event_handler(test_name): return LocalEventHandler() -@pytest.mark.chroma -class TestExperimentChroma: - """Run experiment with chroma vector store and MaaS models.""" +@pytest.mark.milvus +class TestExperimentMilvusLite: + """Run experiment with a local, embedded Milvus Lite store and MaaS models.""" - def test_experiment_chroma_maas_models(self, documents, benchmark_data, foundation_model, embedding_model): + def test_experiment_milvus_lite_maas_models( + self, documents, benchmark_data, foundation_model, embedding_model, tmp_path + ): search_space = AI4RAGSearchSpace( - vector_store_type="chroma", params=[ Parameter(name="foundation_model", param_type="C", values=[foundation_model]), Parameter(name="embedding_model", param_type="C", values=[embedding_model]), @@ -97,8 +98,8 @@ def test_experiment_chroma_maas_models(self, documents, benchmark_data, foundati benchmark_data=benchmark_data, search_space=search_space, optimizer_settings=optimizer_settings, - event_handler=_make_event_handler("chroma_maas_models"), - vector_store_config=ChromaConfig(), + event_handler=_make_event_handler("milvus_lite_maas_models"), + vector_store_config=MilvusLiteConfig(db_path=str(tmp_path / "ai4rag.db")), ) experiment.search(skip_mps=True) @@ -119,7 +120,6 @@ class TestExperimentMilvus: def test_experiment_milvus_maas_models(self, documents, benchmark_data, foundation_model, embedding_model): search_space = AI4RAGSearchSpace( - vector_store_type="milvus", params=[ Parameter(name="foundation_model", param_type="C", values=[foundation_model]), Parameter(name="embedding_model", param_type="C", values=[embedding_model]), @@ -155,7 +155,6 @@ class TestExperimentPGVector: def test_experiment_pgvector_maas_models(self, documents, benchmark_data, foundation_model, embedding_model): search_space = AI4RAGSearchSpace( - vector_store_type="pgvector", params=[ Parameter(name="foundation_model", param_type="C", values=[foundation_model]), Parameter(name="embedding_model", param_type="C", values=[embedding_model]), @@ -185,11 +184,13 @@ def test_experiment_pgvector_maas_models(self, documents, benchmark_data, founda assert 0 <= best_eval.final_score <= 1 -@pytest.mark.chroma -class TestExperimentChromaWithKnownObservations: - """Run experiment with chroma, MaaS models, and known observations.""" +@pytest.mark.milvus +class TestExperimentMilvusLiteWithKnownObservations: + """Run experiment with a local Milvus Lite store, MaaS models, and known observations.""" - def test_experiment_chroma_known_observations(self, documents, benchmark_data, foundation_model, embedding_model): + def test_experiment_milvus_lite_known_observations( + self, documents, benchmark_data, foundation_model, embedding_model, tmp_path + ): known_observations = [ { "foundation_model": foundation_model, @@ -230,7 +231,6 @@ def test_experiment_chroma_known_observations(self, documents, benchmark_data, f ] search_space = AI4RAGSearchSpace( - vector_store_type="chroma", params=[ Parameter(name="foundation_model", param_type="C", values=[foundation_model]), Parameter(name="embedding_model", param_type="C", values=[embedding_model]), @@ -244,8 +244,8 @@ def test_experiment_chroma_known_observations(self, documents, benchmark_data, f benchmark_data=benchmark_data, search_space=search_space, optimizer_settings=optimizer_settings, - event_handler=_make_event_handler("chroma_known_observations"), - vector_store_config=ChromaConfig(), + event_handler=_make_event_handler("milvus_lite_known_observations"), + vector_store_config=MilvusLiteConfig(db_path=str(tmp_path / "ai4rag.db")), known_observations=known_observations, ) diff --git a/tests/functional/test_experiment_mocked_models.py b/tests/functional/test_experiment_mocked_models.py index 1449c18d..968230a2 100644 --- a/tests/functional/test_experiment_mocked_models.py +++ b/tests/functional/test_experiment_mocked_models.py @@ -24,7 +24,7 @@ from ai4rag.core.experiment.mps import ModelsPreSelector from ai4rag.core.hpo.random_opt import RandomOptimizer, RandomOptSettings from ai4rag.evaluator.metric import Metrics -from ai4rag.rag.vector_store.config import ChromaConfig +from ai4rag.rag.vector_store.config import MilvusLiteConfig from ai4rag.search_space.src.parameter import Parameter from ai4rag.search_space.src.search_space import AI4RAGSearchSpace from ai4rag.utils.constants import AI4RAGParamNames @@ -99,9 +99,14 @@ def benchmark_data(): ) +@pytest.fixture +def milvus_lite_config(tmp_path): + """A Milvus Lite config on a per-test temporary database (embedded, no server).""" + return MilvusLiteConfig(db_path=str(tmp_path / "ai4rag.db")) + + def _build_search_space(foundation_models, embedding_models): return AI4RAGSearchSpace( - vector_store_type="chroma", params=[ Parameter(name="foundation_model", param_type="C", values=foundation_models), Parameter(name="embedding_model", param_type="C", values=embedding_models), @@ -109,23 +114,23 @@ def _build_search_space(foundation_models, embedding_models): ) -def _make_experiment(documents, benchmark_data, foundation_models, embedding_models, **kwargs): +def _make_experiment(documents, benchmark_data, foundation_models, embedding_models, vector_store_config, **kwargs): return AI4RAGExperiment( documents=documents, benchmark_data=benchmark_data, search_space=_build_search_space(foundation_models, embedding_models), - vector_store_config=ChromaConfig(), + vector_store_config=vector_store_config, optimizer_settings=RandomOptSettings(max_evals=3), event_handler=LocalEventHandler(), **kwargs, ) -class TestExperimentChromaWithMockedModels: - """Full experiment runs with mocked models, real Chroma, and real UnitxtEvaluator.""" +class TestExperimentMilvusLiteWithMockedModels: + """Full experiment runs with mocked models, real Milvus Lite, and real UnitxtEvaluator.""" def test_mps_is_triggered_and_reduces_model_pool( - self, documents, benchmark_data, foundation_models, embedding_models + self, documents, benchmark_data, foundation_models, embedding_models, milvus_lite_config ): """ With 4 FMs (> DEFAULT_N_FOUNDATION_MODELS=3) and 3 EMs (> DEFAULT_N_EMBEDDING_MODELS=2), @@ -133,7 +138,9 @@ def test_mps_is_triggered_and_reduces_model_pool( at most DEFAULT_N_FOUNDATION_MODELS FMs and DEFAULT_N_EMBEDDING_MODELS EMs, and every selected model must belong to the original input pool. """ - experiment = _make_experiment(documents, benchmark_data, foundation_models, embedding_models) + experiment = _make_experiment( + documents, benchmark_data, foundation_models, embedding_models, milvus_lite_config + ) assert len(experiment.search_space[AI4RAGParamNames.FOUNDATION_MODEL].values) == _N_FOUNDATION_MODELS assert len(experiment.search_space[AI4RAGParamNames.EMBEDDING_MODEL].values) == _N_EMBEDDING_MODELS @@ -158,12 +165,16 @@ def test_mps_is_triggered_and_reduces_model_pool( em in embedding_models for em in em_selected ), "MPS selected an embedding model that was not in the original pool" - def test_skip_mps_preserves_full_model_pool(self, documents, benchmark_data, foundation_models, embedding_models): + def test_skip_mps_preserves_full_model_pool( + self, documents, benchmark_data, foundation_models, embedding_models, milvus_lite_config + ): """ When skip_mps=True, MPS is bypassed entirely. The search space must retain all originally provided models after search() completes. """ - experiment = _make_experiment(documents, benchmark_data, foundation_models, embedding_models) + experiment = _make_experiment( + documents, benchmark_data, foundation_models, embedding_models, milvus_lite_config + ) experiment.search(optimizer=RandomOptimizer, skip_mps=True) @@ -177,7 +188,9 @@ def test_skip_mps_preserves_full_model_pool(self, documents, benchmark_data, fou f"With skip_mps=True, all {_N_EMBEDDING_MODELS} embedding models should remain, " f"got {len(em_after)}" ) - def test_evaluation_scores_are_in_valid_range(self, documents, benchmark_data, foundation_models, embedding_models): + def test_evaluation_scores_are_in_valid_range( + self, documents, benchmark_data, foundation_models, embedding_models, milvus_lite_config + ): """ Every EvaluationResult produced by the experiment must have a final_score in [0, 1] and per-metric mean scores that are either None or in [0, 1]. @@ -188,6 +201,7 @@ def test_evaluation_scores_are_in_valid_range(self, documents, benchmark_data, f benchmark_data, foundation_models, embedding_models, + milvus_lite_config, optimization_metric=Metrics.FAITHFULNESS, metrics=(Metrics.FAITHFULNESS, Metrics.ANSWER_CORRECTNESS, Metrics.CONTEXT_CORRECTNESS), ) diff --git a/tests/functional/test_ragas_evaluator.py b/tests/functional/test_ragas_evaluator.py index f198b4f6..98a6575a 100644 --- a/tests/functional/test_ragas_evaluator.py +++ b/tests/functional/test_ragas_evaluator.py @@ -18,9 +18,9 @@ results in the unit range. 2. ``TestRagasEvaluatorInExperiment`` wires a ``RagasEvaluator`` into a full - :class:`AI4RAGExperiment` run (real Chroma, mocked search-space models) with - only the RAGAS scoring step stubbed, verifying that RAGAS metrics are routed - to the evaluator and land in the experiment results. + :class:`AI4RAGExperiment` run (real Milvus Lite, mocked search-space models) + with only the RAGAS scoring step stubbed, verifying that RAGAS metrics are + routed to the evaluator and land in the experiment results. """ import importlib.util @@ -36,7 +36,7 @@ from ai4rag.evaluator.metric import Metrics from ai4rag.evaluator.ragas_evaluator import RagasEvaluator from ai4rag.evaluator.unitxt_evaluator import UnitxtEvaluator -from ai4rag.rag.vector_store.config import ChromaConfig +from ai4rag.rag.vector_store.config import MilvusLiteConfig from ai4rag.search_space.src.parameter import Parameter from ai4rag.search_space.src.search_space import AI4RAGSearchSpace from ai4rag.utils.event_handler import LocalEventHandler @@ -230,7 +230,7 @@ def benchmark_data(): class TestRagasEvaluatorInExperiment: """RAGAS evaluator wired into a full AI4RAGExperiment run.""" - def test_ragas_metric_flows_into_experiment_results(self, documents, benchmark_data, monkeypatch): + def test_ragas_metric_flows_into_experiment_results(self, documents, benchmark_data, monkeypatch, tmp_path): """A ``RagasEvaluator`` in the evaluator list must score RAGAS metrics. The RAGAS scoring call itself is stubbed (``_run_ragas`` returns a fixed @@ -252,7 +252,6 @@ def _fake_run_ragas(dataset, ragas_metrics): # pylint: disable=unused-argument for i in range(2) ] search_space = AI4RAGSearchSpace( - vector_store_type="chroma", params=[ Parameter(name="foundation_model", param_type="C", values=foundation_models), Parameter(name="embedding_model", param_type="C", values=embedding_models), @@ -263,7 +262,7 @@ def _fake_run_ragas(dataset, ragas_metrics): # pylint: disable=unused-argument documents=documents, benchmark_data=benchmark_data, search_space=search_space, - vector_store_config=ChromaConfig(), + vector_store_config=MilvusLiteConfig(db_path=str(tmp_path / "ai4rag.db")), optimizer_settings=RandomOptSettings(max_evals=2), event_handler=LocalEventHandler(), evaluators=[UnitxtEvaluator(), ragas_evaluator], diff --git a/tests/functional/vector_store/conftest.py b/tests/functional/vector_store/conftest.py index b01a9e25..db31fa07 100644 --- a/tests/functional/vector_store/conftest.py +++ b/tests/functional/vector_store/conftest.py @@ -4,14 +4,11 @@ # ----------------------------------------------------------------------------- """Shared fixtures for the vector store functional (semantic-retrieval) suite. -Each backend gets its own module (``test_chroma``, ``test_milvus``, -``test_pgvector``) rather than a single parametrized test, because the backends -diverge: - -* **Chroma** runs fully in-memory and needs no server; it has no lexical search, - so it only ever exercises dense semantic retrieval. -* **Milvus** and **pgvector** require a live database and will additionally grow - lexical / hybrid-search tests that do not apply to Chroma. +Each backend gets its own module (``test_milvus``, ``test_pgvector``) rather than +a single parametrized test, because the backends diverge in setup and in the +lexical / hybrid-search tests they grow. **Milvus** covers both a remote server +and the embedded Milvus Lite engine (a local ``.db`` file, no server); both +Milvus and pgvector support dense and hybrid (dense + lexical) search. Per-backend modules keep each backend's setup, teardown, and future backend-specific tests isolated, while everything the backends *share* lives diff --git a/tests/functional/vector_store/test_chroma.py b/tests/functional/vector_store/test_chroma.py deleted file mode 100644 index 68e92772..00000000 --- a/tests/functional/vector_store/test_chroma.py +++ /dev/null @@ -1,33 +0,0 @@ -# ----------------------------------------------------------------------------- -# Copyright IBM Corp. 2026 -# SPDX-License-Identifier: Apache-2.0 -# ----------------------------------------------------------------------------- -"""Functional semantic-retrieval test for :class:`ChromaVectorStore`. - -Chroma runs fully in-memory here (an ephemeral :class:`ChromaConfig`), so this -test needs no Chroma server — only MaaS credentials for the real embedding model, -which the shared ``embedding_model`` fixture enforces by skipping when they are -absent. Chroma has no lexical search, so — unlike the Milvus and pgvector -modules — this one covers dense semantic retrieval only. -""" - -import pytest - -from ai4rag.rag.vector_store.chroma import ChromaVectorStore -from ai4rag.rag.vector_store.config import ChromaConfig - - -@pytest.fixture(scope="module") -def vector_store(embedding_model, story_chunks): - """Build an in-memory Chroma store populated with the story; drop it on teardown.""" - store = ChromaVectorStore(embedding_model=embedding_model, config=ChromaConfig()) - store.add_documents(story_chunks) - try: - yield store - finally: - store.clean_collection() - - -def test_question_retrieves_expected_passage(vector_store, check_retrieval): - """Each story question retrieves the single passage that answers it.""" - check_retrieval(lambda question: vector_store.search(question, k=1)) diff --git a/tests/functional/vector_store/test_milvus_lite.py b/tests/functional/vector_store/test_milvus_lite.py new file mode 100644 index 00000000..3a06157b --- /dev/null +++ b/tests/functional/vector_store/test_milvus_lite.py @@ -0,0 +1,56 @@ +# ----------------------------------------------------------------------------- +# Copyright IBM Corp. 2026 +# SPDX-License-Identifier: Apache-2.0 +# ----------------------------------------------------------------------------- +"""Functional test for concurrent querying of Milvus Lite with real MaaS embeddings. + +Reproduces the experiment's concurrent retrieval fan-out — ``query_rag`` runs +every question's retrieval on a thread pool against one shared store (see +:func:`ai4rag.core.experiment.utils.query_rag`) — using a *real* MaaS embedding +model and a local, embedded Milvus Lite store. It verifies that the concurrent +embed-then-search path returns the correct passage for every question with no +errors: the semantic guarantee an experiment depends on when it queries the +store from many threads at once. + +Needs MaaS credentials (real embeddings); Milvus Lite itself needs no server, so +the store runs from a local ``.db`` file. +""" + +import concurrent.futures as cf + +import pytest + +from ai4rag.rag.vector_store.config import MilvusLiteConfig +from ai4rag.rag.vector_store.milvus import MilvusVectorStore +from tests.functional.vector_store.conftest import STORY_QUESTIONS + + +@pytest.fixture +def vector_store(embedding_model, story_chunks, tmp_path): + """A populated, embedded Milvus Lite store on a per-test temporary database.""" + store = MilvusVectorStore( + embedding_model=embedding_model, config=MilvusLiteConfig(db_path=str(tmp_path / "concurrent.db")) + ) + store.add_documents(story_chunks) + try: + yield store + finally: + store.clean_collection() + store.close() + + +def test_concurrent_retrieval_returns_expected_passages(vector_store, check_retrieval): + """Every story question, retrieved concurrently, still returns its answer passage.""" + questions = [question for question, _ in STORY_QUESTIONS] + + with cf.ThreadPoolExecutor(max_workers=len(questions)) as executor: + # Fire every question's retrieval concurrently against the one shared store, + # mirroring query_rag(max_threads=...). ``map`` re-raises the first worker + # exception on iteration, so a concurrency failure fails the test here. + results_by_question = dict( + zip(questions, executor.map(lambda question: vector_store.search(question, k=1), questions)) + ) + + # Reuse the shared correctness assertion, now served from the concurrently + # computed results: each question's top hit must be the passage that answers it. + check_retrieval(lambda question: results_by_question[question]) diff --git a/tests/integration/vector_store/conftest.py b/tests/integration/vector_store/conftest.py index be58a355..7822b388 100644 --- a/tests/integration/vector_store/conftest.py +++ b/tests/integration/vector_store/conftest.py @@ -4,9 +4,9 @@ # ----------------------------------------------------------------------------- """Shared fixtures for the vector store integration suite. -Unlike the unit tests — which run against in-memory or mocked backends — the +Unlike the unit tests — which run against embedded or mocked backends — the modules in this package exercise the concrete vector stores against **real, -externally provisioned databases** (Chroma server, Milvus, PostgreSQL+pgvector). +externally provisioned databases** (a Milvus server and PostgreSQL+pgvector). Each backend module is skipped unless the connection settings for that backend are present in the environment, so the suite is safe to run anywhere: it simply skips the backends that are not reachable. diff --git a/tests/integration/vector_store/test_chroma.py b/tests/integration/vector_store/test_chroma.py deleted file mode 100644 index 7ee2d3d0..00000000 --- a/tests/integration/vector_store/test_chroma.py +++ /dev/null @@ -1,97 +0,0 @@ -# ----------------------------------------------------------------------------- -# Copyright IBM Corp. 2026 -# SPDX-License-Identifier: Apache-2.0 -# ----------------------------------------------------------------------------- -"""Integration tests for :class:`ChromaVectorStore` against a live Chroma server. - -These tests exercise the full collection lifecycle — create, add embeddings, -search, drop — over the network against a running Chroma server, and are skipped -unless ``CHROMA_HOST`` is set (the in-memory and persistent modes are covered by -the unit suite). Connection settings are read via :meth:`ChromaConfig.from_env` -(``CHROMA_HOST``, ``CHROMA_PORT``, ``CHROMA_PERSIST_DIR``); see -``tests/integration/conftest.py`` for how ``.env.local`` is loaded. -If no env variables are set, in-memory chroma with no persistence is used. -""" - -import pytest - -from ai4rag.rag.chunking.chunk import AI4RAGChunk -from ai4rag.rag.vector_store.chroma import ChromaVectorStore -from ai4rag.rag.vector_store.config import ChromaConfig - - -def _collection_exists(store: ChromaVectorStore, name: str) -> bool: - """Return whether a collection named *name* exists on the connected server.""" - return name in {collection.name for collection in store._client.list_collections()} - - -@pytest.mark.chroma -class TestChromaIntegration: - """Full create → add → search → drop lifecycle against a live Chroma server. - - A single class-scoped ``vector_store`` fixture owns the lifecycle: it creates - and populates the collection on setup and drops it on teardown, so the - remaining test methods are order-independent, read-only assertions over the - same populated collection. - """ - - @staticmethod - @pytest.fixture(scope="class") - def vector_store(embedding_model, sample_chunks): - """Create and populate a uniquely named collection; drop it on teardown.""" - store = ChromaVectorStore(embedding_model=embedding_model, config=ChromaConfig.from_env()) - store.add_documents(sample_chunks) - try: - yield store - finally: - store.clean_collection() - - def test_collection_is_created(self, vector_store): - """The store's collection exists on the connected server.""" - assert vector_store.collection_name.startswith("ai4rag") - assert _collection_exists(vector_store, vector_store.collection_name) - - def test_all_documents_are_added(self, vector_store, sample_chunks): - """Every added chunk is persisted and independently retrievable.""" - assert vector_store.count() == len(sample_chunks) - for chunk in sample_chunks: - results = vector_store.search(chunk.text, k=len(sample_chunks)) - assert chunk.text in {result.text for result in results} - - def test_search_returns_relevant_chunk(self, vector_store, sample_chunks): - """A query for a chunk's exact text ranks that chunk first, metadata intact.""" - target = sample_chunks[0] - results = vector_store.search(target.text, k=len(sample_chunks)) - - assert results, "search returned no results" - assert results[0].text == target.text - assert results[0].metadata["document_id"] == target.metadata["document_id"] - - def test_search_respects_k(self, vector_store, sample_chunks): - """``k`` bounds the number of returned chunks.""" - results = vector_store.search(sample_chunks[0].text, k=2) - assert len(results) == 2 - - def test_search_with_scores_is_ranked(self, vector_store, sample_chunks): - """``include_scores`` returns (chunk, score) tuples ordered best-first.""" - target = sample_chunks[0] - results = vector_store.search(target.text, k=len(sample_chunks), include_scores=True) - - assert all(isinstance(result, tuple) for result in results) - top_chunk, top_score = results[0] - assert top_chunk.text == target.text - # Exact match is the most similar, so scores are non-increasing. - scores = [score for _, score in results] - assert scores == sorted(scores, reverse=True) - assert top_score == pytest.approx(1.0, abs=1e-4) - - def test_clean_collection_removes_it(self, embedding_model): - """Dropping a collection removes it from the server.""" - store = ChromaVectorStore(embedding_model=embedding_model, config=ChromaConfig.from_env()) - store.add_documents([AI4RAGChunk(text="ephemeral", metadata={"document_id": "tmp", "sequence_number": 0})]) - name = store.collection_name - assert _collection_exists(store, name) - - store.clean_collection() - - assert not _collection_exists(store, name) diff --git a/tests/integration/vector_store/test_milvus_lite.py b/tests/integration/vector_store/test_milvus_lite.py new file mode 100644 index 00000000..87d5c224 --- /dev/null +++ b/tests/integration/vector_store/test_milvus_lite.py @@ -0,0 +1,86 @@ +# ----------------------------------------------------------------------------- +# Copyright IBM Corp. 2026 +# SPDX-License-Identifier: Apache-2.0 +# ----------------------------------------------------------------------------- +"""Integration test for concurrent querying of an embedded Milvus Lite store. + +The experiment fans retrieval out across a thread pool: ``query_rag`` runs +``ThreadPoolExecutor(max_workers=...).map(...)`` so every question's +``retriever.search()`` executes concurrently against a *single shared* vector +store and client (see :func:`ai4rag.core.experiment.utils.query_rag`). Milvus +Lite is one embedded engine backed by a local ``.db`` file, so this test +verifies the storage-layer contract the experiment relies on: many concurrent +reads on one shared client return correct, uncorrupted results and never raise. + +Milvus Lite needs no server (a local file ``uri`` selects it), so — unlike the +remote-Milvus module in this package — this test is *not* gated on ``MILVUS_URI`` +and runs anywhere. Embeddings are the local ``DeterministicEmbeddingModel`` from +``conftest`` (no MaaS), so search ordering is reproducible: a query for a chunk's +exact text is guaranteed to rank that chunk first. +""" + +import concurrent.futures as cf + +import pytest + +from ai4rag.rag.vector_store.config import MilvusLiteConfig +from ai4rag.rag.vector_store.milvus import MilvusVectorStore + +#: Concurrent workers, matching ``query_rag``'s default ``max_threads``. +_MAX_WORKERS = 10 +#: Total concurrent queries — many more than there are chunks, so workers contend +#: on the shared client and every chunk is queried repeatedly under load. +_QUERY_COUNT = 100 + + +@pytest.fixture +def vector_store(embedding_model, sample_chunks, tmp_path): + """A populated, embedded Milvus Lite store on a per-test temporary database.""" + store = MilvusVectorStore( + embedding_model=embedding_model, config=MilvusLiteConfig(db_path=str(tmp_path / "concurrent.db")) + ) + store.add_documents(sample_chunks) + try: + yield store + finally: + store.clean_collection() + store.close() + + +@pytest.mark.milvus +def test_concurrent_search_is_correct_and_error_free(vector_store, sample_chunks): + """Concurrent dense and hybrid reads on one shared store stay correct and error-free. + + Reproduces the experiment's retrieval fan-out and alternates the two search + modes an experiment may run (dense ``"vector"`` and dense + BM25 ``"hybrid"``) + so both server-side paths are exercised under contention. + """ + queries = [sample_chunks[i % len(sample_chunks)] for i in range(_QUERY_COUNT)] + + def _search(chunk): + # Alternate modes by the chunk's sequence number so the concurrent load + # mixes pure-vector and hybrid (fused) searches, as an experiment might. + hybrid = chunk.metadata["sequence_number"] % 2 == 1 + results = vector_store.search( + chunk.text, + k=len(sample_chunks), + search_mode="hybrid" if hybrid else "vector", + ranker_strategy="rrf" if hybrid else None, + ) + return chunk.text, hybrid, results + + with cf.ThreadPoolExecutor(max_workers=_MAX_WORKERS) as executor: + # ``map`` re-raises the first worker exception when its result is consumed, + # so any concurrency failure (e.g. an embedded-engine race) fails the test. + outcomes = list(executor.map(_search, queries)) + + assert len(outcomes) == _QUERY_COUNT + for query_text, hybrid, results in outcomes: + assert results, "a concurrent search returned no results" + if hybrid: + # Fusion may reorder ties, but the exact-text match must still surface. + assert query_text in {result.text for result in results} + else: + # A pure-vector query for a chunk's own text ranks that chunk first; + # a wrong top hit would betray results cross-contaminated across threads. + assert results[0].text == query_text diff --git a/tests/unit/ai4rag/core/experiment/test_experiment.py b/tests/unit/ai4rag/core/experiment/test_experiment.py index 21286736..83abec60 100644 --- a/tests/unit/ai4rag/core/experiment/test_experiment.py +++ b/tests/unit/ai4rag/core/experiment/test_experiment.py @@ -19,7 +19,7 @@ from ai4rag.evaluator.llmaj_evaluator import LLMaJEvaluator from ai4rag.evaluator.metric import Metrics, RAGMetric from ai4rag.evaluator.unitxt_evaluator import UnitxtEvaluator -from ai4rag.rag.vector_store.config import ChromaConfig +from ai4rag.rag.vector_store.config import MilvusLiteConfig # --------------------------------------------------------------------------- # Helpers @@ -74,7 +74,7 @@ def _build_experiment(evaluators=None, optimization_metric=Metrics.FAITHFULNESS, documents=[], benchmark_data=_BENCHMARK_DF, search_space=MagicMock(), - vector_store_config=ChromaConfig(), + vector_store_config=MilvusLiteConfig(db_path="./ai4rag.db"), optimizer_settings=MagicMock(), event_handler=MagicMock(), client=MagicMock(), diff --git a/tests/unit/ai4rag/core/experiment/test_mps.py b/tests/unit/ai4rag/core/experiment/test_mps.py index eca0a6dc..2e960af2 100644 --- a/tests/unit/ai4rag/core/experiment/test_mps.py +++ b/tests/unit/ai4rag/core/experiment/test_mps.py @@ -2,6 +2,8 @@ # Copyright IBM Corp. 2026 # SPDX-License-Identifier: Apache-2.0 # ----------------------------------------------------------------------------- +from contextlib import contextmanager + import pandas as pd import pytest from docling_core.types.doc import DoclingDocument @@ -11,8 +13,8 @@ AI4RAGChunk, BaseEmbeddingModel, BaseFoundationModel, + BaseVectorStore, BenchmarkData, - ChromaVectorStore, GenerationError, ModelsPreSelector, PreSelectorError, @@ -20,6 +22,21 @@ from ai4rag.evaluator.metric import Metrics +def _patch_temporary_store(mocker, store): + """Patch ``temporary_milvus_lite_store`` to yield *store* from a context manager. + + Mirrors the real helper's contract (a context manager wrapping a disposable + Milvus Lite store) without touching disk, so pre-selection can be exercised + against a fully mocked store. + """ + + @contextmanager + def _cm(*_args, **_kwargs): + yield store + + return mocker.patch("ai4rag.core.experiment.mps.temporary_milvus_lite_store", side_effect=_cm) + + @pytest.fixture def benchmark_data() -> BenchmarkData: benchmark_data = BenchmarkData( @@ -160,7 +177,7 @@ def _make_evaluate_metrics_result(evaluation_data, metrics): @pytest.fixture def fully_mocked_selector(mocker, documents, benchmark_data, embedding_models, foundation_models) -> ModelsPreSelector: - mocker.patch("ai4rag.core.experiment.mps.ChromaVectorStore", autospec=True) + _patch_temporary_store(mocker, mocker.MagicMock(spec=BaseVectorStore)) def side_effect(**kwargs): questions = kwargs.pop("questions") @@ -291,10 +308,9 @@ def test_evaluate_patterns_with_errors(self, mocker, fully_mocked_selector, capl assert expected_log in caplog.text def test_evaluate_pattern_with_failing_embedding(self, mocker, fully_mocked_selector, caplog): - vs = mocker.MagicMock(ChromaVectorStore) - val_err = ValueError("Fake error in embeddings") - vs.add_documents.side_effect = val_err - mocker.patch("ai4rag.core.experiment.mps.ChromaVectorStore", return_value=vs) + vs = mocker.MagicMock(spec=BaseVectorStore) + vs.add_documents.side_effect = ValueError("Fake error in embeddings") + _patch_temporary_store(mocker, vs) with pytest.raises(PreSelectorError) as err: fully_mocked_selector.evaluate_patterns() @@ -305,23 +321,18 @@ def test_evaluate_pattern_with_failing_embedding(self, mocker, fully_mocked_sele assert expected_msg in str(err.value) - def test_create_vector_store(self, mocker, fully_mocked_selector, caplog): - vs = mocker.MagicMock(ChromaVectorStore) + def test_index_documents_retries_then_propagates(self, mocker, fully_mocked_selector, caplog): + vs = mocker.MagicMock(spec=BaseVectorStore) document = mocker.MagicMock(AI4RAGChunk) val_err = ValueError("Fake embeddings error") vs.add_documents.side_effect = val_err - mocker.patch("ai4rag.core.experiment.mps.ChromaVectorStore", return_value=vs) - mocked_em = mocker.MagicMock(BaseEmbeddingModel) - mocked_em.model_id = "embedding_model_id" - with pytest.raises(PreSelectorError) as err: - fully_mocked_selector._create_vector_store( - embedding_model=mocked_em, chunked_documents=[document], collection_name="ai4rag_mps_collection_1" - ) + with pytest.raises(ValueError) as err: + fully_mocked_selector._index_documents(vs, [document]) - exp_msg = f"Failed to create in-memory vector index due to: {repr(val_err)}." - assert exp_msg in caplog.text, "Warning after first embedding fail was not logged" - assert str(err.value) == exp_msg + assert vs.add_documents.call_count == 2, "Indexing should be attempted twice (initial + one retry)." + assert "Failed to build the vector index due to:" in caplog.text, "Retry warning was not logged." + assert err.value is val_err def test_mean_based_scoring(self, pre_selector): top_models_with_scores = pre_selector._mean_based_scoring() diff --git a/tests/unit/ai4rag/rag/vector_store/test_chroma.py b/tests/unit/ai4rag/rag/vector_store/test_chroma.py deleted file mode 100644 index c26a0d83..00000000 --- a/tests/unit/ai4rag/rag/vector_store/test_chroma.py +++ /dev/null @@ -1,465 +0,0 @@ -# ----------------------------------------------------------------------------- -# Copyright IBM Corp. 2026 -# SPDX-License-Identifier: Apache-2.0 -# ----------------------------------------------------------------------------- -"""Unit tests for :class:`ChromaVectorStore` on the native ``chromadb`` client. - -These tests exercise the store end-to-end against a real in-memory -``EphemeralClient`` rather than mocking the client. ``chromadb`` keeps a single -process-wide in-memory system that is shared across every ``EphemeralClient`` -instance, so isolation is achieved by giving each store its own auto-generated -collection name and dropping the collection on teardown (see the ``store`` -fixture). -""" - -import hashlib -from unittest.mock import MagicMock - -import pytest - -from ai4rag.rag.chunking.chunk import AI4RAGChunk -from ai4rag.rag.embedding.base_model import BaseEmbeddingModel -from ai4rag.rag.vector_store.chroma import ChromaVectorStore -from ai4rag.rag.vector_store.config import ChromaConfig - - -class MockEmbeddingModel(BaseEmbeddingModel): - """Deterministic 3-D embedding model for reproducible search ordering. - - The vector is derived from the SHA-256 digest of the text, so identical - texts embed to identical vectors (cosine distance ``0`` → similarity ``1``) - while distinct texts get distinct, non-zero vectors. This makes nearest- - neighbour ordering deterministic without a real embedding backend. - """ - - def __init__(self) -> None: - super().__init__(client=MagicMock(), model_id="mock-embed", params={"embedding_dimension": 3}) - - @staticmethod - def _vector(text: str) -> list[float]: - digest = hashlib.sha256(text.encode()).digest() - # +1.0 keeps every component strictly positive so no vector is the zero - # vector (for which cosine distance is undefined). - return [1.0 + digest[0] / 255.0, 1.0 + digest[1] / 255.0, 1.0 + digest[2] / 255.0] - - def embed_documents(self, texts: list[str]) -> list[list[float]]: - return [self._vector(text) for text in texts] - - def embed_query(self, query: str) -> list[float]: - return self._vector(query) - - -@pytest.fixture -def embedding_model() -> MockEmbeddingModel: - """Provide a fresh deterministic embedding model.""" - return MockEmbeddingModel() - - -@pytest.fixture -def store(embedding_model): - """Provide an ephemeral store with an isolated, auto-cleaned collection.""" - vector_store = ChromaVectorStore(embedding_model=embedding_model) - yield vector_store - # Drop the collection so the shared in-memory system does not leak state - # between tests. - try: - vector_store.clean_collection() - except Exception: # pragma: no cover - teardown best effort - pass - - -def _doc_chunks() -> list[AI4RAGChunk]: - """Build five single-word chunks of one document with disjoint vocabularies. - - Disjoint words keep the merged window text predictable (no incidental - overlap de-duplication) while sequence numbers drive window expansion. - """ - words = ["alpha", "bravo", "charlie", "delta", "echo"] - return [ - AI4RAGChunk(text=word, metadata={"document_id": "docA", "sequence_number": i}) for i, word in enumerate(words) - ] - - -class TestChromaVectorStoreInitialization: - """Initialization and collection-name handling.""" - - def test_init_with_defaults(self, store): - assert store.collection_name.startswith("ai4rag_") - assert store.distance_metric == "cosine" - assert store.DOCUMENT_NAME_FIELD == "document_id" - assert store.SEQUENCE_NUMBER_FIELD == "sequence_number" - assert store.count() == 0 - - def test_init_with_custom_parameters(self, embedding_model): - vector_store = ChromaVectorStore( - embedding_model=embedding_model, - collection_name="ai4rag_custom_collection", - distance_metric="l2", - ) - try: - assert vector_store.collection_name == "ai4rag_custom_collection" - assert vector_store.distance_metric == "l2" - finally: - vector_store.clean_collection() - - def test_init_sanitizes_collection_name(self, embedding_model): - vector_store = ChromaVectorStore(embedding_model=embedding_model, collection_name="ai4rag-collection.v1") - try: - assert vector_store.collection_name == "ai4rag_collection_v1" - finally: - vector_store.clean_collection() - - -class TestChromaVectorStoreClientSelection: - """``_build_client`` selects the client implied by the config.""" - - def test_ephemeral_client_by_default(self, mocker): - ephemeral = mocker.patch("ai4rag.rag.vector_store.chroma.chromadb.EphemeralClient") - persistent = mocker.patch("ai4rag.rag.vector_store.chroma.chromadb.PersistentClient") - http = mocker.patch("ai4rag.rag.vector_store.chroma.chromadb.HttpClient") - - client = ChromaVectorStore._build_client(ChromaConfig()) - - ephemeral.assert_called_once_with() - persistent.assert_not_called() - http.assert_not_called() - assert client is ephemeral.return_value - - def test_persistent_client_when_persist_directory(self, mocker): - persistent = mocker.patch("ai4rag.rag.vector_store.chroma.chromadb.PersistentClient") - ephemeral = mocker.patch("ai4rag.rag.vector_store.chroma.chromadb.EphemeralClient") - - client = ChromaVectorStore._build_client(ChromaConfig(persist_directory="/data/chroma")) - - persistent.assert_called_once_with(path="/data/chroma") - ephemeral.assert_not_called() - assert client is persistent.return_value - - def test_http_client_when_host(self, mocker): - http = mocker.patch("ai4rag.rag.vector_store.chroma.chromadb.HttpClient") - persistent = mocker.patch("ai4rag.rag.vector_store.chroma.chromadb.PersistentClient") - - client = ChromaVectorStore._build_client(ChromaConfig(host="chroma.local", port=9000)) - - http.assert_called_once_with(host="chroma.local", port=9000) - persistent.assert_not_called() - assert client is http.return_value - - def test_host_takes_precedence_over_persist_directory(self, mocker): - http = mocker.patch("ai4rag.rag.vector_store.chroma.chromadb.HttpClient") - persistent = mocker.patch("ai4rag.rag.vector_store.chroma.chromadb.PersistentClient") - - ChromaVectorStore._build_client(ChromaConfig(host="h", persist_directory="/data")) - - http.assert_called_once() - persistent.assert_not_called() - - -class TestChromaVectorStoreDistanceMetric: - """``distance_metric`` property and validation.""" - - def test_distance_metric_getter(self, embedding_model): - vector_store = ChromaVectorStore(embedding_model=embedding_model, distance_metric="l2") - try: - assert vector_store.distance_metric == "l2" - finally: - vector_store.clean_collection() - - def test_distance_metric_setter_valid(self, store): - store.distance_metric = "l2" - assert store.distance_metric == "l2" - - def test_distance_metric_setter_invalid(self, store): - with pytest.raises(ValueError) as exc_info: - store.distance_metric = "invalid_metric" - assert "Invalid distance metric" in str(exc_info.value) - - -class TestChromaVectorStoreDistanceToSimilarity: - """Score mapping keeps the 'higher = more relevant' convention.""" - - def test_cosine_mapping(self, store): - assert store.distance_metric == "cosine" - assert store._distance_to_similarity(0.0) == pytest.approx(1.0) - assert store._distance_to_similarity(0.2) == pytest.approx(0.8) - assert store._distance_to_similarity(2.0) == pytest.approx(-1.0) - - def test_l2_mapping(self, embedding_model): - vector_store = ChromaVectorStore(embedding_model=embedding_model, distance_metric="l2") - try: - assert vector_store._distance_to_similarity(0.0) == pytest.approx(1.0) - assert vector_store._distance_to_similarity(3.0) == pytest.approx(0.25) - finally: - vector_store.clean_collection() - - -class TestChromaVectorStoreAddDocuments: - """``add_documents`` embedding, dedup, and batching.""" - - def test_add_documents_basic(self, store): - chunks = [AI4RAGChunk(text="alpha"), AI4RAGChunk(text="bravo")] - ids = store.add_documents(chunks) - assert len(ids) == 2 - assert ids == [chunk.chunk_id for chunk in chunks] - assert store.count() == 2 - - def test_add_documents_with_metadata(self, store): - store.add_documents([AI4RAGChunk(text="alpha", metadata={"source": "a"})]) - results = store.search("alpha", k=1) - assert results[0].metadata == {"source": "a"} - - def test_add_documents_empty_metadata_roundtrips_as_empty_dict(self, store): - # Chroma rejects empty-dict metadata; the store maps it to None on write - # and it comes back as {} on read. - store.add_documents([AI4RAGChunk(text="lonely")]) - results = store.search("lonely", k=1) - assert results[0].text == "lonely" - assert results[0].metadata == {} - - def test_add_documents_deduplicates_by_chunk_id(self, store): - chunks = [AI4RAGChunk(text="dup"), AI4RAGChunk(text="dup"), AI4RAGChunk(text="unique")] - ids = store.add_documents(chunks) - assert len(ids) == 2 - assert store.count() == 2 - - def test_add_documents_empty_list(self, store): - assert store.add_documents([]) == [] - assert store.count() == 0 - - def test_add_documents_batches_by_max_batch_size(self, store, mocker): - spy = mocker.patch.object(store._collection, "upsert", wraps=store._collection.upsert) - chunks = [AI4RAGChunk(text=f"word{i}") for i in range(5)] - ids = store.add_documents(chunks, max_batch_size=2) - # 5 chunks in batches of 2 -> 3 upsert calls. - assert spy.call_count == 3 - assert len(ids) == 5 - assert store.count() == 5 - - -class TestChromaVectorStoreSearch: - """``search`` returns chunks and 'higher = better' scores.""" - - def test_search_basic(self, store): - store.add_documents([AI4RAGChunk(text="alpha"), AI4RAGChunk(text="bravo")]) - results = store.search("alpha", k=5) - assert all(isinstance(chunk, AI4RAGChunk) for chunk in results) - assert results[0].text == "alpha" - - def test_search_respects_k(self, store): - store.add_documents([AI4RAGChunk(text=w) for w in ("alpha", "bravo", "charlie")]) - assert len(store.search("alpha", k=2)) == 2 - - def test_search_with_scores_higher_is_more_relevant(self, store): - store.add_documents([AI4RAGChunk(text="alpha"), AI4RAGChunk(text="zulu different")]) - results = store.search("alpha", k=2, include_scores=True) - assert isinstance(results[0], tuple) - chunk, score = results[0] - # Exact match ranks first with the maximal cosine similarity (~1.0)... - assert chunk.text == "alpha" - assert score == pytest.approx(1.0, abs=1e-4) - # ...and outranks the dissimilar document. - assert results[0][1] > results[1][1] - - def test_search_with_where_filter(self, store): - store.add_documents( - [ - AI4RAGChunk(text="cat doc", metadata={"category": "animal"}), - AI4RAGChunk(text="car doc", metadata={"category": "vehicle"}), - ] - ) - results = store.search("cat doc", k=5, where={"category": {"$eq": "animal"}}) - assert len(results) == 1 - assert results[0].metadata["category"] == "animal" - - def test_search_with_filter_alias(self, store): - store.add_documents( - [ - AI4RAGChunk(text="cat doc", metadata={"category": "animal"}), - AI4RAGChunk(text="car doc", metadata={"category": "vehicle"}), - ] - ) - results = store.search("car doc", k=5, filter={"category": {"$eq": "vehicle"}}) - assert len(results) == 1 - assert results[0].metadata["category"] == "vehicle" - - def test_search_empty_collection(self, store): - assert store.search("anything", k=5) == [] - - -class TestChromaVectorStoreWindowSearch: - """``window_search`` expands each hit with its neighbouring chunks.""" - - def test_zero_window_size_returns_search_results(self, store): - store.add_documents(_doc_chunks()) - results = store.window_search("charlie", k=1, window_size=0) - assert len(results) == 1 - assert results[0].text == "charlie" - - def test_negative_window_size_returns_search_results(self, store): - store.add_documents(_doc_chunks()) - results = store.window_search("charlie", k=1, window_size=-1) - assert results[0].text == "charlie" - - def test_window_search_without_scores_merges_neighbours(self, store): - store.add_documents(_doc_chunks()) - results = store.window_search("charlie", k=1, window_size=1) - assert isinstance(results[0], AI4RAGChunk) - # seq 1 (bravo), 2 (charlie), 3 (delta) merged in sequence order. - assert results[0].text == "bravo charlie delta" - - def test_window_search_with_scores_merges_and_keeps_score(self, store): - store.add_documents(_doc_chunks()) - results = store.window_search("charlie", k=1, window_size=1, include_scores=True) - chunk, score = results[0] - assert isinstance(chunk, AI4RAGChunk) - assert chunk.text == "bravo charlie delta" - assert score == pytest.approx(1.0, abs=1e-4) - - def test_window_clamped_at_document_edges(self, store): - store.add_documents(_doc_chunks()) - # Centered on the first chunk: only seq 0 and 1 exist to the right. - results = store.window_search("alpha", k=1, window_size=1) - assert results[0].text == "alpha bravo" - - -class TestChromaVectorStoreWindowExtendAndMerge: - """``_window_extend_and_merge`` validation and merging.""" - - def test_missing_document_id_raises(self, store): - chunk = AI4RAGChunk(text="x", metadata={"sequence_number": 1}) - with pytest.raises(ValueError, match="document_id"): - store._window_extend_and_merge(chunk, window_size=2) - - def test_missing_sequence_number_raises(self, store): - chunk = AI4RAGChunk(text="x", metadata={"document_id": "docA"}) - with pytest.raises(ValueError, match="sequence_number"): - store._window_extend_and_merge(chunk, window_size=2) - - def test_basic_merge(self, store): - store.add_documents(_doc_chunks()) - center = AI4RAGChunk(text="charlie", metadata={"document_id": "docA", "sequence_number": 2}) - merged = store._window_extend_and_merge(center, window_size=1) - assert isinstance(merged, AI4RAGChunk) - assert merged.text == "bravo charlie delta" - - -class TestChromaVectorStoreGetWindowDocuments: - """``_get_window_documents`` fetches a contiguous slice of one document.""" - - def test_returns_chunks_within_range(self, store): - store.add_documents(_doc_chunks()) - results = store._get_window_documents("docA", [1, 2, 3]) - assert all(isinstance(chunk, AI4RAGChunk) for chunk in results) - assert sorted(chunk.metadata["sequence_number"] for chunk in results) == [1, 2, 3] - - def test_filters_by_document_id(self, store): - chunks = _doc_chunks() - chunks.append(AI4RAGChunk(text="foreign", metadata={"document_id": "docB", "sequence_number": 2})) - store.add_documents(chunks) - results = store._get_window_documents("docA", [0, 1, 2]) - texts = {chunk.text for chunk in results} - assert "foreign" not in texts - assert all(chunk.metadata["document_id"] == "docA" for chunk in results) - - -class TestChromaVectorStoreLifecycle: - """``count``, ``clear``, ``delete``, and ``clean_collection``.""" - - def test_count_reflects_contents(self, store): - assert store.count() == 0 - store.add_documents([AI4RAGChunk(text="alpha"), AI4RAGChunk(text="bravo")]) - assert store.count() == 2 - - def test_clear_removes_all_entries(self, store): - store.add_documents([AI4RAGChunk(text="alpha"), AI4RAGChunk(text="bravo")]) - store.clear() - assert store.count() == 0 - - def test_clear_on_empty_is_noop(self, store): - store.clear() - assert store.count() == 0 - - def test_delete_by_ids(self, store): - chunks = [AI4RAGChunk(text="alpha"), AI4RAGChunk(text="bravo")] - ids = store.add_documents(chunks) - store.delete([ids[0]]) - assert store.count() == 1 - remaining = store.search("bravo", k=5) - assert remaining[0].text == "bravo" - - def test_clean_collection_drops_collection(self, embedding_model): - vector_store = ChromaVectorStore(embedding_model=embedding_model) - vector_store.add_documents([AI4RAGChunk(text="alpha")]) - name = vector_store.collection_name - vector_store.clean_collection() - existing = {collection.name for collection in vector_store._client.list_collections()} - assert name not in existing - - -class TestChromaVectorStoreClose: - """``close()`` releases OS resources without destroying shared ephemeral data.""" - - def test_ephemeral_data_survives_close_and_reopen(self, embedding_model): - """Regression: closing an ephemeral store must not wipe its shared in-memory data. - - chromadb backs every ``EphemeralClient`` with one process-wide, - reference-counted in-memory ``System``. A ``close()`` that decremented that - count to zero would stop the ``System`` and discard all collections — wiping - data a later store still depends on, exactly as happens across HPO trials - that reuse a collection by name (see ``ChromaVectorStore.close``). This - asserts a closed-then-reopened ephemeral store still sees its documents. - """ - collection_name = "ai4rag_reuse_regression" - with ChromaVectorStore(embedding_model=embedding_model, collection_name=collection_name) as store: - store.add_documents([AI4RAGChunk(text="alpha"), AI4RAGChunk(text="bravo")]) - assert store.count() == 2 - - # A fresh store over the same collection — as a subsequent HPO trial would - # open — must still find the data the first (now-closed) store wrote. - reopened = ChromaVectorStore(embedding_model=embedding_model, collection_name=collection_name) - try: - assert reopened.count() == 2 - assert reopened.search("alpha", k=1)[0].text == "alpha" - finally: - reopened.clean_collection() - - def test_close_is_noop_for_ephemeral_client(self, embedding_model, mocker): - """An ephemeral client holds no OS resource; close() must not tear it down. - - Calling the underlying ``client.close()`` would decrement the shared - ``System``'s refcount and risk destroying in-memory data other stores use, - so the store must skip it entirely for the ephemeral mode. - """ - store = ChromaVectorStore(embedding_model=embedding_model, collection_name="ai4rag_noop_close") - spy = mocker.spy(store._client, "close") - try: - store.close() - spy.assert_not_called() - finally: - store.clean_collection() - - def test_close_releases_persistent_client(self, embedding_model, mocker): - """A persistent client holds a SQLite file lock; close() must release it.""" - persistent = mocker.patch("ai4rag.rag.vector_store.chroma.chromadb.PersistentClient") - store = ChromaVectorStore( - embedding_model=embedding_model, - config=ChromaConfig(persist_directory="/tmp/ai4rag-chroma-persist"), - collection_name="ai4rag_persist_close", - ) - - store.close() - - persistent.return_value.close.assert_called_once() - - def test_close_releases_http_client(self, embedding_model, mocker): - """An HTTP client holds client-side sockets; close() must release them.""" - http = mocker.patch("ai4rag.rag.vector_store.chroma.chromadb.HttpClient") - store = ChromaVectorStore( - embedding_model=embedding_model, - config=ChromaConfig(host="chroma.local", port=9000), - collection_name="ai4rag_http_close", - ) - - store.close() - - http.return_value.close.assert_called_once() diff --git a/tests/unit/ai4rag/rag/vector_store/test_config.py b/tests/unit/ai4rag/rag/vector_store/test_config.py index d9551e23..9ec752c1 100644 --- a/tests/unit/ai4rag/rag/vector_store/test_config.py +++ b/tests/unit/ai4rag/rag/vector_store/test_config.py @@ -2,59 +2,20 @@ # Copyright IBM Corp. 2026 # SPDX-License-Identifier: Apache-2.0 # ----------------------------------------------------------------------------- -import os from dataclasses import FrozenInstanceError import pytest from ai4rag.rag.vector_store.config import ( - ChromaConfig, + DEFAULT_MILVUS_LITE_DB_PATH, MilvusConfig, + MilvusLiteConfig, PGVectorConfig, get_vector_store_config, get_vector_store_env_vars, ) -class TestChromaConfig: - """Tests for ChromaConfig dataclass.""" - - def test_defaults_are_ephemeral(self): - cfg = ChromaConfig() - assert cfg.persist_directory is None - assert cfg.host is None - assert cfg.port == 8000 - assert cfg.provider == "chroma" - - def test_custom_values(self): - cfg = ChromaConfig(persist_directory="/data/chroma", host="chroma.local", port=9000) - assert cfg.persist_directory == "/data/chroma" - assert cfg.host == "chroma.local" - assert cfg.port == 9000 - - def test_frozen(self): - cfg = ChromaConfig() - with pytest.raises(FrozenInstanceError): - cfg.host = "other" - - def test_from_env_defaults(self, monkeypatch): - for var in ("CHROMA_PERSIST_DIR", "CHROMA_HOST", "CHROMA_PORT"): - monkeypatch.delenv(var, raising=False) - cfg = ChromaConfig.from_env() - assert cfg.persist_directory is None - assert cfg.host is None - assert cfg.port == 8000 - - def test_from_env_custom(self, monkeypatch): - monkeypatch.setenv("CHROMA_PERSIST_DIR", "/tmp/chroma") - monkeypatch.setenv("CHROMA_HOST", "chroma-server") - monkeypatch.setenv("CHROMA_PORT", "9001") - cfg = ChromaConfig.from_env() - assert cfg.persist_directory == "/tmp/chroma" - assert cfg.host == "chroma-server" - assert cfg.port == 9001 - - class TestMilvusConfig: """Tests for MilvusConfig dataclass.""" @@ -64,6 +25,16 @@ def test_constructor_uri_only(self): assert cfg.token is None assert cfg.server_cert is None + @pytest.mark.parametrize("bad_uri", ["./ai4rag.db", "ai4rag.db", "/tmp/x.db", "localhost:19530", "milvus", ""]) + def test_non_url_uri_is_rejected(self, bad_uri): + """A uri that is not an http(s) server URL must raise, not silently become Milvus Lite. + + This is the guard against a mistyped MILVUS_URI in production spinning up a + throwaway local database instead of connecting to the intended server. + """ + with pytest.raises(ValueError, match="must be a Milvus server URL"): + MilvusConfig(uri=bad_uri) + def test_constructor_uri_and_token(self): cfg = MilvusConfig(uri="https://milvus:19530", token="root:Milvus") assert cfg.uri == "https://milvus:19530" @@ -108,6 +79,40 @@ def test_from_env_missing_uri_raises(self, monkeypatch): MilvusConfig.from_env() +class TestMilvusLiteConfig: + """Tests for the MilvusLiteConfig dataclass (embedded, local database).""" + + def test_defaults(self): + cfg = MilvusLiteConfig() + assert cfg.db_path == DEFAULT_MILVUS_LITE_DB_PATH + assert cfg.provider == "milvus_lite" + + def test_custom_db_path(self): + cfg = MilvusLiteConfig(db_path="/data/store.db") + assert cfg.db_path == "/data/store.db" + + @pytest.mark.parametrize("bad_path", ["http://host:19530", "https://host:19530"]) + def test_server_url_db_path_is_rejected(self, bad_path): + """A server URL is not a local database file; it belongs in MilvusConfig.""" + with pytest.raises(ValueError, match="must be a local filesystem path"): + MilvusLiteConfig(db_path=bad_path) + + def test_frozen(self): + cfg = MilvusLiteConfig() + with pytest.raises(FrozenInstanceError): + cfg.db_path = "other.db" + + def test_from_env_default(self, monkeypatch): + monkeypatch.delenv("MILVUS_LITE_DB_PATH", raising=False) + cfg = MilvusLiteConfig.from_env() + assert cfg.db_path == DEFAULT_MILVUS_LITE_DB_PATH + + def test_from_env_custom(self, monkeypatch): + monkeypatch.setenv("MILVUS_LITE_DB_PATH", "/tmp/custom.db") + cfg = MilvusLiteConfig.from_env() + assert cfg.db_path == "/tmp/custom.db" + + class TestPGVectorConfig: """Tests for PGVectorConfig dataclass.""" @@ -165,13 +170,6 @@ def test_from_env_custom(self, monkeypatch): class TestGetVectorStoreConfig: """Tests for the ``get_vector_store_config`` provider factory.""" - def test_returns_chroma_config(self, monkeypatch): - for var in ("CHROMA_PERSIST_DIR", "CHROMA_HOST", "CHROMA_PORT"): - monkeypatch.delenv(var, raising=False) - cfg = get_vector_store_config("chroma") - assert isinstance(cfg, ChromaConfig) - assert cfg.provider == "chroma" - def test_returns_pgvector_config(self, monkeypatch): for var in ("PGVECTOR_HOST", "PGVECTOR_PORT", "PGVECTOR_DB", "PGVECTOR_USER", "PGVECTOR_PASSWORD"): monkeypatch.delenv(var, raising=False) @@ -187,6 +185,12 @@ def test_returns_milvus_config_from_env(self, monkeypatch): assert isinstance(cfg, MilvusConfig) assert cfg.uri == "http://host:19530" + def test_returns_milvus_lite_config_from_env(self, monkeypatch): + monkeypatch.setenv("MILVUS_LITE_DB_PATH", "/tmp/lite.db") + cfg = get_vector_store_config("milvus_lite") + assert isinstance(cfg, MilvusLiteConfig) + assert cfg.db_path == "/tmp/lite.db" + def test_milvus_missing_uri_raises_key_error(self, monkeypatch): """The factory must surface the backend's own ``from_env`` failure.""" monkeypatch.delenv("MILVUS_URI", raising=False) @@ -203,7 +207,7 @@ class TestGetVectorStoreEnvVars: @pytest.mark.parametrize( ("provider", "config_cls"), - [("chroma", ChromaConfig), ("milvus", MilvusConfig), ("pgvector", PGVectorConfig)], + [("milvus", MilvusConfig), ("milvus_lite", MilvusLiteConfig), ("pgvector", PGVectorConfig)], ) def test_matches_config_class_env_vars(self, provider, config_cls): """The helper must return the exact ``env_vars`` tuple declared on the config class.""" diff --git a/tests/unit/ai4rag/rag/vector_store/test_get_vector_store.py b/tests/unit/ai4rag/rag/vector_store/test_get_vector_store.py index 1c023418..e7813810 100644 --- a/tests/unit/ai4rag/rag/vector_store/test_get_vector_store.py +++ b/tests/unit/ai4rag/rag/vector_store/test_get_vector_store.py @@ -8,8 +8,7 @@ import pytest from ai4rag.rag.embedding.base_model import BaseEmbeddingModel -from ai4rag.rag.vector_store.chroma import ChromaVectorStore -from ai4rag.rag.vector_store.config import ChromaConfig, MilvusConfig, PGVectorConfig +from ai4rag.rag.vector_store.config import MilvusConfig, MilvusLiteConfig, PGVectorConfig from ai4rag.rag.vector_store.get_vector_store import get_vector_store @@ -41,68 +40,60 @@ def mock_embedding_model(): return MockEmbeddingModel() -class TestGetVectorStoreChroma: - """Test suite for get_vector_store with Chroma provider.""" - - def test_get_vector_store_chroma_default(self, mock_embedding_model): - """Test getting Chroma vector store with default parameters.""" - - vector_store = get_vector_store( - embedding_model=mock_embedding_model, - config=ChromaConfig(), - ) - - assert isinstance(vector_store, ChromaVectorStore) - assert vector_store.embedding_model == mock_embedding_model +class TestGetVectorStoreMilvus: + """Test suite for get_vector_store with Milvus provider.""" - def test_get_vector_store_chroma_with_collection_name(self, mock_embedding_model): - """Test getting Chroma vector store with custom collection name.""" + @patch("ai4rag.rag.vector_store.milvus.MilvusClient") + def test_milvus_returns_vector_store(self, MockClient, mock_embedding_model): + MockClient.return_value.has_collection.return_value = False + config = MilvusConfig(uri="http://localhost:19530") vector_store = get_vector_store( embedding_model=mock_embedding_model, - config=ChromaConfig(), - collection_name="ai4rag_my_collection", + config=config, ) - assert isinstance(vector_store, ChromaVectorStore) - assert vector_store.collection_name == "ai4rag_my_collection" - - def test_get_vector_store_chroma_with_none_collection_name(self, mock_embedding_model): - """Test Chroma with None collection name uses an auto-generated default.""" + from ai4rag.rag.vector_store.milvus import MilvusVectorStore - vector_store = get_vector_store( - embedding_model=mock_embedding_model, - config=ChromaConfig(), - collection_name=None, - ) + assert isinstance(vector_store, MilvusVectorStore) - assert isinstance(vector_store, ChromaVectorStore) - assert vector_store.collection_name is not None + def test_milvus_with_wrong_config_type_raises_type_error(self, mock_embedding_model): + """A config whose provider claims 'milvus' but isn't a MilvusConfig must raise TypeError.""" + with pytest.raises(TypeError, match="MilvusConfig is required"): + get_vector_store( + embedding_model=mock_embedding_model, + config=PGVectorConfig(provider="milvus"), + ) -class TestGetVectorStoreMilvus: - """Test suite for get_vector_store with Milvus provider.""" +class TestGetVectorStoreMilvusLite: + """Test suite for get_vector_store with the Milvus Lite provider.""" @patch("ai4rag.rag.vector_store.milvus.MilvusClient") - def test_milvus_returns_vector_store(self, MockClient, mock_embedding_model): + def test_milvus_lite_returns_vector_store(self, MockClient, mock_embedding_model): + """A MilvusLiteConfig selects the embedded engine via the shared MilvusVectorStore.""" MockClient.return_value.has_collection.return_value = False - config = MilvusConfig(uri="http://localhost:19530") vector_store = get_vector_store( embedding_model=mock_embedding_model, - config=config, + config=MilvusLiteConfig(db_path="./ai4rag.db"), + collection_name="ai4rag_my_collection", ) from ai4rag.rag.vector_store.milvus import MilvusVectorStore assert isinstance(vector_store, MilvusVectorStore) + assert vector_store.collection_name == "ai4rag_my_collection" + # The embedded engine connects with the local db_path as its uri, and no + # auth/TLS kwargs. + MockClient.assert_called_once_with(uri="./ai4rag.db") - def test_milvus_with_wrong_config_type_raises_type_error(self, mock_embedding_model): - """A config whose provider claims 'milvus' but isn't a MilvusConfig must raise TypeError.""" - with pytest.raises(TypeError, match="MilvusConfig is required"): + def test_milvus_lite_with_wrong_config_type_raises_type_error(self, mock_embedding_model): + """A config whose provider claims 'milvus_lite' but isn't a MilvusLiteConfig must raise TypeError.""" + with pytest.raises(TypeError, match="MilvusLiteConfig is required"): get_vector_store( embedding_model=mock_embedding_model, - config=PGVectorConfig(provider="milvus"), + config=PGVectorConfig(provider="milvus_lite"), ) @@ -165,21 +156,25 @@ def test_get_vector_store_old_ogx_provider_no_longer_supported(self, mock_embedd class TestGetVectorStoreEdgeCases: """Test suite for edge cases in get_vector_store.""" - def test_get_vector_store_case_sensitive(self, mock_embedding_model): + @patch("ai4rag.rag.vector_store.milvus.MilvusClient") + def test_get_vector_store_case_sensitive(self, MockClient, mock_embedding_model): """Test that config.provider is case-sensitive.""" + MockClient.return_value.has_collection.return_value = False - # "chroma" should work + # "milvus" should work vector_store = get_vector_store( embedding_model=mock_embedding_model, - config=ChromaConfig(), + config=MilvusConfig(uri="http://localhost:19530"), ) - assert isinstance(vector_store, ChromaVectorStore) + from ai4rag.rag.vector_store.milvus import MilvusVectorStore + + assert isinstance(vector_store, MilvusVectorStore) - # "CHROMA" should not work + # "MILVUS" should not work with pytest.raises(ValueError): get_vector_store( embedding_model=mock_embedding_model, - config=_fake_config("CHROMA"), + config=_fake_config("MILVUS"), ) def test_get_vector_store_whitespace_provider(self, mock_embedding_model): @@ -187,12 +182,12 @@ def test_get_vector_store_whitespace_provider(self, mock_embedding_model): with pytest.raises(ValueError): get_vector_store( embedding_model=mock_embedding_model, - config=_fake_config(" chroma "), + config=_fake_config(" milvus "), ) def test_get_vector_store_similar_provider_names(self, mock_embedding_model): """Test that similar but incorrect provider names raise errors.""" - invalid_providers = ["chromadb", "chroma_db", "ogx_milvus"] + invalid_providers = ["chromadb", "milvuslite", "ogx_milvus"] for invalid_provider in invalid_providers: with pytest.raises(ValueError): @@ -205,12 +200,14 @@ def test_get_vector_store_similar_provider_names(self, mock_embedding_model): class TestGetVectorStoreReturnTypes: """Test suite for verifying return types from get_vector_store.""" - def test_chroma_returns_base_vector_store_interface(self, mock_embedding_model): - """Test that Chroma vector store implements BaseVectorStore interface.""" + @patch("ai4rag.rag.vector_store.milvus.MilvusClient") + def test_milvus_returns_base_vector_store_interface(self, MockClient, mock_embedding_model): + """Test that the Milvus vector store implements the BaseVectorStore interface.""" + MockClient.return_value.has_collection.return_value = False vector_store = get_vector_store( embedding_model=mock_embedding_model, - config=ChromaConfig(), + config=MilvusConfig(uri="http://localhost:19530"), ) assert hasattr(vector_store, "search") diff --git a/tests/unit/ai4rag/rag/vector_store/test_milvus_lite.py b/tests/unit/ai4rag/rag/vector_store/test_milvus_lite.py new file mode 100644 index 00000000..e66af474 --- /dev/null +++ b/tests/unit/ai4rag/rag/vector_store/test_milvus_lite.py @@ -0,0 +1,186 @@ +# ----------------------------------------------------------------------------- +# Copyright IBM Corp. 2026 +# SPDX-License-Identifier: Apache-2.0 +# ----------------------------------------------------------------------------- +"""Unit tests for the Milvus store running against a real embedded Milvus Lite. + +These tests exercise :class:`MilvusVectorStore` end-to-end against a real, local +Milvus Lite database (a temporary ``.db`` file) rather than mocking the client. +Milvus Lite ships with ``pymilvus[milvus-lite]`` and needs no server, so this +suite runs in the standard unit tier and is the primary guard that the store's +BM25 hybrid schema is actually supported by the installed Milvus Lite version. + +Isolation is achieved by giving each store its own temporary database directory +(and an auto-generated collection name), removed on teardown. +""" + +import hashlib +import shutil +import tempfile +from pathlib import Path + +import pytest + +from ai4rag.rag.chunking.chunk import AI4RAGChunk +from ai4rag.rag.embedding.base_model import BaseEmbeddingModel +from ai4rag.rag.vector_store.config import MilvusLiteConfig +from ai4rag.rag.vector_store.local_store import temporary_milvus_lite_store +from ai4rag.rag.vector_store.milvus import MilvusVectorStore + +_EMBEDDING_DIMENSION = 16 + + +class DeterministicEmbeddingModel(BaseEmbeddingModel): + """Hash-based embedding model: identical text always yields the same vector. + + Each text maps to a fixed-length vector derived from its SHA-256 digest, so a + query embedded from the exact text of a stored chunk lands on that chunk + (cosine distance ~0) without contacting any embedding service. Components stay + strictly positive to avoid a zero vector. + """ + + def __init__(self) -> None: + super().__init__(client=None, model_id="deterministic", params={"embedding_dimension": _EMBEDDING_DIMENSION}) + + @staticmethod + def _vector(text: str) -> list[float]: + out: list[float] = [] + counter = 0 + while len(out) < _EMBEDDING_DIMENSION: + digest = hashlib.sha256(f"{text}:{counter}".encode()).digest() + out.extend(1.0 + byte / 255.0 for byte in digest) + counter += 1 + return out[:_EMBEDDING_DIMENSION] + + def embed_documents(self, texts: list[str]) -> list[list[float]]: + return [self._vector(text) for text in texts] + + def embed_query(self, query: str) -> list[float]: + return self._vector(query) + + +@pytest.fixture(scope="module") +def embedding_model() -> DeterministicEmbeddingModel: + return DeterministicEmbeddingModel() + + +@pytest.fixture +def sample_chunks() -> list[AI4RAGChunk]: + texts = [ + "The quick brown fox jumps over the lazy dog.", + "Milvus Lite is an embedded vector database.", + "Retrieval augmented generation improves grounded answers.", + "A fox is a small wild animal related to dogs.", + ] + return [ + AI4RAGChunk(text=text, metadata={"document_id": "doc", "sequence_number": i}) for i, text in enumerate(texts) + ] + + +@pytest.fixture +def store(embedding_model): + """A Milvus Lite store on a private temporary database, dropped on teardown.""" + tmp_dir = tempfile.mkdtemp(prefix="ai4rag-milvus-lite-test-") + vector_store = MilvusVectorStore( + embedding_model=embedding_model, config=MilvusLiteConfig(db_path=str(Path(tmp_dir) / "s.db")) + ) + try: + yield vector_store + finally: + vector_store.clean_collection() + vector_store.close() + shutil.rmtree(tmp_dir, ignore_errors=True) + + +class TestMilvusLiteCollection: + def test_collection_is_created_with_bm25_schema(self, store): + """Constructing the store creates its collection, including the BM25 sparse field.""" + assert store._client.has_collection(store.collection_name) + # The BM25 hybrid schema requires a sparse field; its presence proves + # Milvus Lite accepted the BM25 Function-backed schema. + field_names = {field["name"] for field in store._client.describe_collection(store.collection_name)["fields"]} + assert {"chunk_id", "content", "vector", "metadata", "sparse"}.issubset(field_names) + + +class TestMilvusLiteAddAndSearch: + def test_add_documents_then_vector_search(self, store, sample_chunks): + store.add_documents(sample_chunks) + + results = store.search(sample_chunks[1].text, k=1) + + assert len(results) == 1 + assert results[0].text == sample_chunks[1].text + + def test_search_include_scores_returns_float_pairs(self, store, sample_chunks): + store.add_documents(sample_chunks) + + results = store.search(sample_chunks[0].text, k=2, include_scores=True) + + assert len(results) == 2 + for chunk, score in results: + assert isinstance(chunk, AI4RAGChunk) + assert isinstance(score, float) + + def test_add_documents_deduplicates_by_chunk_id(self, store, sample_chunks): + # Re-adding the same chunks (identical chunk_id) must not create duplicates. + store.add_documents(sample_chunks) + store.add_documents(sample_chunks) + + results = store.search(sample_chunks[0].text, k=10) + assert len(results) == len(sample_chunks) + + def test_add_empty_documents_is_noop(self, store): + store.add_documents([]) + assert store.search("anything", k=5) == [] + + +class TestMilvusLiteHybridSearch: + """Milvus Lite must support dense + BM25 hybrid search with server-side fusion.""" + + def test_hybrid_search_rrf(self, store, sample_chunks): + store.add_documents(sample_chunks) + + results = store.search("fox", k=2, search_mode="hybrid", ranker_strategy="rrf") + + assert results + assert any("fox" in chunk.text for chunk in results) + + def test_hybrid_search_weighted(self, store, sample_chunks): + store.add_documents(sample_chunks) + + results = store.search( + "vector database", k=2, search_mode="hybrid", ranker_strategy="weighted", ranker_alpha=0.5 + ) + + assert results + + +class TestMilvusLiteLifecycle: + def test_clean_collection_drops_collection(self, embedding_model): + tmp_dir = tempfile.mkdtemp(prefix="ai4rag-milvus-lite-test-") + try: + vector_store = MilvusVectorStore( + embedding_model=embedding_model, config=MilvusLiteConfig(db_path=str(Path(tmp_dir) / "s.db")) + ) + name = vector_store.collection_name + assert vector_store._client.has_collection(name) + + vector_store.clean_collection() + + assert not vector_store._client.has_collection(name) + vector_store.close() + finally: + shutil.rmtree(tmp_dir, ignore_errors=True) + + def test_temporary_milvus_lite_store_indexes_and_cleans_up(self, embedding_model, sample_chunks): + captured_dir = {} + + with temporary_milvus_lite_store(embedding_model) as vector_store: + vector_store.add_documents(sample_chunks) + results = vector_store.search(sample_chunks[0].text, k=1) + assert results + assert results[0].text == sample_chunks[0].text + # Record the on-disk database directory so we can assert it is removed. + captured_dir["path"] = Path(vector_store._config.db_path).parent + + assert not captured_dir["path"].exists(), "temporary Milvus Lite directory must be removed on exit" diff --git a/tests/unit/ai4rag/search_space/prepare/test_prepare_search_space.py b/tests/unit/ai4rag/search_space/prepare/test_prepare_search_space.py index 78fdfcb9..4ad3fa40 100644 --- a/tests/unit/ai4rag/search_space/prepare/test_prepare_search_space.py +++ b/tests/unit/ai4rag/search_space/prepare/test_prepare_search_space.py @@ -130,23 +130,8 @@ def test_missing_embedding_models_raises_error(self): with pytest.raises(SearchSpaceValueError, match="Provide both 'foundation_models' and 'embedding_models'"): prepare_search_space_with_maas(_payload(embedding_ids=None), client) - def test_chroma_vector_store_excludes_hybrid_params(self, mocker): - """The chroma vector store type excludes hybrid search parameters.""" - client = _setup_client(mocker, ["default-llm", "default-embedding"]) - - result = prepare_search_space_with_maas(_payload(), client, vector_store_type="chroma") - - param_names = [p.name for p in result.params] - assert "search_mode" in param_names - assert "ranker_strategy" not in param_names - assert "ranker_k" not in param_names - assert "ranker_alpha" not in param_names - - search_mode_param = result["search_mode"] - assert search_mode_param.values == ("vector",) - - def test_non_chroma_vector_store_includes_hybrid_params_by_default(self, mocker): - """Non-chroma vector stores (default milvus) include hybrid search parameters.""" + def test_search_space_includes_hybrid_params_by_default(self, mocker): + """All supported vector store backends support hybrid search, so it is always included.""" client = _setup_client(mocker, ["default-llm", "default-embedding"]) result = prepare_search_space_with_maas(_payload(), client) diff --git a/tests/unit/ai4rag/search_space/src/test_search_space.py b/tests/unit/ai4rag/search_space/src/test_search_space.py index 56157b60..1a5b5acb 100644 --- a/tests/unit/ai4rag/search_space/src/test_search_space.py +++ b/tests/unit/ai4rag/search_space/src/test_search_space.py @@ -301,8 +301,8 @@ def _custom_rule(combination: dict) -> bool: class TestGetDefaultSearchSpaceParameters: - def test_milvus_includes_hybrid_params_by_default(self): - params = get_default_ai4rag_search_space_parameters(vector_store_type="milvus") + def test_includes_hybrid_params_by_default(self): + params = get_default_ai4rag_search_space_parameters() param_names = {p.name for p in params} assert "search_mode" in param_names @@ -314,25 +314,7 @@ def test_milvus_includes_hybrid_params_by_default(self): assert "vector" in search_mode_param.values assert "hybrid" in search_mode_param.values - def test_chroma_excludes_hybrid_params(self): - params = get_default_ai4rag_search_space_parameters(vector_store_type="chroma") - param_names = {p.name for p in params} - - assert "search_mode" in param_names - assert "ranker_strategy" not in param_names - assert "ranker_k" not in param_names - assert "ranker_alpha" not in param_names - - search_mode_param = next(p for p in params if p.name == "search_mode") - assert search_mode_param.values == ("vector",) - assert "hybrid" not in search_mode_param.values - - def test_default_is_milvus(self): - params_default = get_default_ai4rag_search_space_parameters() - params_milvus = get_default_ai4rag_search_space_parameters(vector_store_type="milvus") - assert params_default == params_milvus - - def test_common_params_present_for_both_types(self): + def test_common_params_present(self): common_params = { "chunking_method", "chunk_size", @@ -342,45 +324,33 @@ def test_common_params_present_for_both_types(self): "number_of_chunks", "search_mode", } - for vs_type in ("milvus", "chroma"): - params = get_default_ai4rag_search_space_parameters(vector_store_type=vs_type) - param_names = {p.name for p in params} - assert common_params.issubset(param_names) + params = get_default_ai4rag_search_space_parameters() + param_names = {p.name for p in params} + assert common_params.issubset(param_names) -class TestAI4RAGSearchSpaceVectorStoreType: - def test_milvus_includes_hybrid_params_by_default(self): - ss = AI4RAGSearchSpace(params=list(_REQUIRED_PARAMS), vector_store_type="milvus") +class TestAI4RAGSearchSpaceHybridDefaults: + def test_includes_hybrid_params_by_default(self): + ss = AI4RAGSearchSpace(params=list(_REQUIRED_PARAMS)) param_names = {p.name for p in ss.params} assert _HYBRID_PARAM_NAMES.issubset(param_names) - def test_chroma_excludes_hybrid_params(self): - ss = AI4RAGSearchSpace(params=list(_REQUIRED_PARAMS), vector_store_type="chroma") - param_names = {p.name for p in ss.params} - assert "search_mode" in param_names - assert not _HYBRID_PARAM_NAMES.intersection({"ranker_strategy", "ranker_k", "ranker_alpha"}).intersection( - param_names - ) - - def test_chroma_search_mode_only_vector(self): - ss = AI4RAGSearchSpace(params=list(_REQUIRED_PARAMS), vector_store_type="chroma") + def test_search_mode_includes_hybrid(self): + ss = AI4RAGSearchSpace(params=list(_REQUIRED_PARAMS)) search_mode_param = ss["search_mode"] - assert search_mode_param.values == ("vector",) + assert "vector" in search_mode_param.values + assert "hybrid" in search_mode_param.values - def test_default_vector_store_type_is_milvus(self): + def test_applies_hybrid_rules(self): ss = AI4RAGSearchSpace(params=list(_REQUIRED_PARAMS)) - param_names = {p.name for p in ss.params} - assert _HYBRID_PARAM_NAMES.issubset(param_names) - - def test_chroma_does_not_apply_hybrid_rules(self): - ss = AI4RAGSearchSpace(params=list(_REQUIRED_PARAMS), vector_store_type="chroma") + search_modes = {c.get("search_mode") for c in ss.combinations} + assert "hybrid" in search_modes for combination in ss.combinations: - assert "ranker_strategy" not in combination - assert "ranker_k" not in combination - assert "ranker_alpha" not in combination + if combination.get("search_mode") == "vector": + assert combination["ranker_strategy"] == "" - def test_milvus_default_includes_hybrid_mode(self): - ss = AI4RAGSearchSpace(params=list(_REQUIRED_PARAMS), vector_store_type="milvus") + def test_default_includes_hybrid_mode(self): + ss = AI4RAGSearchSpace(params=list(_REQUIRED_PARAMS)) search_modes = {c["search_mode"] for c in ss.combinations} assert "vector" in search_modes assert "hybrid" in search_modes @@ -401,14 +371,14 @@ def test_milvus_default_includes_hybrid_mode(self): else: assert combination["ranker_alpha"] == 1 - def test_milvus_user_provided_hybrid_params_apply_rules(self): + def test_user_provided_hybrid_params_apply_rules(self): hybrid_params = list(_REQUIRED_PARAMS) + [ Parameter(name="search_mode", values=("vector", "hybrid")), Parameter(name="ranker_strategy", values=("", "rrf", "weighted")), Parameter(name="ranker_k", values=(0, 60)), Parameter(name="ranker_alpha", values=(1, 0.5)), ] - ss = AI4RAGSearchSpace(params=hybrid_params, vector_store_type="milvus") + ss = AI4RAGSearchSpace(params=hybrid_params) for combination in ss.combinations: search_mode = combination.get("search_mode") if search_mode == "vector": diff --git a/tests/unit/ai4rag/utils/event_handler/test_event_handler.py b/tests/unit/ai4rag/utils/event_handler/test_event_handler.py index d017f98b..b91483b5 100644 --- a/tests/unit/ai4rag/utils/event_handler/test_event_handler.py +++ b/tests/unit/ai4rag/utils/event_handler/test_event_handler.py @@ -30,7 +30,7 @@ "duration_seconds": 42, "final_score": 0.9, "settings": { - "vector_store_binding": {"provider_id": "local_chroma", "vector_store_id": "col_1"}, + "vector_store_binding": {"provider_type": "local_milvus", "collection_name": "col_1"}, "chunking": {"method": "recursive", "chunk_size": 512, "chunk_overlap": 64}, "embedding": {"model_id": "em-1", "distance_metric": "cosine", "embedding_params": {}}, "retrieval": {"method": "simple", "number_of_chunks": 3, "search_mode": "vector"}, diff --git a/tests/unit/ai4rag/utils/test_compat.py b/tests/unit/ai4rag/utils/test_compat.py deleted file mode 100644 index d6f31111..00000000 --- a/tests/unit/ai4rag/utils/test_compat.py +++ /dev/null @@ -1,128 +0,0 @@ -# ----------------------------------------------------------------------------- -# Copyright IBM Corp. 2026 -# SPDX-License-Identifier: Apache-2.0 -# ----------------------------------------------------------------------------- -import sys -import types - -import pytest - - -class TestEnsureSqlite3: - """Test suite for :func:`ensure_sqlite3` idempotency and error handling.""" - - @pytest.fixture(autouse=True) - def _isolate_sys_modules(self): - """Snapshot ``sys.modules`` before each test and restore it after. - - This prevents cross-test contamination from sqlite3/pysqlite3 patching. - """ - original_modules = sys.modules.copy() - yield - sys.modules.clear() - sys.modules.update(original_modules) - - # ------------------------------------------------------------------ - # Fresh import helper -- forces a clean re-evaluation of the function - # ------------------------------------------------------------------ - - @staticmethod - def _import_ensure_sqlite3(): - """Import ``ensure_sqlite3`` fresh to avoid cached module state.""" - from ai4rag.utils.compat import ensure_sqlite3 - - return ensure_sqlite3 - - # ------------------------------------------------------------------ - # Idempotency - # ------------------------------------------------------------------ - - def test_patches_sqlite3_even_if_pysqlite3_already_imported(self): - """``ensure_sqlite3`` must substitute ``sqlite3`` even when ``pysqlite3`` was already imported independently.""" - fake_pysqlite3 = types.ModuleType("pysqlite3") - sys.modules["pysqlite3"] = fake_pysqlite3 - - ensure_sqlite3 = self._import_ensure_sqlite3() - ensure_sqlite3() - - assert sys.modules["sqlite3"] is fake_pysqlite3 - - def test_idempotent_when_sqlite3_already_patched(self): - """Calling ``ensure_sqlite3`` is a no-op if ``sqlite3`` is already the pysqlite3 module.""" - fake_pysqlite3 = types.ModuleType("pysqlite3") - fake_pysqlite3.__name__ = "pysqlite3" - sys.modules["sqlite3"] = fake_pysqlite3 - - ensure_sqlite3 = self._import_ensure_sqlite3() - ensure_sqlite3() - - # Should still be the same object. - assert sys.modules["sqlite3"] is fake_pysqlite3 - - def test_double_call_is_noop(self): - """Two successive calls must not raise and must leave ``sys.modules`` consistent.""" - fake_pysqlite3 = types.ModuleType("pysqlite3") - sys.modules["pysqlite3"] = fake_pysqlite3 - - ensure_sqlite3 = self._import_ensure_sqlite3() - ensure_sqlite3() - ensure_sqlite3() # second call -- should be harmless - - # ------------------------------------------------------------------ - # pysqlite3 unavailable (ImportError path) - # ------------------------------------------------------------------ - - def test_noop_when_pysqlite3_unavailable(self, mocker): - """``ensure_sqlite3`` silently passes when ``pysqlite3`` cannot be imported.""" - # Import the function *before* patching __import__. - ensure_sqlite3 = self._import_ensure_sqlite3() - - # Remove pysqlite3 from sys.modules so the early-exit check fails. - sys.modules.pop("pysqlite3", None) - # Ensure sqlite3.__name__ is not "pysqlite3" (second early-exit check). - original_sqlite3 = sys.modules.get("sqlite3") - if original_sqlite3 is not None: - original_name = getattr(original_sqlite3, "__name__", None) - if original_name == "pysqlite3": - original_sqlite3.__name__ = "sqlite3" - - import builtins - - original_import = builtins.__import__ - - def _guarded_import(name, *args, **kwargs): - if name == "pysqlite3": - raise ImportError("no pysqlite3") - return original_import(name, *args, **kwargs) - - mocker.patch("builtins.__import__", side_effect=_guarded_import) - - ensure_sqlite3() # must not raise - - # sqlite3 should NOT have been replaced (the ImportError was caught). - if original_sqlite3 is not None: - assert sys.modules.get("sqlite3") is original_sqlite3 - - # ------------------------------------------------------------------ - # Successful patching path - # ------------------------------------------------------------------ - - def test_patches_sqlite3_when_pysqlite3_available(self): - """When ``pysqlite3`` can be imported, ``sys.modules["sqlite3"]`` is replaced.""" - sys.modules.pop("pysqlite3", None) - - fake_pysqlite3 = types.ModuleType("pysqlite3") - sys.modules["pysqlite3"] = fake_pysqlite3 - - # Remove the early-exit condition. - original_sqlite3 = sys.modules.get("sqlite3") - if original_sqlite3 is not None and getattr(original_sqlite3, "__name__", None) == "pysqlite3": - # Reset so the function doesn't short-circuit. - original_sqlite3.__name__ = "sqlite3" - - ensure_sqlite3 = self._import_ensure_sqlite3() - ensure_sqlite3() - - # pysqlite3 should have been registered, triggering the early-exit - # on any subsequent call. - assert "pysqlite3" in sys.modules diff --git a/uv.lock b/uv.lock index 9f1e99a1..43f12ad7 100644 --- a/uv.lock +++ b/uv.lock @@ -31,7 +31,6 @@ source = { editable = "." } dependencies = [ { name = "asyncpg" }, { name = "boto3" }, - { name = "chromadb" }, { name = "docling-slim", extra = ["feat-chunking"] }, { name = "langchain-community" }, { name = "langchain-text-splitters" }, @@ -41,7 +40,7 @@ dependencies = [ { name = "pgvector" }, { name = "pydantic" }, { name = "pygam" }, - { name = "pymilvus" }, + { name = "pymilvus", extra = ["milvus-lite"] }, { name = "ragas" }, { name = "scikit-learn" }, { name = "unitxt" }, @@ -101,7 +100,6 @@ requires-dist = [ { name = "beautifulsoup4", marker = "extra == 'dev'" }, { name = "black", marker = "extra == 'code-check'" }, { name = "boto3", specifier = ">=1.28" }, - { name = "chromadb", specifier = ">=1.5,<2" }, { name = "docling-slim", extras = ["feat-chunking"], specifier = "~=2.121.0" }, { name = "docling-slim", extras = ["format-audio", "format-opendocument", "standard"], marker = "extra == 'text-extraction'", specifier = "~=2.121.0" }, { name = "dotenv", marker = "extra == 'dev'" }, @@ -123,7 +121,7 @@ requires-dist = [ { name = "pydantic", specifier = "==2.11.*" }, { name = "pygam", specifier = "~=0.12.0" }, { name = "pylint", marker = "extra == 'code-check'" }, - { name = "pymilvus", specifier = "~=3.0.1" }, + { name = "pymilvus", extras = ["milvus-lite"], specifier = "~=3.0.1" }, { name = "pytest", marker = "extra == 'test'" }, { name = "pytest-cov", marker = "extra == 'test'" }, { name = "pytest-mock", marker = "extra == 'test'" }, @@ -358,59 +356,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/8a/69/47a3dc20abc4fa5486655fde681bd55e63211b46c886d8c02223d6468431/backrefs-8.0-py313-none-any.whl", hash = "sha256:601ce68ca12385dbda06ce264406b4c4210cf5b79fd0fd627592365c92f29a88", size = 400040, upload-time = "2026-07-26T19:54:21.194Z" }, ] -[[package]] -name = "bcrypt" -version = "5.0.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/d4/36/3329e2518d70ad8e2e5817d5a4cac6bba05a47767ec416c7d020a965f408/bcrypt-5.0.0.tar.gz", hash = "sha256:f748f7c2d6fd375cc93d3fba7ef4a9e3a092421b8dbf34d8d4dc06be9492dfdd", size = 25386, upload-time = "2025-09-25T19:50:47.829Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/13/85/3e65e01985fddf25b64ca67275bb5bdb4040bd1a53b66d355c6c37c8a680/bcrypt-5.0.0-cp313-cp313t-macosx_10_12_universal2.whl", hash = "sha256:f3c08197f3039bec79cee59a606d62b96b16669cff3949f21e74796b6e3cd2be", size = 481806, upload-time = "2025-09-25T19:49:05.102Z" }, - { url = "https://files.pythonhosted.org/packages/44/dc/01eb79f12b177017a726cbf78330eb0eb442fae0e7b3dfd84ea2849552f3/bcrypt-5.0.0-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:200af71bc25f22006f4069060c88ed36f8aa4ff7f53e67ff04d2ab3f1e79a5b2", size = 268626, upload-time = "2025-09-25T19:49:06.723Z" }, - { url = "https://files.pythonhosted.org/packages/8c/cf/e82388ad5959c40d6afd94fb4743cc077129d45b952d46bdc3180310e2df/bcrypt-5.0.0-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:baade0a5657654c2984468efb7d6c110db87ea63ef5a4b54732e7e337253e44f", size = 271853, upload-time = "2025-09-25T19:49:08.028Z" }, - { url = "https://files.pythonhosted.org/packages/ec/86/7134b9dae7cf0efa85671651341f6afa695857fae172615e960fb6a466fa/bcrypt-5.0.0-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:c58b56cdfb03202b3bcc9fd8daee8e8e9b6d7e3163aa97c631dfcfcc24d36c86", size = 269793, upload-time = "2025-09-25T19:49:09.727Z" }, - { url = "https://files.pythonhosted.org/packages/cc/82/6296688ac1b9e503d034e7d0614d56e80c5d1a08402ff856a4549cb59207/bcrypt-5.0.0-cp313-cp313t-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:4bfd2a34de661f34d0bda43c3e4e79df586e4716ef401fe31ea39d69d581ef23", size = 289930, upload-time = "2025-09-25T19:49:11.204Z" }, - { url = "https://files.pythonhosted.org/packages/d1/18/884a44aa47f2a3b88dd09bc05a1e40b57878ecd111d17e5bba6f09f8bb77/bcrypt-5.0.0-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:ed2e1365e31fc73f1825fa830f1c8f8917ca1b3ca6185773b349c20fd606cec2", size = 272194, upload-time = "2025-09-25T19:49:12.524Z" }, - { url = "https://files.pythonhosted.org/packages/0e/8f/371a3ab33c6982070b674f1788e05b656cfbf5685894acbfef0c65483a59/bcrypt-5.0.0-cp313-cp313t-manylinux_2_34_aarch64.whl", hash = "sha256:83e787d7a84dbbfba6f250dd7a5efd689e935f03dd83b0f919d39349e1f23f83", size = 269381, upload-time = "2025-09-25T19:49:14.308Z" }, - { url = "https://files.pythonhosted.org/packages/b1/34/7e4e6abb7a8778db6422e88b1f06eb07c47682313997ee8a8f9352e5a6f1/bcrypt-5.0.0-cp313-cp313t-manylinux_2_34_x86_64.whl", hash = "sha256:137c5156524328a24b9fac1cb5db0ba618bc97d11970b39184c1d87dc4bf1746", size = 271750, upload-time = "2025-09-25T19:49:15.584Z" }, - { url = "https://files.pythonhosted.org/packages/c0/1b/54f416be2499bd72123c70d98d36c6cd61a4e33d9b89562c22481c81bb30/bcrypt-5.0.0-cp313-cp313t-musllinux_1_1_aarch64.whl", hash = "sha256:38cac74101777a6a7d3b3e3cfefa57089b5ada650dce2baf0cbdd9d65db22a9e", size = 303757, upload-time = "2025-09-25T19:49:17.244Z" }, - { url = "https://files.pythonhosted.org/packages/13/62/062c24c7bcf9d2826a1a843d0d605c65a755bc98002923d01fd61270705a/bcrypt-5.0.0-cp313-cp313t-musllinux_1_1_x86_64.whl", hash = "sha256:d8d65b564ec849643d9f7ea05c6d9f0cd7ca23bdd4ac0c2dbef1104ab504543d", size = 306740, upload-time = "2025-09-25T19:49:18.693Z" }, - { url = "https://files.pythonhosted.org/packages/d5/c8/1fdbfc8c0f20875b6b4020f3c7dc447b8de60aa0be5faaf009d24242aec9/bcrypt-5.0.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:741449132f64b3524e95cd30e5cd3343006ce146088f074f31ab26b94e6c75ba", size = 334197, upload-time = "2025-09-25T19:49:20.523Z" }, - { url = "https://files.pythonhosted.org/packages/a6/c1/8b84545382d75bef226fbc6588af0f7b7d095f7cd6a670b42a86243183cd/bcrypt-5.0.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:212139484ab3207b1f0c00633d3be92fef3c5f0af17cad155679d03ff2ee1e41", size = 352974, upload-time = "2025-09-25T19:49:22.254Z" }, - { url = "https://files.pythonhosted.org/packages/10/a6/ffb49d4254ed085e62e3e5dd05982b4393e32fe1e49bb1130186617c29cd/bcrypt-5.0.0-cp313-cp313t-win32.whl", hash = "sha256:9d52ed507c2488eddd6a95bccee4e808d3234fa78dd370e24bac65a21212b861", size = 148498, upload-time = "2025-09-25T19:49:24.134Z" }, - { url = "https://files.pythonhosted.org/packages/48/a9/259559edc85258b6d5fc5471a62a3299a6aa37a6611a169756bf4689323c/bcrypt-5.0.0-cp313-cp313t-win_amd64.whl", hash = "sha256:f6984a24db30548fd39a44360532898c33528b74aedf81c26cf29c51ee47057e", size = 145853, upload-time = "2025-09-25T19:49:25.702Z" }, - { url = "https://files.pythonhosted.org/packages/2d/df/9714173403c7e8b245acf8e4be8876aac64a209d1b392af457c79e60492e/bcrypt-5.0.0-cp313-cp313t-win_arm64.whl", hash = "sha256:9fffdb387abe6aa775af36ef16f55e318dcda4194ddbf82007a6f21da29de8f5", size = 139626, upload-time = "2025-09-25T19:49:26.928Z" }, - { url = "https://files.pythonhosted.org/packages/84/29/6237f151fbfe295fe3e074ecc6d44228faa1e842a81f6d34a02937ee1736/bcrypt-5.0.0-cp38-abi3-macosx_10_12_universal2.whl", hash = "sha256:fc746432b951e92b58317af8e0ca746efe93e66555f1b40888865ef5bf56446b", size = 494553, upload-time = "2025-09-25T19:49:49.006Z" }, - { url = "https://files.pythonhosted.org/packages/45/b6/4c1205dde5e464ea3bd88e8742e19f899c16fa8916fb8510a851fae985b5/bcrypt-5.0.0-cp38-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:c2388ca94ffee269b6038d48747f4ce8df0ffbea43f31abfa18ac72f0218effb", size = 275009, upload-time = "2025-09-25T19:49:50.581Z" }, - { url = "https://files.pythonhosted.org/packages/3b/71/427945e6ead72ccffe77894b2655b695ccf14ae1866cd977e185d606dd2f/bcrypt-5.0.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:560ddb6ec730386e7b3b26b8b4c88197aaed924430e7b74666a586ac997249ef", size = 278029, upload-time = "2025-09-25T19:49:52.533Z" }, - { url = "https://files.pythonhosted.org/packages/17/72/c344825e3b83c5389a369c8a8e58ffe1480b8a699f46c127c34580c4666b/bcrypt-5.0.0-cp38-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:d79e5c65dcc9af213594d6f7f1fa2c98ad3fc10431e7aa53c176b441943efbdd", size = 275907, upload-time = "2025-09-25T19:49:54.709Z" }, - { url = "https://files.pythonhosted.org/packages/0b/7e/d4e47d2df1641a36d1212e5c0514f5291e1a956a7749f1e595c07a972038/bcrypt-5.0.0-cp38-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:2b732e7d388fa22d48920baa267ba5d97cca38070b69c0e2d37087b381c681fd", size = 296500, upload-time = "2025-09-25T19:49:56.013Z" }, - { url = "https://files.pythonhosted.org/packages/0f/c3/0ae57a68be2039287ec28bc463b82e4b8dc23f9d12c0be331f4782e19108/bcrypt-5.0.0-cp38-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:0c8e093ea2532601a6f686edbc2c6b2ec24131ff5c52f7610dd64fa4553b5464", size = 278412, upload-time = "2025-09-25T19:49:57.356Z" }, - { url = "https://files.pythonhosted.org/packages/45/2b/77424511adb11e6a99e3a00dcc7745034bee89036ad7d7e255a7e47be7d8/bcrypt-5.0.0-cp38-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:5b1589f4839a0899c146e8892efe320c0fa096568abd9b95593efac50a87cb75", size = 275486, upload-time = "2025-09-25T19:49:59.116Z" }, - { url = "https://files.pythonhosted.org/packages/43/0a/405c753f6158e0f3f14b00b462d8bca31296f7ecfc8fc8bc7919c0c7d73a/bcrypt-5.0.0-cp38-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:89042e61b5e808b67daf24a434d89bab164d4de1746b37a8d173b6b14f3db9ff", size = 277940, upload-time = "2025-09-25T19:50:00.869Z" }, - { url = "https://files.pythonhosted.org/packages/62/83/b3efc285d4aadc1fa83db385ec64dcfa1707e890eb42f03b127d66ac1b7b/bcrypt-5.0.0-cp38-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:e3cf5b2560c7b5a142286f69bde914494b6d8f901aaa71e453078388a50881c4", size = 310776, upload-time = "2025-09-25T19:50:02.393Z" }, - { url = "https://files.pythonhosted.org/packages/95/7d/47ee337dacecde6d234890fe929936cb03ebc4c3a7460854bbd9c97780b8/bcrypt-5.0.0-cp38-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:f632fd56fc4e61564f78b46a2269153122db34988e78b6be8b32d28507b7eaeb", size = 312922, upload-time = "2025-09-25T19:50:04.232Z" }, - { url = "https://files.pythonhosted.org/packages/d6/3a/43d494dfb728f55f4e1cf8fd435d50c16a2d75493225b54c8d06122523c6/bcrypt-5.0.0-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:801cad5ccb6b87d1b430f183269b94c24f248dddbbc5c1f78b6ed231743e001c", size = 341367, upload-time = "2025-09-25T19:50:05.559Z" }, - { url = "https://files.pythonhosted.org/packages/55/ab/a0727a4547e383e2e22a630e0f908113db37904f58719dc48d4622139b5c/bcrypt-5.0.0-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:3cf67a804fc66fc217e6914a5635000259fbbbb12e78a99488e4d5ba445a71eb", size = 359187, upload-time = "2025-09-25T19:50:06.916Z" }, - { url = "https://files.pythonhosted.org/packages/1b/bb/461f352fdca663524b4643d8b09e8435b4990f17fbf4fea6bc2a90aa0cc7/bcrypt-5.0.0-cp38-abi3-win32.whl", hash = "sha256:3abeb543874b2c0524ff40c57a4e14e5d3a66ff33fb423529c88f180fd756538", size = 153752, upload-time = "2025-09-25T19:50:08.515Z" }, - { url = "https://files.pythonhosted.org/packages/41/aa/4190e60921927b7056820291f56fc57d00d04757c8b316b2d3c0d1d6da2c/bcrypt-5.0.0-cp38-abi3-win_amd64.whl", hash = "sha256:35a77ec55b541e5e583eb3436ffbbf53b0ffa1fa16ca6782279daf95d146dcd9", size = 150881, upload-time = "2025-09-25T19:50:09.742Z" }, - { url = "https://files.pythonhosted.org/packages/54/12/cd77221719d0b39ac0b55dbd39358db1cd1246e0282e104366ebbfb8266a/bcrypt-5.0.0-cp38-abi3-win_arm64.whl", hash = "sha256:cde08734f12c6a4e28dc6755cd11d3bdfea608d93d958fffbe95a7026ebe4980", size = 144931, upload-time = "2025-09-25T19:50:11.016Z" }, - { url = "https://files.pythonhosted.org/packages/5d/ba/2af136406e1c3839aea9ecadc2f6be2bcd1eff255bd451dd39bcf302c47a/bcrypt-5.0.0-cp39-abi3-macosx_10_12_universal2.whl", hash = "sha256:0c418ca99fd47e9c59a301744d63328f17798b5947b0f791e9af3c1c499c2d0a", size = 495313, upload-time = "2025-09-25T19:50:12.309Z" }, - { url = "https://files.pythonhosted.org/packages/ac/ee/2f4985dbad090ace5ad1f7dd8ff94477fe089b5fab2040bd784a3d5f187b/bcrypt-5.0.0-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:ddb4e1500f6efdd402218ffe34d040a1196c072e07929b9820f363a1fd1f4191", size = 275290, upload-time = "2025-09-25T19:50:13.673Z" }, - { url = "https://files.pythonhosted.org/packages/e4/6e/b77ade812672d15cf50842e167eead80ac3514f3beacac8902915417f8b7/bcrypt-5.0.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7aeef54b60ceddb6f30ee3db090351ecf0d40ec6e2abf41430997407a46d2254", size = 278253, upload-time = "2025-09-25T19:50:15.089Z" }, - { url = "https://files.pythonhosted.org/packages/36/c4/ed00ed32f1040f7990dac7115f82273e3c03da1e1a1587a778d8cea496d8/bcrypt-5.0.0-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:f0ce778135f60799d89c9693b9b398819d15f1921ba15fe719acb3178215a7db", size = 276084, upload-time = "2025-09-25T19:50:16.699Z" }, - { url = "https://files.pythonhosted.org/packages/e7/c4/fa6e16145e145e87f1fa351bbd54b429354fd72145cd3d4e0c5157cf4c70/bcrypt-5.0.0-cp39-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:a71f70ee269671460b37a449f5ff26982a6f2ba493b3eabdd687b4bf35f875ac", size = 297185, upload-time = "2025-09-25T19:50:18.525Z" }, - { url = "https://files.pythonhosted.org/packages/24/b4/11f8a31d8b67cca3371e046db49baa7c0594d71eb40ac8121e2fc0888db0/bcrypt-5.0.0-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:f8429e1c410b4073944f03bd778a9e066e7fad723564a52ff91841d278dfc822", size = 278656, upload-time = "2025-09-25T19:50:19.809Z" }, - { url = "https://files.pythonhosted.org/packages/ac/31/79f11865f8078e192847d2cb526e3fa27c200933c982c5b2869720fa5fce/bcrypt-5.0.0-cp39-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:edfcdcedd0d0f05850c52ba3127b1fce70b9f89e0fe5ff16517df7e81fa3cbb8", size = 275662, upload-time = "2025-09-25T19:50:21.567Z" }, - { url = "https://files.pythonhosted.org/packages/d4/8d/5e43d9584b3b3591a6f9b68f755a4da879a59712981ef5ad2a0ac1379f7a/bcrypt-5.0.0-cp39-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:611f0a17aa4a25a69362dcc299fda5c8a3d4f160e2abb3831041feb77393a14a", size = 278240, upload-time = "2025-09-25T19:50:23.305Z" }, - { url = "https://files.pythonhosted.org/packages/89/48/44590e3fc158620f680a978aafe8f87a4c4320da81ed11552f0323aa9a57/bcrypt-5.0.0-cp39-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:db99dca3b1fdc3db87d7c57eac0c82281242d1eabf19dcb8a6b10eb29a2e72d1", size = 311152, upload-time = "2025-09-25T19:50:24.597Z" }, - { url = "https://files.pythonhosted.org/packages/5f/85/e4fbfc46f14f47b0d20493669a625da5827d07e8a88ee460af6cd9768b44/bcrypt-5.0.0-cp39-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:5feebf85a9cefda32966d8171f5db7e3ba964b77fdfe31919622256f80f9cf42", size = 313284, upload-time = "2025-09-25T19:50:26.268Z" }, - { url = "https://files.pythonhosted.org/packages/25/ae/479f81d3f4594456a01ea2f05b132a519eff9ab5768a70430fa1132384b1/bcrypt-5.0.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:3ca8a166b1140436e058298a34d88032ab62f15aae1c598580333dc21d27ef10", size = 341643, upload-time = "2025-09-25T19:50:28.02Z" }, - { url = "https://files.pythonhosted.org/packages/df/d2/36a086dee1473b14276cd6ea7f61aef3b2648710b5d7f1c9e032c29b859f/bcrypt-5.0.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:61afc381250c3182d9078551e3ac3a41da14154fbff647ddf52a769f588c4172", size = 359698, upload-time = "2025-09-25T19:50:31.347Z" }, - { url = "https://files.pythonhosted.org/packages/c0/f6/688d2cd64bfd0b14d805ddb8a565e11ca1fb0fd6817175d58b10052b6d88/bcrypt-5.0.0-cp39-abi3-win32.whl", hash = "sha256:64d7ce196203e468c457c37ec22390f1a61c85c6f0b8160fd752940ccfb3a683", size = 153725, upload-time = "2025-09-25T19:50:34.384Z" }, - { url = "https://files.pythonhosted.org/packages/9f/b9/9d9a641194a730bda138b3dfe53f584d61c58cd5230e37566e83ec2ffa0d/bcrypt-5.0.0-cp39-abi3-win_amd64.whl", hash = "sha256:64ee8434b0da054d830fa8e89e1c8bf30061d539044a39524ff7dec90481e5c2", size = 150912, upload-time = "2025-09-25T19:50:35.69Z" }, - { url = "https://files.pythonhosted.org/packages/27/44/d2ef5e87509158ad2187f4dd0852df80695bb1ee0cfe0a684727b01a69e0/bcrypt-5.0.0-cp39-abi3-win_arm64.whl", hash = "sha256:f2347d3534e76bf50bca5500989d6c1d05ed64b440408057a37673282c654927", size = 144953, upload-time = "2025-09-25T19:50:37.32Z" }, -] - [[package]] name = "beautifulsoup4" version = "4.15.0" @@ -479,20 +424,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/72/b0/2981d533ebf7f93a3fa8493fb243e11225f31ad5ae36f3259fe7215d9141/botocore-1.43.88-py3-none-any.whl", hash = "sha256:1b59b6d74fb77b0c3934014b6693d56da4a9a172edb9454f9a5f711b4b3f7353", size = 15765383, upload-time = "2026-09-03T19:23:57.936Z" }, ] -[[package]] -name = "build" -version = "1.6.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "colorama", marker = "(python_full_version < '3.13' and os_name == 'nt') or (os_name == 'nt' and sys_platform != 'darwin')" }, - { name = "packaging" }, - { name = "pyproject-hooks" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/4d/b7/1db48a9ce2984842c8c886432ec8a2719613322e868a966ba82a28862f25/build-1.6.0.tar.gz", hash = "sha256:bd2c8afc603e7a2e0ce70e2ea85f0a6d02043bafbd307f5bada0f98669eca5af", size = 113825, upload-time = "2026-08-27T21:01:16.458Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/ab/e5/aa1e81b21aea0ce0ba435311837a37d4cb936e7461f9fecac08580073ba9/build-1.6.0-py3-none-any.whl", hash = "sha256:f7aaf1ebbb79178a02ba248bb524f2176b256017e17e8e4bd4289c7b38cc2bad", size = 31187, upload-time = "2026-08-27T21:01:14.957Z" }, -] - [[package]] name = "cachetools" version = "7.1.8" @@ -610,48 +541,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/cc/61/d01fc49b8dea277640b55a9e15960dbca9fdc8c9fde18e572d39c59f4019/charset_normalizer-3.5.1-py3-none-any.whl", hash = "sha256:6df0ec430f9a831772c23ca5a224cba36517a58a84bb32c32bb59a9fa67c47f6", size = 68658, upload-time = "2026-08-15T08:20:43.306Z" }, ] -[[package]] -name = "chromadb" -version = "1.5.9" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "bcrypt" }, - { name = "build" }, - { name = "grpcio" }, - { name = "httpx" }, - { name = "importlib-resources" }, - { name = "jsonschema" }, - { name = "kubernetes" }, - { name = "mmh3" }, - { name = "numpy" }, - { name = "onnxruntime" }, - { name = "opentelemetry-api" }, - { name = "opentelemetry-exporter-otlp-proto-grpc" }, - { name = "opentelemetry-sdk" }, - { name = "orjson" }, - { name = "overrides" }, - { name = "pybase64" }, - { name = "pydantic" }, - { name = "pydantic-settings" }, - { name = "pypika" }, - { name = "pyyaml" }, - { name = "rich" }, - { name = "tenacity" }, - { name = "tokenizers" }, - { name = "tqdm" }, - { name = "typer" }, - { name = "typing-extensions" }, - { name = "uvicorn", extra = ["standard"] }, -] -sdist = { url = "https://files.pythonhosted.org/packages/92/d1/5e33b26985f0c7046a0be1cee2158ada1748ee700d2545057fde1468d74d/chromadb-1.5.9.tar.gz", hash = "sha256:5c20e62a455c28bacac927f26116a73fd8e1799e0d908be8e8a4f02197a54731", size = 2595635, upload-time = "2026-05-05T05:54:51.713Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/dd/5b/3cced915244f43ed14b53fe9f63a37f05f865064f4e4fe7d9448d3f2a352/chromadb-1.5.9-cp39-abi3-macosx_10_12_x86_64.whl", hash = "sha256:60701011b5e6409647fa40d12c7c5a66b2b0bfcf33a52db2ad53a30a2abc4957", size = 22564540, upload-time = "2026-05-05T05:54:48.906Z" }, - { url = "https://files.pythonhosted.org/packages/34/4c/adcef1f4e82a2ef69ccd3711d55fc289193d54c4c0ff7a0292a3631db46f/chromadb-1.5.9-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:814b9c95617377f6501e5757d63dfddb554a283a7739c87b9fa573850174e6f3", size = 21699698, upload-time = "2026-05-05T05:54:45.078Z" }, - { url = "https://files.pythonhosted.org/packages/38/4e/937bc4d2e6f8ab9664ec79931fbbd69efff47e513ec2924b071e4b0ff774/chromadb-1.5.9-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9192d111bd662241625867962333d99369a00769a50f8b2f58cb388731274d7e", size = 22680924, upload-time = "2026-05-05T05:54:36.25Z" }, - { url = "https://files.pythonhosted.org/packages/e6/ec/0c42039e80b9acc534f67b73b7a42471948042859b3a64867b50a4a77fa3/chromadb-1.5.9-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:cc09b3df76e5a5cb386aed2715a2eea152e3949f9e1ba93c7119505377749929", size = 23316203, upload-time = "2026-05-05T05:54:41.157Z" }, - { url = "https://files.pythonhosted.org/packages/eb/ce/0f7be6e5d0feafa2cda54b12e6542afeea7dea89d2d411e14da90f8abb96/chromadb-1.5.9-cp39-abi3-win_amd64.whl", hash = "sha256:4fd0b560e56761b7f3cb4d5c6205fd5f20814484b4a3e4e9af9038c2b428fc6c", size = 23542454, upload-time = "2026-05-05T05:54:54.942Z" }, -] - [[package]] name = "click" version = "8.5.0" @@ -1107,15 +996,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/b2/b7/545d2c10c1fc15e48653c91efde329a790f2eecfbbf2bd16003b5db2bab0/dotenv-0.9.9-py2.py3-none-any.whl", hash = "sha256:29cf74a087b31dafdb5a446b6d7e11cbce8ed2741540e2339c69fbef92c94ce9", size = 1892, upload-time = "2025-02-19T22:15:01.647Z" }, ] -[[package]] -name = "durationpy" -version = "0.11" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/b0/5d/5f8571bd5dedc80863191621ac4be001f3f3dd8315d2ec078705dab7dec1/durationpy-0.11.tar.gz", hash = "sha256:181898e1ae282e288f0a2291829656bf1b6b3aadf30a97993b85db4943642905", size = 3582, upload-time = "2026-08-26T13:56:00.991Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/e6/c4/ebdf7837bc4ef6fd98cfb013c28855bb358467bf86c1af011bbc21e21df0/durationpy-0.11-py3-none-any.whl", hash = "sha256:a739fe2b8972c250ff72f8e2c488d18cf25f7b852f49ee76048775d5171df30c", size = 4133, upload-time = "2026-08-26T13:55:59.456Z" }, -] - [[package]] name = "et-xmlfile" version = "2.0.0" @@ -1156,6 +1036,23 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/c1/ea/53f2148663b321f21b5a606bd5f191517cf40b7072c0497d3c92c4a13b1e/executing-2.2.1-py2.py3-none-any.whl", hash = "sha256:760643d3452b4d777d295bb167ccc74c64a81df23fb5e08eff250c425a4b2017", size = 28317, upload-time = "2025-09-01T09:48:08.5Z" }, ] +[[package]] +name = "faiss-cpu" +version = "1.15.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy" }, + { name = "packaging" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/59/68/20e91694ad9a8b2bb48af956899e52b645cb1501e7e2ec31cb733da4d4c5/faiss_cpu-1.15.0-cp310-abi3-macosx_14_0_arm64.whl", hash = "sha256:50ea471ef1f4f3580eda8ab0ec9727d4bf65fd71c444bf306ce7cdbba8a42b21", size = 4904897, upload-time = "2026-08-03T17:49:37.003Z" }, + { url = "https://files.pythonhosted.org/packages/d2/cd/ef4cf498977c4a84af7a8920bc97ca49fc19060c8464c63fab58847b4692/faiss_cpu-1.15.0-cp310-abi3-macosx_15_0_x86_64.whl", hash = "sha256:dd383bb1ce06fabcff5785f998f253aa88f88dcbe1fe36c922417cd6666dd896", size = 7087977, upload-time = "2026-08-03T17:49:38.947Z" }, + { url = "https://files.pythonhosted.org/packages/94/c8/88b072bf55714405d0d7e11c12349510f15a69ae56033b1cd894fb2be7d6/faiss_cpu-1.15.0-cp310-abi3-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5d0a2d5d33fe023e263d0d355a837f20db67578e3be27fc5f4012a273274abf6", size = 9835009, upload-time = "2026-08-03T17:49:40.8Z" }, + { url = "https://files.pythonhosted.org/packages/c8/3b/8878dbfc78a0084bbd408b34827a58b530be98132fcf620b7e15f9191614/faiss_cpu-1.15.0-cp310-abi3-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ec9b29aae29e428c085c2d49dbb02e4673cdea75db418d420f9e60e0b4184498", size = 18764625, upload-time = "2026-08-03T17:49:43.676Z" }, + { url = "https://files.pythonhosted.org/packages/db/2a/654116e6ee2808562a6b2a11c396bdb46d45689e3bf7206ee99400589cab/faiss_cpu-1.15.0-cp310-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:30da3029952f0de69f16ce31946fd63fc3e292c867749bbcd2c0a0f09fd06f65", size = 11413863, upload-time = "2026-08-03T17:49:46.471Z" }, + { url = "https://files.pythonhosted.org/packages/9d/8c/0a0f09659c1972aa83b9820cd3dd7f68f6678cfcfebde542e1c23d7d8663/faiss_cpu-1.15.0-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:88fbe1acac6978869063cb2f9477f85718da596a6e0a17751618f9c756bce255", size = 19470092, upload-time = "2026-08-03T17:49:50.253Z" }, +] + [[package]] name = "faker" version = "40.38.0" @@ -1195,14 +1092,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/18/79/1b8fa1bb3568781e84c9200f951c735f3f157429f44be0495da55894d620/filetype-1.2.0-py2.py3-none-any.whl", hash = "sha256:7ce71b6880181241cf7ac8697a2f1eb6a8bd9b429f7ad6d27b8db9ba5f1c2d25", size = 19970, upload-time = "2022-11-02T17:34:01.425Z" }, ] -[[package]] -name = "flatbuffers" -version = "25.12.19" -source = { registry = "https://pypi.org/simple" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/e8/2d/d2a548598be01649e2d46231d151a6c56d10b964d94043a335ae56ea2d92/flatbuffers-25.12.19-py2.py3-none-any.whl", hash = "sha256:7634f50c427838bb021c2d66a3d1168e9d199b0607e6329399f04846d42e20b4", size = 26661, upload-time = "2025-12-19T23:16:13.622Z" }, -] - [[package]] name = "frozenlist" version = "1.8.0" @@ -1310,18 +1199,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/6f/5e/49cc172da4d0578644ba37cec5cb365b1fefc603b26edea9bcac1c7f830a/gitpython-3.1.61-py3-none-any.whl", hash = "sha256:8ab28c9da863cdd9e7d7694ec46cf3e6c9a12d8a30a1acd3447aec11975d530c", size = 222118, upload-time = "2026-08-28T11:01:12.262Z" }, ] -[[package]] -name = "googleapis-common-protos" -version = "1.75.3" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "protobuf" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/8a/c5/4353a188e2c335aee33269e8b654af228278cca8e5f0b4b5f11e5d0e9adb/googleapis_common_protos-1.75.3.tar.gz", hash = "sha256:57c435ac2c68b108999b6db075d9053e4d7a936ba57b4a3d45667b1346f1738a", size = 153905, upload-time = "2026-09-03T22:31:21.869Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/1a/7a/7d79170c6ce6f12e109df2b3879d6b934010cf4f99aea8de8b7e5408c174/googleapis_common_protos-1.75.3-py3-none-any.whl", hash = "sha256:a018d2bf098ca9fb6faa08d5bb780e2a2c2f73c566f069761331386c9596d3f2", size = 306984, upload-time = "2026-09-03T22:30:45.133Z" }, -] - [[package]] name = "greenlet" version = "3.5.5" @@ -1445,28 +1322,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/d2/74/d370e55600d9bcfa0d9794b0166126d49291a3d2b20c268fc98c453a4948/httpcore2-2.12.0-py3-none-any.whl", hash = "sha256:7e04258ce01013d7d615e5b910a3b27fac937d7a95038227e79652b4ba3b4ceb", size = 83074, upload-time = "2026-08-18T13:22:05.854Z" }, ] -[[package]] -name = "httptools" -version = "0.8.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/43/e5/d471fcb0e14523fe1c3f4ba58ca52480e7bd70ad7109a3846bc75892f7fb/httptools-0.8.0.tar.gz", hash = "sha256:6b2a32f18d97e16e90827d7a819ffa8dbd8cc245fc4e1fa9d1095b54ef4bd999", size = 271342, upload-time = "2026-05-25T22:17:48.841Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/14/88/1d21a36da8f5cb0fa49eafd4b169eba5608d57e75bbcf61845cbc6243216/httptools-0.8.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:880490234c10f70a9830743097e8958d6e4b9f5a0ffc24515023afeef984054d", size = 208247, upload-time = "2026-05-25T22:17:07.843Z" }, - { url = "https://files.pythonhosted.org/packages/a5/42/cc4feea2945cb3051038f090c9b36bd5b8a9d7f5a894a506a8983e33fd1c/httptools-0.8.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:5931891fb7b441b8a3853cf1b85c82c903defce084dd5f6771ca46e31bf862c5", size = 113064, upload-time = "2026-05-25T22:17:09.136Z" }, - { url = "https://files.pythonhosted.org/packages/e3/a6/febbb8b8db0f58b38e44ad6cb946e6a255ae49b55f2e8543408fb7501ccd/httptools-0.8.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:b15fc622b0f869d19207c4089a501d9bcc63ca5e071ffdd2f03f922df882dcb2", size = 523851, upload-time = "2026-05-25T22:17:10.106Z" }, - { url = "https://files.pythonhosted.org/packages/b7/e4/f90a0df0b83beff265b7e3b65f2a4cefd95792d4be0ac3e16049f2acd3c2/httptools-0.8.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:425f83884fd6343828d8c565f046cb72b6d19063f6924093e11bcd8e1548cd09", size = 518842, upload-time = "2026-05-25T22:17:11.218Z" }, - { url = "https://files.pythonhosted.org/packages/9e/2d/0c9ac76dd2c893841fbf6498d6acec4f2442e1b7067f6e3e316a80e494e8/httptools-0.8.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:ef7c3c97f4311c7be57e2986629df89d49cb434dbff78eafcd48c2bff986b15a", size = 501238, upload-time = "2026-05-25T22:17:12.728Z" }, - { url = "https://files.pythonhosted.org/packages/ca/42/906adc91ae3a5fa9c59c0a2f21c139725bd7e5b41ae6acd485cd14123ebf/httptools-0.8.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:a1afd7c9fbff0d9f5d489c4ce2768bd09c84a46ddefc7161e6aa82ae35c85745", size = 509567, upload-time = "2026-05-25T22:17:13.842Z" }, - { url = "https://files.pythonhosted.org/packages/05/0b/4240efeb672751ee5b9b380cb0e3fdc050bc05f68adc7a8aefc4fcd9a69a/httptools-0.8.0-cp312-cp312-win_amd64.whl", hash = "sha256:cd96f29b4bab1d42fa6e3d008711c75e0f79e94e06827330160e3a304227f150", size = 90918, upload-time = "2026-05-25T22:17:15.155Z" }, - { url = "https://files.pythonhosted.org/packages/5e/e5/8cfcabc5546e8022f168be28bcdaa128a240a0befdd03b59d558b4f18bd6/httptools-0.8.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:614ceea8ea606848bece2338ac03b3ce5324bcb4be8dc7d377ed708012fa4db8", size = 205148, upload-time = "2026-05-25T22:17:16.333Z" }, - { url = "https://files.pythonhosted.org/packages/2a/0e/0fb14848c19a686c8062ff9067c1a48793e3224b47bc5b201535b6036fce/httptools-0.8.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2d689918c15a013c65ef52d9fd495d766893ab831a2c8d89f2ac5940a5df847c", size = 111368, upload-time = "2026-05-25T22:17:17.586Z" }, - { url = "https://files.pythonhosted.org/packages/2e/1b/46f1cecf06b9bbde8e4b8c88034ac7908989e5ff7a3a388ef38392949c1f/httptools-0.8.0-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:eb3028cca2fc0a6d720e52ef61d8ebb62fcbfeb1de56874546d858d3f25a26b7", size = 486447, upload-time = "2026-05-25T22:17:18.564Z" }, - { url = "https://files.pythonhosted.org/packages/77/00/258bfc0837221f81d9725c45f9b948a6a6b2994a147a4fb66e85100c668f/httptools-0.8.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:88bdd940f2b5d487b4d032c6afa5489a7dc4694410d43de3c38c4fb3af0dc45d", size = 482448, upload-time = "2026-05-25T22:17:19.912Z" }, - { url = "https://files.pythonhosted.org/packages/04/ab/d1cef3b5523f4d272a70f42a776c3169a2dddfe3a54de4b2ce4a36341528/httptools-0.8.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:6a43c9dd399758ccc0531acb0a3c4a6c299ee893ee9400e9c893b7bdcfae0681", size = 464460, upload-time = "2026-05-25T22:17:20.882Z" }, - { url = "https://files.pythonhosted.org/packages/ce/48/5d1d072442277bb2b3434e0e60690b8e8c23840ef7de8b6ea54040a536d3/httptools-0.8.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:0770728beb05094c809b98e814edff5fef69d26ad7d21185f2f6d5884a0ba683", size = 471312, upload-time = "2026-05-25T22:17:22.085Z" }, - { url = "https://files.pythonhosted.org/packages/0d/66/b96623b27e51a68199ef4efdda0613cced9233fe3062ac74e50749c5ad37/httptools-0.8.0-cp313-cp313-win_amd64.whl", hash = "sha256:7685df791fad561384bfb139e77fde27a1ffd93134e016f95a0db424ffbf77b1", size = 90117, upload-time = "2026-05-25T22:17:23.074Z" }, -] - [[package]] name = "httpx" version = "0.28.1" @@ -1546,15 +1401,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/57/b0/0e52c878c53f245edd3a11020f20979b3f490f245af532c7cae3027754b5/idna-3.19-py3-none-any.whl", hash = "sha256:815e7be7a7806d54abb586dc943addc79e8b2ee16915059658cbeff4b1b43bf4", size = 68550, upload-time = "2026-08-18T05:14:22.343Z" }, ] -[[package]] -name = "importlib-resources" -version = "7.1.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/e4/06/b56dfa750b44e86157093bc8fca0ab81dccbf5260510de4eaf1cb69b5b99/importlib_resources-7.1.0.tar.gz", hash = "sha256:0722d4c6212489c530f2a145a34c0a7a3b4721bc96a15fada5930e2a0b760708", size = 44985, upload-time = "2026-04-12T16:36:09.232Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/8a/db/55a262f3606bebcae07cc14095338471ad7c0bbcaa37707e6f0ee49725b7/importlib_resources-7.1.0-py3-none-any.whl", hash = "sha256:1bd7b48b4088eddb2cd16382150bb515af0bd2c70128194392725f82ad2c96a1", size = 37232, upload-time = "2026-04-12T16:36:08.219Z" }, -] - [[package]] name = "iniconfig" version = "2.3.0" @@ -1860,27 +1706,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/e7/e7/80988e32bf6f73919a113473a604f5a8f09094de312b9d52b79c2df7612b/jupyter_core-5.9.1-py3-none-any.whl", hash = "sha256:ebf87fdc6073d142e114c72c9e29a9d7ca03fad818c5d300ce2adc1fb0743407", size = 29032, upload-time = "2025-10-16T19:19:16.783Z" }, ] -[[package]] -name = "kubernetes" -version = "36.0.3" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "aiohttp" }, - { name = "certifi" }, - { name = "durationpy" }, - { name = "python-dateutil" }, - { name = "pyyaml" }, - { name = "requests" }, - { name = "requests-oauthlib" }, - { name = "six" }, - { name = "urllib3" }, - { name = "websocket-client" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/ca/57/b07b96353f902aa1bdbe00e878e3a12a137977d03a962479785576aa8ec9/kubernetes-36.0.3.tar.gz", hash = "sha256:36993ed25ce59b789c9341473a228fcf268504a2fec7c2b2b1531d73072e5ce7", size = 2337528, upload-time = "2026-07-13T20:38:12.128Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/5b/30/a96d47df739689ac0001ade0afefc16e3b477fc2fb426b568515fdc8afce/kubernetes-36.0.3-py2.py3-none-any.whl", hash = "sha256:8fde9241c4b298e6374a069dcf728359b4e72c2fb29489a975ba4e1c047cf10f", size = 4618066, upload-time = "2026-07-13T20:38:10.172Z" }, -] - [[package]] name = "langchain" version = "1.4.0" @@ -2280,6 +2105,21 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/2c/19/04f9b178c2d8a15b076c8b5140708fa6ffc5601fb6f1e975537072df5b2a/mergedeep-1.3.4-py3-none-any.whl", hash = "sha256:70775750742b25c0d8f36c55aed03d24c3384d17c951b3175d898bd778ef0307", size = 6354, upload-time = "2021-02-05T18:55:29.583Z" }, ] +[[package]] +name = "milvus-lite" +version = "3.2.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "faiss-cpu" }, + { name = "grpcio" }, + { name = "numpy" }, + { name = "pyarrow" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/16/48/3826b9b18621aa38548ed59749a3a2df0578a1262f8aeeed5cdeb7a9fb94/milvus_lite-3.2.1.tar.gz", hash = "sha256:4d988fe0a6bbdfc708046014ad69f4b140c43567ecafd8b6df300ad7d37210f3", size = 722376, upload-time = "2026-08-25T03:14:40.772Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f8/ac/7534ce7526191f776e0d8f28c32ea69f0ae9516b1510a5653eeb5038e1b9/milvus_lite-3.2.1-py3-none-any.whl", hash = "sha256:0742fb4c54858bcc6fc3c2142ec651e2b0f4ae30f19e21d4fbc6db65b9e16c7b", size = 269551, upload-time = "2026-08-25T03:14:37.101Z" }, +] + [[package]] name = "mkdocs" version = "1.6.1" @@ -2477,51 +2317,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/22/b7/a35232812a2ccfffcb7614ba96a91338551a660a0e9815cee668bf5743f0/mlx_whisper-0.4.3-py3-none-any.whl", hash = "sha256:6b82b6597a994643a3e5496c7bc229a672e5ca308458455bfe276e76ae024489", size = 890544, upload-time = "2025-08-29T14:56:13.815Z" }, ] -[[package]] -name = "mmh3" -version = "5.3.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/aa/69/d00269fee7a3102fcc0f04f0a312e41c6b237762bdcad4c19426f18e697c/mmh3-5.3.0.tar.gz", hash = "sha256:95832419b87b882bec9dcd7d041d74887ba7745b3659c14be1ae1db5cfa35cad", size = 33607, upload-time = "2026-08-26T04:58:20.042Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/ef/97/28c3905a7e27e100ef50322798659e0af514eff600aabffbab00e3cd27ad/mmh3-5.3.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:5e7373e6834e4bdf2c24dbb1a0c6dd834bb5a189efb65723ebb58a8f3e76204b", size = 55484, upload-time = "2026-08-26T04:56:12.197Z" }, - { url = "https://files.pythonhosted.org/packages/44/13/629beb4d3e92ffcf1486c81cbb0605d9e6685b2721eb6753946a9d217359/mmh3-5.3.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:430ed4de594d0084d9b7956b05075a9054d290a3a0d7b370553a9096a4fd429f", size = 40091, upload-time = "2026-08-26T04:56:13.289Z" }, - { url = "https://files.pythonhosted.org/packages/c7/64/0a5832cd45207c507ee83bb7286dfeccf51c438aa8b6217f44f286f354f4/mmh3-5.3.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:bee76669a5b588cd806aa619ea9eb8f0c8a00e6991001d830e07cc69258962a9", size = 39694, upload-time = "2026-08-26T04:56:14.244Z" }, - { url = "https://files.pythonhosted.org/packages/13/ac/e9a157fbeebf44da5e39e49ef3901cbd68651d966658b58303074f349422/mmh3-5.3.0-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:c6fbc4e3017fb99e639abdb58a6a31e14bcbd270562805a0b80a102f8a4f3024", size = 97220, upload-time = "2026-08-26T04:56:15.312Z" }, - { url = "https://files.pythonhosted.org/packages/af/a6/5012e363699c598166fb955f7267975284dd51eadc0383771a1a593d4ac2/mmh3-5.3.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:98db40c6ef8bbeb028e0424736a6bef3b1d8d0a02399236eb00db0dd0b7ca957", size = 103238, upload-time = "2026-08-26T04:56:16.425Z" }, - { url = "https://files.pythonhosted.org/packages/47/53/5710edf5edd969bbc42985577ddc73758a1c90a1613ffc4356087b5e330f/mmh3-5.3.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fa216ac716e7c99e4dc4b039c6219a31cd381cc0588ca45cf66f36011613f3ed", size = 106138, upload-time = "2026-08-26T04:56:17.54Z" }, - { url = "https://files.pythonhosted.org/packages/c5/dc/e7737d142ecb5847aad04fc92d63bd455587f5dc33ebbfddeb7936f98713/mmh3-5.3.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ef9fe783b932927da8070f5b2913ce412e42c80bf17fd523042325ee3a44f756", size = 112973, upload-time = "2026-08-26T04:56:18.644Z" }, - { url = "https://files.pythonhosted.org/packages/cd/04/34e1242a23e14c78a39dd76ffdf860e76d01c917e4103b9729c27efc22f7/mmh3-5.3.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:1d5de36640293374673ec6813b7a23d8a9621bbd87f079c6ec4c5e8585cb1f64", size = 120621, upload-time = "2026-08-26T04:56:19.837Z" }, - { url = "https://files.pythonhosted.org/packages/f3/6b/665f1bd97666095f8a92b4191e22c21038c3e5f6bccad2a09218962e0541/mmh3-5.3.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:0a6bd95c410ec9500d9515a4fe522e24452f71df38de47395f99aebc085a5d5a", size = 99020, upload-time = "2026-08-26T04:56:21.07Z" }, - { url = "https://files.pythonhosted.org/packages/e7/a8/0ce084753cc8a82f8cbadcb6723f2e17c341b17683eafc49729a55db4930/mmh3-5.3.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:f0c7a36ccb66bfc8fcfa7a9722614b959231e325f0e08862c6ea70a7283a6520", size = 98497, upload-time = "2026-08-26T04:56:22.252Z" }, - { url = "https://files.pythonhosted.org/packages/01/1e/501564ca0687e4138c01c12d5b1bc049511753f5a5db2538ad662bf16d97/mmh3-5.3.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:767c6c0cf3f67c3e4e246ae7e1cc9ce7755f174f994aa3111c8357f16a587161", size = 106369, upload-time = "2026-08-26T04:56:23.375Z" }, - { url = "https://files.pythonhosted.org/packages/f7/af/7e072e63c10f81b0d4f2abff29363029144c62006baa1fbe437302d24f05/mmh3-5.3.0-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:408d37be08e12a154b482dcb300781d3dd154abab8e002ab32ecde6aa6a325bc", size = 109882, upload-time = "2026-08-26T04:56:24.495Z" }, - { url = "https://files.pythonhosted.org/packages/a6/89/90e3f0f59eb13b362cb0c091a1ce43b94325fae8c6c20a991dc9ee6bd0b7/mmh3-5.3.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:a2b77b3e6a9d817822407b32c514205b44ceeb8ab197bee09de19e5f1e04ce18", size = 97421, upload-time = "2026-08-26T04:56:25.675Z" }, - { url = "https://files.pythonhosted.org/packages/f4/be/eaa4b95e3fdb2617ec1c1502ab15f5941408c02a3c80a5df63ab4c412abd/mmh3-5.3.0-cp312-cp312-win32.whl", hash = "sha256:d4cc2cb5f117da6460c14c65cbc0d1cf0976af3e56de6cd627cc36019f323e15", size = 40467, upload-time = "2026-08-26T04:56:26.804Z" }, - { url = "https://files.pythonhosted.org/packages/39/86/695592b763d2c0a5739ff50e62be102a1790bd5817a5f7f9ed2ffcbe198f/mmh3-5.3.0-cp312-cp312-win_amd64.whl", hash = "sha256:ad5e6b56000d4b1b82a380c664982371939dc8f728fcbb73d017edc035247dcb", size = 41864, upload-time = "2026-08-26T04:56:27.779Z" }, - { url = "https://files.pythonhosted.org/packages/d1/ea/b534107b184454994865f5c1695df2f666672f16636b814da182b3ab0f6f/mmh3-5.3.0-cp312-cp312-win_arm64.whl", hash = "sha256:4b660543eee66d5f07408fc1cdd0d017416f0f5cd1725def314cca3f67b0cac5", size = 39206, upload-time = "2026-08-26T04:56:28.789Z" }, - { url = "https://files.pythonhosted.org/packages/8b/fa/e07f6b9e2fba550fe539b6c66ee7fc28e44f5bd445a7203ae4c169d4aa72/mmh3-5.3.0-cp313-cp313-android_24_arm64_v8a.whl", hash = "sha256:6576400e7a748ec5c7ea72f38d626939876dd1756f4a0ccf552b8646dcf6f3e9", size = 41040, upload-time = "2026-08-26T04:56:29.766Z" }, - { url = "https://files.pythonhosted.org/packages/62/9c/cb0f23e71bddcc519331c9787ec029e0d2fef64ed1bc490ad84b00a43950/mmh3-5.3.0-cp313-cp313-android_24_x86_64.whl", hash = "sha256:2b4cd2fcf1b517872530d9ef1a2de2ef9b86e5a0f8927539ea0b68337618244e", size = 42153, upload-time = "2026-08-26T04:56:30.954Z" }, - { url = "https://files.pythonhosted.org/packages/27/1c/99f2ff480922046a496c2e53728a13a467b68f01a0f48370577c0825a763/mmh3-5.3.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:f401a82d80c53d88605b82a80623edd95d922732d2c513c1c5f8e4b5e10c2913", size = 38796, upload-time = "2026-08-26T04:56:32.207Z" }, - { url = "https://files.pythonhosted.org/packages/ad/05/e137452583b33a56d053ad48643a7c56e4cf466efc8a145da6a736913ae3/mmh3-5.3.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:86238bb78ff65c9fc1e6b371b78f271e23c5d61898222c201122209dc8eadc76", size = 39474, upload-time = "2026-08-26T04:56:33.196Z" }, - { url = "https://files.pythonhosted.org/packages/74/e8/a9031fe6ed0eb06ab1c98eb76182eb01dc484dc46873a8e5ef3097d9bdf4/mmh3-5.3.0-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:222ea0a485e23bdcb29e28d15b8b01ebe34e8720bad4b5f92b645ed86e3fc715", size = 39983, upload-time = "2026-08-26T04:56:34.195Z" }, - { url = "https://files.pythonhosted.org/packages/f9/ae/11459b39af5341de2a048998a717f57b3a1c4e6a9edf8fe09e314b2c263c/mmh3-5.3.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:d95ee6696aa5b7283f4a27b67eb7db1c4fb5bb7a9117205d29ebaaa7f6294d7b", size = 55464, upload-time = "2026-08-26T04:56:35.23Z" }, - { url = "https://files.pythonhosted.org/packages/49/b7/841b580415a614a3ad836db1cb8d57c425bbeb717c80263a9d979b1a4eac/mmh3-5.3.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e2f439ffd4fd7d64b77f6a287d4605700bad26fe12bb1b63b4ee45211344e2fc", size = 40084, upload-time = "2026-08-26T04:56:36.44Z" }, - { url = "https://files.pythonhosted.org/packages/02/f6/f5cb0e2f7bb7df876847dab63ee984c3f6173569d5b892edc04bc797c1b7/mmh3-5.3.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:7f9cb34c661454f73432112a81ac522ebe69500feeb8d77f744f6bd3e8b2f2ba", size = 39693, upload-time = "2026-08-26T04:56:37.428Z" }, - { url = "https://files.pythonhosted.org/packages/b5/61/9506d0b30d7388846cc3b884ec63b8d67f3ad2d521e61775315111e70ec3/mmh3-5.3.0-cp313-cp313-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:b6b7397804e9299bd0c01ea426245fa3d730d3e9c31f583f51aa87bed399c481", size = 97247, upload-time = "2026-08-26T04:56:38.439Z" }, - { url = "https://files.pythonhosted.org/packages/da/6f/c6a4acf4715fede1e0acb17d6080ebe9b88290d113bbf9513a8728c65b4d/mmh3-5.3.0-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:4a19b00097fcc8e3008bb006cd6bfaf0544e9fa2abc4cc77fbad57971a37dcc0", size = 103256, upload-time = "2026-08-26T04:56:39.52Z" }, - { url = "https://files.pythonhosted.org/packages/a3/2a/f4fe4ebf2e44f49953ed1ee6b90d329d6843ebb949e60e881dbfde84e17d/mmh3-5.3.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:83d93abf6a68d54b4e2c4c041ffcffeb94b1c9ab3171443fda3f5f19024be517", size = 106172, upload-time = "2026-08-26T04:56:40.67Z" }, - { url = "https://files.pythonhosted.org/packages/1b/61/23168dc6fa92e1d9b57905b89c694a521a00570f0fe325bc3f6422fe6119/mmh3-5.3.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:598350a6adefb5799c800fd00bbeacbc115ee560e2fd7b35f703608c1037a2ed", size = 113020, upload-time = "2026-08-26T04:56:42.035Z" }, - { url = "https://files.pythonhosted.org/packages/6e/f6/68dbb5461727bb14fbca2342a7bba4521400d679d64e00dd98a992eb0be4/mmh3-5.3.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5c0772d6bba19a5601d24b3c6ce6627484fd5a3fd1d402913e1578b1447d51a0", size = 120648, upload-time = "2026-08-26T04:56:43.339Z" }, - { url = "https://files.pythonhosted.org/packages/92/18/a879ce4c26e8c1741b575e6c854b2323fd7e116000965c8904428e66fba6/mmh3-5.3.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:00105934e7d52f80b4364282918c37e2cc3cf9868ef4052016cbc39d8711c3f4", size = 99034, upload-time = "2026-08-26T04:56:44.771Z" }, - { url = "https://files.pythonhosted.org/packages/1b/28/b0548d78133f8e79bb16d6487f2df504ff4e16ea1330954f26be83b645ca/mmh3-5.3.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:99a5b0a908beb01b0e134b7b085d0ea6bfb7ed28ba3ed0737365aa2ce9bda0e4", size = 98587, upload-time = "2026-08-26T04:56:46.089Z" }, - { url = "https://files.pythonhosted.org/packages/97/10/abfa952c6a443d07efb286b53b439d4418f7cfbac49fca8974cd78c17427/mmh3-5.3.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:da4ad7a0d4c589069c46101dcb55ee304616293bcf614f4c445b3ecc961fa836", size = 106407, upload-time = "2026-08-26T04:56:47.16Z" }, - { url = "https://files.pythonhosted.org/packages/a6/61/3b974a1cfe683c6cb01d2d3f2d5eddfca7e0a175d976d2307269143f4e0e/mmh3-5.3.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:8962a67c314f1da82957aee5b940698aaffff13e41b3298baa59d30cbddb23e2", size = 109922, upload-time = "2026-08-26T04:56:48.451Z" }, - { url = "https://files.pythonhosted.org/packages/0e/d2/ac7e18d6fcee3b26e06b3d33ecab4459b7aa6aff9859b13eb13e353a69ab/mmh3-5.3.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:f61f2850b318c043961662f6cdd08e69b05f1d25d0e321782a3995d39f811548", size = 97456, upload-time = "2026-08-26T04:56:49.559Z" }, - { url = "https://files.pythonhosted.org/packages/ee/80/7911629b930d46ef8cb3165bce44f021d31361a2d7c56e35efe7b0d90493/mmh3-5.3.0-cp313-cp313-win32.whl", hash = "sha256:d7eec1b09bde3a9b6e2102717a587b9c9a96c360a1ef478b5668414619cac606", size = 40458, upload-time = "2026-08-26T04:56:50.785Z" }, - { url = "https://files.pythonhosted.org/packages/45/c7/138d77c740f9d33ceffa968786fe23abe24ce442c210772b5b7cb6e0c198/mmh3-5.3.0-cp313-cp313-win_amd64.whl", hash = "sha256:b1829bfe98d1f6e7bd79646b78e73dcef92c5aa32aaa622b9e07bf39df98c9b5", size = 41879, upload-time = "2026-08-26T04:56:51.785Z" }, - { url = "https://files.pythonhosted.org/packages/8b/e1/4312e5c18c8ef4060c9b50f1a645c9af7c3b79fad012c18fbe64a1b17103/mmh3-5.3.0-cp313-cp313-win_arm64.whl", hash = "sha256:3fb0d4918b7d827ac804069849fde03d516628cbbf7bffe0b957ba6f1440cca6", size = 39203, upload-time = "2026-08-26T04:56:52.746Z" }, -] - [[package]] name = "more-itertools" version = "11.1.0" @@ -2891,15 +2686,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/a8/64/3708a90d1ebe202ffdeb7185f878a3c84d15c2b2c31858da2ce0583e2def/nvidia_nvtx-13.0.85-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:cb7780edb6b14107373c835bf8b72e7a178bac7367e23da7acb108f973f157a6", size = 148878, upload-time = "2025-09-04T08:28:53.627Z" }, ] -[[package]] -name = "oauthlib" -version = "3.3.1" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/0b/5f/19930f824ffeb0ad4372da4812c50edbd1434f678c90c2733e1188edfc63/oauthlib-3.3.1.tar.gz", hash = "sha256:0f0f8aa759826a193cf66c12ea1af1637f87b9b4622d46e866952bb022e538c9", size = 185918, upload-time = "2025-06-19T22:48:08.269Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/be/9c/92789c596b8df838baa98fa71844d84283302f7604ed565dafe5a6b5041a/oauthlib-3.3.1-py3-none-any.whl", hash = "sha256:88119c938d2b8fb88561af5f6ee0eec8cc8d552b7bb1f712743136eb7523b7a1", size = 160065, upload-time = "2025-06-19T22:48:06.508Z" }, -] - [[package]] name = "odfdo" version = "3.24.7" @@ -2935,31 +2721,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/a4/0e/152509871bf30df6fc38569f52a2db9b55dd41aae957adae50a053ac7778/omegaconf-2.3.1-py3-none-any.whl", hash = "sha256:3d701d14e9a8828f1edd28bb70b725908b34277cdd72cf7d6a83f94dadc6b6a0", size = 79502, upload-time = "2026-06-11T05:05:09.954Z" }, ] -[[package]] -name = "onnxruntime" -version = "1.29.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "flatbuffers" }, - { name = "numpy" }, - { name = "packaging" }, - { name = "protobuf" }, -] -wheels = [ - { url = "https://files.pythonhosted.org/packages/d4/80/381c1e9efed9cc32d00aa7cab0547dc84116cec906c3ffe3613686d6963a/onnxruntime-1.29.0-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:3a3814c041251d6a77fdf513fb282056538ee826d2f1178a0df3c549d3fff6ba", size = 21430049, upload-time = "2026-08-17T22:53:48.286Z" }, - { url = "https://files.pythonhosted.org/packages/30/12/4be0e345d38fe707a701ca07e8f63c05b152a2e6285d1e43a7faf63fedd2/onnxruntime-1.29.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:d2fb19e848f7c33ed8d3182b52504aaa11c5e8da438bbb47296f85b133cbcf6b", size = 20816870, upload-time = "2026-08-17T22:53:51.169Z" }, - { url = "https://files.pythonhosted.org/packages/96/eb/e6968f5e41aac3125f2ff5708855f09cb0b70d85ed3115b625b0b58305ba/onnxruntime-1.29.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:2b80d8c7ec2cc7438e4da3760b88c24568cba72c9ace96d668800a6c79419acb", size = 23136745, upload-time = "2026-08-17T22:53:53.92Z" }, - { url = "https://files.pythonhosted.org/packages/b4/80/5b28f1f1111210fc4a336ddbc6950f468ebf9a6a265420568f4f43fa33ce/onnxruntime-1.29.0-cp312-cp312-win_amd64.whl", hash = "sha256:4acf2b4948b7ede87221ca6332344b8facdc8059d6ac751a7d367d04532b02dd", size = 14001407, upload-time = "2026-08-17T22:53:56.486Z" }, - { url = "https://files.pythonhosted.org/packages/6f/d6/6883f89ea4b044e6e8447ebfaf9bcecdf457b7d80a683635e130b25498e0/onnxruntime-1.29.0-cp312-cp312-win_arm64.whl", hash = "sha256:dc61a79cb39afd66ab3f01fd2c23591a7f01de89c1668e1fb6315067fc279164", size = 13746981, upload-time = "2026-08-17T22:53:58.977Z" }, - { url = "https://files.pythonhosted.org/packages/41/f8/d375facf60edaf41f5732f9f689c98a800fcc52df5cf6ddfb406703eb5a1/onnxruntime-1.29.0-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:be0f8ed688cfb1d4d5765a137193b7bfab0c8ea214eed99260b380bb525a3a7f", size = 21429708, upload-time = "2026-08-17T22:54:01.44Z" }, - { url = "https://files.pythonhosted.org/packages/c9/17/b9ad04051a8c4f504852ce0e8e10f9a6b2f1a331eedcdcc503df776dd0ea/onnxruntime-1.29.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:d67673c5367727860922c5262d724472f1b5539fb7ccf4c81a638f9b71719803", size = 20816263, upload-time = "2026-08-17T22:54:04.088Z" }, - { url = "https://files.pythonhosted.org/packages/83/2c/d8eb945d2a372149df9705a8d5c8d7c6c46c987c5446dbcea9e1ea7f6556/onnxruntime-1.29.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:e2128f31f449e922c62dbe5d8b6b7b079f0bcaf2d56a102fa203cb6e5bb5ab19", size = 23136817, upload-time = "2026-08-17T22:54:06.714Z" }, - { url = "https://files.pythonhosted.org/packages/e1/3b/66b424c63fa92dfaa48d1719efaae66fc8c256b9426a832eda51d8dfe1e9/onnxruntime-1.29.0-cp313-cp313-win_amd64.whl", hash = "sha256:2945e1f82f81f27e88decea88c7861f45baea23818950d467bf3909aa303119e", size = 14001310, upload-time = "2026-08-17T22:54:09.13Z" }, - { url = "https://files.pythonhosted.org/packages/83/22/d6a700e3a6322fa3d56fbe7cee9ffc53f35e77ffcd6b7e97f4b7722a27ab/onnxruntime-1.29.0-cp313-cp313-win_arm64.whl", hash = "sha256:4b940b0d777590c7e20bf298f5c16af1ea6ad1b400a1c822a6be192f64f4d954", size = 13747112, upload-time = "2026-08-17T22:54:11.608Z" }, - { url = "https://files.pythonhosted.org/packages/4a/89/c4af146de3d60a32c89fea48d5d34bfd044faaf8957270043a03bd1b462b/onnxruntime-1.29.0-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:533f8370ce124304e5cb08ab961836cf755631e3dd77adc5f3bbdab70c2b7d99", size = 20826136, upload-time = "2026-08-17T22:54:14.315Z" }, - { url = "https://files.pythonhosted.org/packages/9d/f2/e6bbacd11dfe8d070613261a758795ea128b9fc9bea391a2a7da2e4c7a08/onnxruntime-1.29.0-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:c1ad3f437153fe77f9d01a08fbaac0beb030e09b8a80ace1603bcf69b6c95481", size = 23138951, upload-time = "2026-08-17T22:54:17.154Z" }, -] - [[package]] name = "openai" version = "2.53.0" @@ -3025,87 +2786,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/c0/da/977ded879c29cbd04de313843e76868e6e13408a94ed6b987245dc7c8506/openpyxl-3.1.5-py2.py3-none-any.whl", hash = "sha256:5282c12b107bffeef825f4617dc029afaf41d0ea60823bbb665ef3079dc79de2", size = 250910, upload-time = "2024-06-28T14:03:41.161Z" }, ] -[[package]] -name = "opentelemetry-api" -version = "1.44.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "typing-extensions" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/ee/8b/aa9e2d8b8dfa7c946f7dec5d1f8f6ba8eca062f43509a06bdb5ce93d26c0/opentelemetry_api-1.44.0.tar.gz", hash = "sha256:67647e5e9566edcf421166fdf022b3537f818635daa852b289e34604dc6fb33a", size = 72406, upload-time = "2026-07-16T15:25:32.678Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/ca/6f/a04e900f465ff3221ccc395522503e2d10e79fa21f2723c8e177aae1e0d1/opentelemetry_api-1.44.0-py3-none-any.whl", hash = "sha256:94b98c893a91b88657eaac1e3ba89618cdb85be6918196705354f34728b2cdef", size = 60018, upload-time = "2026-07-16T15:25:11.657Z" }, -] - -[[package]] -name = "opentelemetry-exporter-otlp-proto-common" -version = "1.44.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "opentelemetry-proto" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/61/09/4d717852c1cf3f854b76c7110a5d00883bc3c99288b9b0dbcbeb9e306eb6/opentelemetry_exporter_otlp_proto_common-1.44.0.tar.gz", hash = "sha256:dc87a5a5bc58f149a56d1547e4691588fa12994cdc3bc039a694ccb3375862ac", size = 20202, upload-time = "2026-07-16T15:25:37.658Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/5e/71/65fd9d54c10b860f87c045ccee1264cab7011268895d3528818a29c1172a/opentelemetry_exporter_otlp_proto_common-1.44.0-py3-none-any.whl", hash = "sha256:9a9fe61bba73d802904bc989f1d6b4a7b1ee40f06c40e98d6f85af65aaebb694", size = 17045, upload-time = "2026-07-16T15:25:18.201Z" }, -] - -[[package]] -name = "opentelemetry-exporter-otlp-proto-grpc" -version = "1.44.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "googleapis-common-protos" }, - { name = "grpcio" }, - { name = "opentelemetry-api" }, - { name = "opentelemetry-exporter-otlp-proto-common" }, - { name = "opentelemetry-proto" }, - { name = "opentelemetry-sdk" }, - { name = "typing-extensions" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/1f/47/80d9e9d468dc5de3af5096f5ccdb065fa4dd1470f74495cc53e59e397f47/opentelemetry_exporter_otlp_proto_grpc-1.44.0.tar.gz", hash = "sha256:40d1ae9e03fcc36de3cbac610cc99f35894938bff9cfd90fc4ec68bd85448463", size = 27225, upload-time = "2026-07-16T15:25:38.308Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/54/29/6ae42ba32b153ae0a44ae125f0caff2188bbe62d99c82d1768da30864e72/opentelemetry_exporter_otlp_proto_grpc-1.44.0-py3-none-any.whl", hash = "sha256:6a1a645ea182a2f59440c51fa8301d309f3324a8f9d65f8395584b064b67ee4e", size = 19624, upload-time = "2026-07-16T15:25:19.096Z" }, -] - -[[package]] -name = "opentelemetry-proto" -version = "1.44.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "protobuf" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/64/01/40ac4ae9a149263cc52c2cee200ddd80cb6d8db1a4610abf8eabce0fe771/opentelemetry_proto-1.44.0.tar.gz", hash = "sha256:c547a79c2f8c0c515d31509154682e5921c7cfd5ca67b70e1f9266e2c3e103f3", size = 46488, upload-time = "2026-07-16T15:25:45.34Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/d1/7c/8be563d68e93bbefa5c8affb82ddcff91b3ad858ce49957ba7b16fd3e0ab/opentelemetry_proto-1.44.0-py3-none-any.whl", hash = "sha256:898b155a0e1557afd867478fb6158e8122a46329ca0bb8dc53cc55e98f017f56", size = 72483, upload-time = "2026-07-16T15:25:28.429Z" }, -] - -[[package]] -name = "opentelemetry-sdk" -version = "1.44.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "opentelemetry-api" }, - { name = "opentelemetry-semantic-conventions" }, - { name = "typing-extensions" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/5d/77/a6592cbc7c8d9bcc9d6757a9df45e04a7c585e3e6e7a13456da522b21109/opentelemetry_sdk-1.44.0.tar.gz", hash = "sha256:cebe7f65dc12f26ead75c6064de12fd2a9052e5060c0272d402cfa203aae123b", size = 208624, upload-time = "2026-07-16T15:25:46.078Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/e7/23/ff077e61886ee020a17ce9c8b6fa11c601c8d8345b09ea24f605445df62a/opentelemetry_sdk-1.44.0-py3-none-any.whl", hash = "sha256:df081c4c6bcfdb1211e3e86140376792643128a25f8d72d1d27675936e7e96ad", size = 137221, upload-time = "2026-07-16T15:25:29.534Z" }, -] - -[[package]] -name = "opentelemetry-semantic-conventions" -version = "0.65b0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "opentelemetry-api" }, - { name = "typing-extensions" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/8f/73/0cbdebcb4cf545fdd328da14f5137e37d0770c3f26185e478b0d15d94f50/opentelemetry_semantic_conventions-0.65b0.tar.gz", hash = "sha256:f9b2b81e9d5b64f11bc952075e7e9c7fb0aab075c7fd1c46d597f1b919852d60", size = 148774, upload-time = "2026-07-16T15:25:46.902Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/a6/0e/49df70d9b81fb5cbae4bbf2a49d865b09bcbcbc4eb53f5851b1027738d78/opentelemetry_semantic_conventions-0.65b0-py3-none-any.whl", hash = "sha256:1cacde7b0ad306f84c5ef08c3dbe1bbaf20165bba6f8bff43b670e555a086bcb", size = 204645, upload-time = "2026-07-16T15:25:30.688Z" }, -] - [[package]] name = "orjson" version = "3.12.0" @@ -3162,15 +2842,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/7a/d4/0cfeea1e960d550a131001a7f38a5132c7ae3ebde4c82af1f364ccc5d904/ormsgpack-1.12.2-cp313-cp313-win_arm64.whl", hash = "sha256:baca4b6773d20a82e36d6fd25f341064244f9f86a13dead95dd7d7f996f51709", size = 111577, upload-time = "2026-01-18T20:55:43.605Z" }, ] -[[package]] -name = "overrides" -version = "7.7.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/36/86/b585f53236dec60aba864e050778b25045f857e17f6e5ea0ae95fe80edd2/overrides-7.7.0.tar.gz", hash = "sha256:55158fa3d93b98cc75299b1e67078ad9003ca27945c76162c1c0766d6f91820a", size = 22812, upload-time = "2024-01-27T21:01:33.423Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/2c/ab/fc8290c6a4c722e5514d80f62b2dc4c4df1a68a41d1364e625c35990fcf3/overrides-7.7.0-py3-none-any.whl", hash = "sha256:c7ed9d062f78b8e4c1a7b70bd8796b35ead4d9f510227ef9c5dc7626c60d7e49", size = 17832, upload-time = "2024-01-27T21:01:31.393Z" }, -] - [[package]] name = "packaging" version = "26.3" @@ -3483,63 +3154,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/31/24/97e8bd98f1e3b07e2ba08bcdff690674fbe16d69a7d2712cc3884665e615/pyarrow-25.0.1-cp313-cp313-win_amd64.whl", hash = "sha256:31e49a7888fcdf3a835da33ae777f6bb9a866334e5a789282fc26dcf426f7f15", size = 27870159, upload-time = "2026-08-10T12:39:26.161Z" }, ] -[[package]] -name = "pybase64" -version = "1.5.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/d7/65/c513eab7211590250f729a06aacc0bc95eaf760b9235666e933d200105d0/pybase64-1.5.0.tar.gz", hash = "sha256:545ab2a433769e3b8e1ce2b4f7b07218bbde202f4954fbfe52948b2522120727", size = 149492, upload-time = "2026-08-08T15:42:00.205Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/84/f4/dba60f937caf26a6e2be6a138f5422da9f4ec988db49bd4e329bcb435cd2/pybase64-1.5.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:9732eba18ba7fe44c1b2827bfaadf381fed3789bd7e20c990e6c8d1ceba0179b", size = 47155, upload-time = "2026-08-08T15:38:16.705Z" }, - { url = "https://files.pythonhosted.org/packages/b6/61/302d65a981c9baf156e4becbbbe49f38de72906c430ab373d6d1ca0d4258/pybase64-1.5.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:d1149b7360dd99ef1ad10618df2a4f54a00385bc8d2c1aa244c0301a548ac415", size = 40490, upload-time = "2026-08-08T15:38:17.95Z" }, - { url = "https://files.pythonhosted.org/packages/1d/66/9f1be6a4db86577eebf3106496a2a791b37e5fb74695d4c8eeedbd04490a/pybase64-1.5.0-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:80b171f1546935be4dae1e01bfd8630d2712271e067858b7135726e7d9bc7cce", size = 91058, upload-time = "2026-08-08T15:38:18.983Z" }, - { url = "https://files.pythonhosted.org/packages/af/36/4e44a0688efe26434bf378b4565b01ac94f81422e8a5746291a03472cd56/pybase64-1.5.0-cp312-cp312-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:1a2b9cf39b4d30f600df8c56cccbc03adfc6e1ae8c04cd6b181105a432d4a515", size = 94681, upload-time = "2026-08-08T15:38:20.59Z" }, - { url = "https://files.pythonhosted.org/packages/aa/d1/fc02005906fd48081b7b8f077cd422a55399fa351c2a6d3e5fed951794ce/pybase64-1.5.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:865b7db127a95e33640ebcdb4bb3aad165d4873ee7c1008949129f3c4f900dd8", size = 84634, upload-time = "2026-08-08T15:38:21.711Z" }, - { url = "https://files.pythonhosted.org/packages/c0/c6/5bb0f21a9f4d231339a42f16ebabc7c6d9a7d619e756327b15a474650ece/pybase64-1.5.0-cp312-cp312-manylinux2014_armv7l.manylinux_2_17_armv7l.whl", hash = "sha256:3344ce336d9d8292125369c1475d1663e7e1a06894e8e5150307e11f782c6afd", size = 80455, upload-time = "2026-08-08T15:38:23.05Z" }, - { url = "https://files.pythonhosted.org/packages/b8/04/0ba9a1f2ea39baf081dd44d22d710d9b050ce15991d641982f1814508484/pybase64-1.5.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:1aaae81669bf18b5a35dcb43dbb200f52b13f847a56bed7a2e82f31cc6f9f74d", size = 82304, upload-time = "2026-08-08T15:38:24.156Z" }, - { url = "https://files.pythonhosted.org/packages/c1/9e/6b380ff964dd77b79cc1ce565b73780345132e0e181d315f31a2263c5e1f/pybase64-1.5.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:fb5dc922ce3cb4211caa7e29e6daee98f319e59f297a904acd74f2fdd0674356", size = 81259, upload-time = "2026-08-08T15:38:25.327Z" }, - { url = "https://files.pythonhosted.org/packages/b9/93/dd7fd7f8ed228f7735ec59a9f85f3c683cef371a76b29520344655bf7c97/pybase64-1.5.0-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:356e7bd1453551c06231df8411bfbaed9998fbcba2da723d84fb270ff1f977a7", size = 78360, upload-time = "2026-08-08T15:38:26.678Z" }, - { url = "https://files.pythonhosted.org/packages/d8/99/b5e9e7d4b5e49d7a984c4a26b48bdf988ec62c2778df80144af1a39bd4b1/pybase64-1.5.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:11dfa286f6c5fe6795430bf08fc44b64c98e208558215b0590c9f28fd99a92e3", size = 82358, upload-time = "2026-08-08T15:38:27.856Z" }, - { url = "https://files.pythonhosted.org/packages/67/fa/19d11ee70fbdb10e574a39ad7fc7adc06e5635a2b2ac291a6554c7c651ae/pybase64-1.5.0-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:6be40c3311eabe8a816e00041844f9b249828015dc98be8a48a7c3275954ee76", size = 76384, upload-time = "2026-08-08T15:38:29.169Z" }, - { url = "https://files.pythonhosted.org/packages/71/32/a83622dfa3162dd6fcd019dd8fbb766f0ce064fe67b3d3d2759881dbac4e/pybase64-1.5.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:4e8b163c8d2d2a5f414f2c31cdd91024e0c91c72e735a9a564a62460ac838acb", size = 91407, upload-time = "2026-08-08T15:38:30.306Z" }, - { url = "https://files.pythonhosted.org/packages/6d/b5/1707748813784af0b1340f6c6525887f1ecb393c3f88070a2bb2d86bd94e/pybase64-1.5.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:0030a64fe91791e5e553edaff3a55d319cd07fb5e097b09c5f7f45e4905c40cb", size = 79687, upload-time = "2026-08-08T15:38:31.771Z" }, - { url = "https://files.pythonhosted.org/packages/e9/ee/8101e43b5cc070c0adf298f87500154c13b9097d4456a2c1aadd71339329/pybase64-1.5.0-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:28d5db510433bb1544dc128c4e7ebd85ae57cec2a4608edd1f7ca4fed3e53b3d", size = 77913, upload-time = "2026-08-08T15:38:32.898Z" }, - { url = "https://files.pythonhosted.org/packages/70/8c/43b2281077ca9a531bd896b7a9fe871d091d80d172d68e439c7aa6337033/pybase64-1.5.0-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:26422429a0bb2f15773dacc0fcb1bcddfce68c6b2d41fc14bc7fc17f8c529542", size = 79172, upload-time = "2026-08-08T15:38:33.974Z" }, - { url = "https://files.pythonhosted.org/packages/c2/1a/b536e571518eb2f4a2db1c6c7c5913af5780ff82c9eefb41f674fed71ceb/pybase64-1.5.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:ccbae849677648be456ea0de769a78e432d2d24f71cbdc739741e69f8160e0d7", size = 93636, upload-time = "2026-08-08T15:38:35.102Z" }, - { url = "https://files.pythonhosted.org/packages/54/c0/318f79b614fa03089bf4672194325dfa732790546530697b55a53612637b/pybase64-1.5.0-cp312-cp312-win32.whl", hash = "sha256:d691553d1a88ed87cf1837babec3663275b29de906b48433c15b298e262e5243", size = 42443, upload-time = "2026-08-08T15:38:36.217Z" }, - { url = "https://files.pythonhosted.org/packages/e0/80/eecc05ebac8d08a2bf855cc7bbe6a37d8c76cd19c6337c9b9fbe3225ee19/pybase64-1.5.0-cp312-cp312-win_amd64.whl", hash = "sha256:125945f5b3cde8b79a8f942cfdb0390f4388fb9458a41f5f2a93746e1ef3c546", size = 44565, upload-time = "2026-08-08T15:38:37.734Z" }, - { url = "https://files.pythonhosted.org/packages/b4/87/193dbb1eaf7751527a7e0510f5670efeed8642ec647b4c7177c384a6f7e9/pybase64-1.5.0-cp312-cp312-win_arm64.whl", hash = "sha256:c8b5f52776f0277e72a9c7e7944f682de2b3ee4655b7972a48c53f871963741a", size = 39918, upload-time = "2026-08-08T15:38:38.808Z" }, - { url = "https://files.pythonhosted.org/packages/2a/7c/b359e979a2b53f1aa9d8f2d9f90b29eda90d7dd126c2871dc49db4d6d8cf/pybase64-1.5.0-cp313-cp313-android_24_arm64_v8a.whl", hash = "sha256:2e79853f8e52ab0afa7b3ae445de23767b033fa0e58ad11099d3c6b79d012c7d", size = 44413, upload-time = "2026-08-08T15:38:39.883Z" }, - { url = "https://files.pythonhosted.org/packages/e5/9a/7412bd0e2c011069c754a1ac3e05ded9eab56614eea6d9251c74a434a472/pybase64-1.5.0-cp313-cp313-android_24_x86_64.whl", hash = "sha256:7661246f93c902bf147d5f7d72874902ef3e49a63ca3f0de333cb8e85765d2fd", size = 49859, upload-time = "2026-08-08T15:38:41.048Z" }, - { url = "https://files.pythonhosted.org/packages/a5/17/a1fc8e55551530876d3be31079b8701b7f5ac8451b63a08a19a4f9714454/pybase64-1.5.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:75d21d0a2cae0bb071c68686d77e5100be611ec4e80e0d97f8736c27da0ab197", size = 39681, upload-time = "2026-08-08T15:38:42.145Z" }, - { url = "https://files.pythonhosted.org/packages/9d/2c/b46f7e0c1ea482db0f8445d5bfad7e5a4f39d977868e10b4c3823e94fa20/pybase64-1.5.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:1bde27266ec4a56c38ef8e17998e430d30cc6310fde76332381bf5aaa81872ba", size = 40200, upload-time = "2026-08-08T15:38:43.354Z" }, - { url = "https://files.pythonhosted.org/packages/da/12/085dc70e757e6101c8f61239bae538640aac60ddfebb41e2534af3712e14/pybase64-1.5.0-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:220d8ab003d44144d80f8b776019adedc23fdc7bcb270396744b9805a8186d0e", size = 46726, upload-time = "2026-08-08T15:38:44.378Z" }, - { url = "https://files.pythonhosted.org/packages/60/7b/f3213973e61b8a8d1bb78203fe226e7f368698fb931249eacc09048d2141/pybase64-1.5.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:d42196f594460a083084d8e3c2f2554c958ebd8fe19bc30ef1b938197436e7d5", size = 47242, upload-time = "2026-08-08T15:38:45.478Z" }, - { url = "https://files.pythonhosted.org/packages/1b/64/e847e8710261596b3e7cf0935041a1c96a50fb2a7f3e9e09bc495510b25a/pybase64-1.5.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:aa56c549af248664ed7e1cc8ebc4dd7f1505b1444d8f3bf15b6a89b43dd4151f", size = 40628, upload-time = "2026-08-08T15:38:46.597Z" }, - { url = "https://files.pythonhosted.org/packages/e5/c3/8171fd18a57218c5e7c252f658709f9bd3d0eece9d4196542230103a53d6/pybase64-1.5.0-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:a1529b8e08a93dd9c00d1e3b3c2b627a9600d96c2f40143dc0b3a85f48fa85e5", size = 92183, upload-time = "2026-08-08T15:38:48.038Z" }, - { url = "https://files.pythonhosted.org/packages/23/84/b91aabd22a65a3679633855dde720dfb86571e15f88a9b1b295adda90e8c/pybase64-1.5.0-cp313-cp313-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:0be37689b624ae293394fc826c9a048c6118520d6a962de033ffb054564bf61f", size = 95718, upload-time = "2026-08-08T15:38:49.104Z" }, - { url = "https://files.pythonhosted.org/packages/67/cd/441fd3b9bc7a49846362fb52a0971cee6da4dca2eb8545100ec043b2a0da/pybase64-1.5.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:bf98b77c6cca5c5da30135b69b30668da07a32d41210c62121b34c84239d9d4a", size = 86068, upload-time = "2026-08-08T15:38:50.683Z" }, - { url = "https://files.pythonhosted.org/packages/2f/24/48cfe7e1b776c0af1ce5240f7e71383890cd361242e537b6c510804a68d2/pybase64-1.5.0-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.whl", hash = "sha256:0578c54f1ae89e6175eddb742dbaf2e95a060735ec11f4b661f762b635680cbd", size = 81077, upload-time = "2026-08-08T15:38:51.825Z" }, - { url = "https://files.pythonhosted.org/packages/db/38/5b47895e2f19f9775a3daaec98a652ba7c0ccfb480c223d981c2ec75c0ed/pybase64-1.5.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:ae78cdaec57f21e7f44cc5f9866d694cc072e1b1082286f30fd74e7545fa2916", size = 83387, upload-time = "2026-08-08T15:38:52.921Z" }, - { url = "https://files.pythonhosted.org/packages/74/2d/115526e63080e96ce039619a1a29a4fe49d138c5d7d525b6adbccf0c1c0f/pybase64-1.5.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:1f315f07b269f074995c445b65dfde62d12c0e889e9c3b0534befdb05866e880", size = 82460, upload-time = "2026-08-08T15:38:54.436Z" }, - { url = "https://files.pythonhosted.org/packages/53/b8/8970ecca7a5945f81d34f9a91d23169f7e62e2487ef3694e0004943e7243/pybase64-1.5.0-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:99570e43605b9c849ff1606e1691e503962250f80ec3e827249f7ad820e402d8", size = 79359, upload-time = "2026-08-08T15:38:55.69Z" }, - { url = "https://files.pythonhosted.org/packages/55/06/eea9cb5955430d5f789c18eab854284c66b1a024efae4928992d44bcde65/pybase64-1.5.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:e0143b3515b97bb3c4743fbdf10f53950c0bb1fe1a2db1054b422ba370594333", size = 83768, upload-time = "2026-08-08T15:38:56.793Z" }, - { url = "https://files.pythonhosted.org/packages/de/2f/a121c58260d63d16861fd936373d07c4ab0cef51b0d7391cafaf8e4648c0/pybase64-1.5.0-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:b0597ca31c472f3071844648ce5ab86a1732033ca230daffd8f87c6f8596a8ae", size = 77416, upload-time = "2026-08-08T15:38:57.995Z" }, - { url = "https://files.pythonhosted.org/packages/24/6a/ea3a1078de626ce765402d6d3e1cb6d69f83104646bcf2e2772983be77aa/pybase64-1.5.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:8d303baddeddaccada149bbee270b3e2eedcaec2df082834895cdd897a602674", size = 92473, upload-time = "2026-08-08T15:38:59.149Z" }, - { url = "https://files.pythonhosted.org/packages/8d/9c/22878279f1663bea15b5211056e3c8cb19c4783d2566a0032bcfa37d678b/pybase64-1.5.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:a34261348f88443d9e234f251a1f1fcb711c1cc006824fdb29b649735d8ac35f", size = 80804, upload-time = "2026-08-08T15:39:00.271Z" }, - { url = "https://files.pythonhosted.org/packages/9b/f0/57c36867282341ccc47c0db67590dd8f0c621fd435aa5944bec4713138b5/pybase64-1.5.0-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:e675b15b7a7b81e5b1a1e747cc49f9f9e6649d3b5e8a61719b46b9a671433210", size = 78871, upload-time = "2026-08-08T15:39:01.429Z" }, - { url = "https://files.pythonhosted.org/packages/4d/ce/23b80fde747156f6387a2f769fac1384e2e34cd4f07daa32e990991eb64a/pybase64-1.5.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:a1f8f1bb4158069291fe6ac2d34db942418f2804564d04b8e97722041035f843", size = 80451, upload-time = "2026-08-08T15:39:02.764Z" }, - { url = "https://files.pythonhosted.org/packages/bf/02/1486ad47fc065bbaa45c12229673bb03f0480dabdba408b04a54ac480264/pybase64-1.5.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:0abc0f2312c17765bf92dd382982cca9dc1b0148bf0d708f5f88339d84bb7687", size = 94725, upload-time = "2026-08-08T15:39:03.877Z" }, - { url = "https://files.pythonhosted.org/packages/43/ec/bf6a0df18b4a627a2ad3c8897e67797cb8128fed8cda2b654dd9ddebba25/pybase64-1.5.0-cp313-cp313-pyemscripten_2025_0_wasm32.whl", hash = "sha256:92998479a2a4464d141ef709e52dc3e4d4d4ce7f3b9cb5052d2c56c55b405b15", size = 33074, upload-time = "2026-08-08T15:39:04.939Z" }, - { url = "https://files.pythonhosted.org/packages/2c/4b/58a70d9655842161bcc3ae73efede60ad83d6d195fdf110f0c0ed808bca0/pybase64-1.5.0-cp313-cp313-win32.whl", hash = "sha256:91aceea4287299ee60c1176909efd6f2de091da24c0d93d2f9861c93e3776ef7", size = 42557, upload-time = "2026-08-08T15:39:05.992Z" }, - { url = "https://files.pythonhosted.org/packages/ba/43/157fddaa16e53e50813dc73b2cb9e4d03e797427394657e89e14a1a8843f/pybase64-1.5.0-cp313-cp313-win_amd64.whl", hash = "sha256:d01e4d495c5b10e79de3449501e41d2bc2a4aa90844a3735eb962a3a01645971", size = 44628, upload-time = "2026-08-08T15:39:07.067Z" }, - { url = "https://files.pythonhosted.org/packages/f2/c5/b5814d726d05749e6d5343a61c270a3c14a1f41faa20f4044ceb4f96d87c/pybase64-1.5.0-cp313-cp313-win_arm64.whl", hash = "sha256:1f7ddf3a7f1c85061f246a481c63a70d7aadd0a49add8e6c109b65360fbf923e", size = 39953, upload-time = "2026-08-08T15:39:08.188Z" }, - { url = "https://files.pythonhosted.org/packages/e4/99/9cc7eadd3dcc3b9d814a15381fe78bc59dff133d25ba3a8e49e4380fff30/pybase64-1.5.0-graalpy312-graalpy250_312_native-macosx_10_13_x86_64.whl", hash = "sha256:a9bcbdefd0858372c2e3c657ca8c1e2cdf0af5963cb45085cc861dfac0ddd422", size = 48565, upload-time = "2026-08-08T15:41:27.275Z" }, - { url = "https://files.pythonhosted.org/packages/77/04/0b073d5fe8d035c3334d44252218e82ca0717f71a1139efdbc1600c38463/pybase64-1.5.0-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:8b47a5b4a359e42b4b726cbd9558347c5324194aadaf12e4ad219efc89dc9812", size = 43122, upload-time = "2026-08-08T15:41:28.596Z" }, - { url = "https://files.pythonhosted.org/packages/ba/dc/cd57bd8629965d69eaaa721cf915f3c0590ba468811d290bbcdd3908f0ee/pybase64-1.5.0-graalpy312-graalpy250_312_native-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:b618ecec8f13b3f9dd58e257aa98fc9b017829a1bdc4f576e9146998956ec2c7", size = 54270, upload-time = "2026-08-08T15:41:29.872Z" }, - { url = "https://files.pythonhosted.org/packages/aa/22/67ad2ddf8ed03e0fc94341ebfc6ed694a36b9c908dd5a08b3ca366e31892/pybase64-1.5.0-graalpy312-graalpy250_312_native-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:5d09d63b219adfb1b40e104036dc2462234d2f06c05e436918e08f31a09a973b", size = 45919, upload-time = "2026-08-08T15:41:31.177Z" }, - { url = "https://files.pythonhosted.org/packages/2b/bb/4d080faff127cc8e5e0f5f6bb94d3a079235f83d0ef7355663f4bf214935/pybase64-1.5.0-graalpy312-graalpy250_312_native-win_amd64.whl", hash = "sha256:b059b951347a6e16d29b1488f624a7b213c7e8482869b1eac2b684e6fb1ac236", size = 45025, upload-time = "2026-08-08T15:41:32.601Z" }, -] - [[package]] name = "pyclipper" version = "1.4.0" @@ -3721,6 +3335,11 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/60/9d/7011887b29f452905745e8bd321f404068d5bfe78fe84e42c0b7cd81a065/pymilvus-3.0.1-py3-none-any.whl", hash = "sha256:c5a8d5c1fa1de7b416e3529d383d8cc2e7da2170433ffa4a2d9087e14f70171a", size = 386820, upload-time = "2026-07-29T14:55:44.279Z" }, ] +[package.optional-dependencies] +milvus-lite = [ + { name = "milvus-lite", marker = "sys_platform != 'win32'" }, +] + [[package]] name = "pypdfium2" version = "5.13.0" @@ -3750,24 +3369,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/50/7f/d39f6e64375c2ffd50ea100e3c73af79085c880c2791eb7203bc61d8913f/pypdfium2-5.13.0-py3-none-win_arm64.whl", hash = "sha256:554a0b23376460af1410e3c915906895e2dac67a086b9e6ccde0643a795d3b0d", size = 3700026, upload-time = "2026-08-13T10:58:14.206Z" }, ] -[[package]] -name = "pypika" -version = "0.51.1" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/f8/78/cbaebba88e05e2dcda13ca203131b38d3640219f20ebb49676d26714861b/pypika-0.51.1.tar.gz", hash = "sha256:c30c7c1048fbf056fd3920c5a2b88b0c29dd190a9b2bee971fd17e4abe4d0ebe", size = 80919, upload-time = "2026-02-04T11:27:48.304Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/57/83/c77dfeed04022e8930b08eedca2b6e5efed256ab3321396fde90066efb65/pypika-0.51.1-py2.py3-none-any.whl", hash = "sha256:77985b4d7ce71b9905255bf12468cf598349e98837c037541cfc240e528aec46", size = 60585, upload-time = "2026-02-04T11:27:46.251Z" }, -] - -[[package]] -name = "pyproject-hooks" -version = "1.2.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/e7/82/28175b2414effca1cdac8dc99f76d660e7a4fb0ceefa4b4ab8f5f6742925/pyproject_hooks-1.2.0.tar.gz", hash = "sha256:1e859bd5c40fae9448642dd871adf459e5e2084186e8d2c2a79a824c970da1f8", size = 19228, upload-time = "2024-09-29T09:24:13.293Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/bd/24/12818598c362d7f300f18e74db45963dbcb85150324092410c8b49405e42/pyproject_hooks-1.2.0-py3-none-any.whl", hash = "sha256:9e5c6bfa8dcc30091c74b0cf803c81fdd29d94f01992a7707bc97babb1141913", size = 10216, upload-time = "2024-09-29T09:24:11.978Z" }, -] - [[package]] name = "pytest" version = "9.1.1" @@ -4110,19 +3711,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", hash = "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", size = 73075, upload-time = "2026-05-14T19:25:26.443Z" }, ] -[[package]] -name = "requests-oauthlib" -version = "2.0.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "oauthlib" }, - { name = "requests" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/42/f2/05f29bc3913aea15eb670be136045bf5c5bbf4b99ecb839da9b422bb2c85/requests-oauthlib-2.0.0.tar.gz", hash = "sha256:b3dffaebd884d8cd778494369603a9e7b58d29111bf6b41bdc2dcd87203af4e9", size = 55650, upload-time = "2024-03-22T20:32:29.939Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/3b/5d/63d4ae3b9daea098d5d6f5da83984853c1bbacd5dc826764b249fe119d24/requests_oauthlib-2.0.0-py2.py3-none-any.whl", hash = "sha256:7dd8a5c40426b779b0868c404bdef9768deccf22749cde15852df527e6269b36", size = 24179, upload-time = "2024-03-22T20:32:28.055Z" }, -] - [[package]] name = "requests-toolbelt" version = "1.0.0" @@ -4918,49 +4506,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/1a/52/024ebece265b387154115dc4f1d9727174ef82623069f4bec8b7ed7e73f7/uuid_utils-0.17.0-cp313-cp313-win_arm64.whl", hash = "sha256:c351737e2e65497c7200ab4ffb8af97e9f48be6488309abdd265fe08d66ee92f", size = 173214, upload-time = "2026-07-09T13:49:04.836Z" }, ] -[[package]] -name = "uvicorn" -version = "0.52.4" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "click" }, - { name = "h11" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/f2/0f/3f86e61397dd33bf2ccf28188c40db6a740658aeebbbf6e7dbc101a1f487/uvicorn-0.52.4.tar.gz", hash = "sha256:73acfee47a0b133c5de13d219492d62d8a31e935f4fe6e41a232451a15379f86", size = 100627, upload-time = "2026-08-19T06:27:41.821Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/f1/79/4a20b54ab0491485ccd8c077db2d39187c7f12b3e15485d38a7be37c81b4/uvicorn-0.52.4-py3-none-any.whl", hash = "sha256:f86e41a149d7d05a9969337e3946a9c171c06a5d42680896daaba624aeac8da1", size = 79871, upload-time = "2026-08-19T06:27:40.36Z" }, -] - -[package.optional-dependencies] -standard = [ - { name = "httptools" }, - { name = "python-dotenv" }, - { name = "pyyaml" }, - { name = "uvloop", marker = "platform_python_implementation != 'PyPy' and sys_platform != 'cygwin' and sys_platform != 'win32'" }, - { name = "watchfiles" }, - { name = "websockets" }, -] - -[[package]] -name = "uvloop" -version = "0.22.1" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/06/f0/18d39dbd1971d6d62c4629cc7fa67f74821b0dc1f5a77af43719de7936a7/uvloop-0.22.1.tar.gz", hash = "sha256:6c84bae345b9147082b17371e3dd5d42775bddce91f885499017f4607fdaf39f", size = 2443250, upload-time = "2025-10-16T22:17:19.342Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/3d/ff/7f72e8170be527b4977b033239a83a68d5c881cc4775fca255c677f7ac5d/uvloop-0.22.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:fe94b4564e865d968414598eea1a6de60adba0c040ba4ed05ac1300de402cd42", size = 1359936, upload-time = "2025-10-16T22:16:29.436Z" }, - { url = "https://files.pythonhosted.org/packages/c3/c6/e5d433f88fd54d81ef4be58b2b7b0cea13c442454a1db703a1eea0db1a59/uvloop-0.22.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:51eb9bd88391483410daad430813d982010f9c9c89512321f5b60e2cddbdddd6", size = 752769, upload-time = "2025-10-16T22:16:30.493Z" }, - { url = "https://files.pythonhosted.org/packages/24/68/a6ac446820273e71aa762fa21cdcc09861edd3536ff47c5cd3b7afb10eeb/uvloop-0.22.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:700e674a166ca5778255e0e1dc4e9d79ab2acc57b9171b79e65feba7184b3370", size = 4317413, upload-time = "2025-10-16T22:16:31.644Z" }, - { url = "https://files.pythonhosted.org/packages/5f/6f/e62b4dfc7ad6518e7eff2516f680d02a0f6eb62c0c212e152ca708a0085e/uvloop-0.22.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7b5b1ac819a3f946d3b2ee07f09149578ae76066d70b44df3fa990add49a82e4", size = 4426307, upload-time = "2025-10-16T22:16:32.917Z" }, - { url = "https://files.pythonhosted.org/packages/90/60/97362554ac21e20e81bcef1150cb2a7e4ffdaf8ea1e5b2e8bf7a053caa18/uvloop-0.22.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:e047cc068570bac9866237739607d1313b9253c3051ad84738cbb095be0537b2", size = 4131970, upload-time = "2025-10-16T22:16:34.015Z" }, - { url = "https://files.pythonhosted.org/packages/99/39/6b3f7d234ba3964c428a6e40006340f53ba37993f46ed6e111c6e9141d18/uvloop-0.22.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:512fec6815e2dd45161054592441ef76c830eddaad55c8aa30952e6fe1ed07c0", size = 4296343, upload-time = "2025-10-16T22:16:35.149Z" }, - { url = "https://files.pythonhosted.org/packages/89/8c/182a2a593195bfd39842ea68ebc084e20c850806117213f5a299dfc513d9/uvloop-0.22.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:561577354eb94200d75aca23fbde86ee11be36b00e52a4eaf8f50fb0c86b7705", size = 1358611, upload-time = "2025-10-16T22:16:36.833Z" }, - { url = "https://files.pythonhosted.org/packages/d2/14/e301ee96a6dc95224b6f1162cd3312f6d1217be3907b79173b06785f2fe7/uvloop-0.22.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:1cdf5192ab3e674ca26da2eada35b288d2fa49fdd0f357a19f0e7c4e7d5077c8", size = 751811, upload-time = "2025-10-16T22:16:38.275Z" }, - { url = "https://files.pythonhosted.org/packages/b7/02/654426ce265ac19e2980bfd9ea6590ca96a56f10c76e63801a2df01c0486/uvloop-0.22.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6e2ea3d6190a2968f4a14a23019d3b16870dd2190cd69c8180f7c632d21de68d", size = 4288562, upload-time = "2025-10-16T22:16:39.375Z" }, - { url = "https://files.pythonhosted.org/packages/15/c0/0be24758891ef825f2065cd5db8741aaddabe3e248ee6acc5e8a80f04005/uvloop-0.22.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0530a5fbad9c9e4ee3f2b33b148c6a64d47bbad8000ea63704fa8260f4cf728e", size = 4366890, upload-time = "2025-10-16T22:16:40.547Z" }, - { url = "https://files.pythonhosted.org/packages/d2/53/8369e5219a5855869bcee5f4d317f6da0e2c669aecf0ef7d371e3d084449/uvloop-0.22.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:bc5ef13bbc10b5335792360623cc378d52d7e62c2de64660616478c32cd0598e", size = 4119472, upload-time = "2025-10-16T22:16:41.694Z" }, - { url = "https://files.pythonhosted.org/packages/f8/ba/d69adbe699b768f6b29a5eec7b47dd610bd17a69de51b251126a801369ea/uvloop-0.22.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:1f38ec5e3f18c8a10ded09742f7fb8de0108796eb673f30ce7762ce1b8550cad", size = 4239051, upload-time = "2025-10-16T22:16:43.224Z" }, -] - [[package]] name = "watchdog" version = "6.0.0" @@ -4985,56 +4530,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/33/e8/e40370e6d74ddba47f002a32919d91310d6074130fe4e17dabcafc15cbf1/watchdog-6.0.0-py3-none-win_ia64.whl", hash = "sha256:a1914259fa9e1454315171103c6a30961236f508b9b623eae470268bbcc6a22f", size = 79067, upload-time = "2024-11-01T14:07:11.845Z" }, ] -[[package]] -name = "watchfiles" -version = "1.2.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "anyio" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/cd/41/5e1a4bb12aac5f1493fa1bdc11154eca3b258ca4eba65d39c473fe19d8e9/watchfiles-1.2.0.tar.gz", hash = "sha256:c995fba777f1ea992f090f9236e9284cf7a5d1a0130dd5a3d82c598cacd76838", size = 108252, upload-time = "2026-05-18T04:32:04.251Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/b8/2f/e42c992d2afda3108ea1c02acecc991b9f31d05c14adc2a7cee9ee211fc4/watchfiles-1.2.0-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:bc13eb17538be00c874699dc0abe4ee2bc8d50bb1166a6b9e175ef3fd7eb8f26", size = 400115, upload-time = "2026-05-18T04:32:02.06Z" }, - { url = "https://files.pythonhosted.org/packages/5f/8f/6af2ea19065c91d8b0ea3516fdfc8c0d349f407e8e9fbf4e5a17360de8ad/watchfiles-1.2.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:2d95ddc1eb6914154253d239089900813f6a767e174b8e6a50e7fdacb7e4236c", size = 393659, upload-time = "2026-05-18T04:30:50.951Z" }, - { url = "https://files.pythonhosted.org/packages/13/01/b32a967c56fb3e3e5be3db52c3d3b87fa4513aa367d8ed1ad96d42952e5f/watchfiles-1.2.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8f70d8b291ef6e88d19b1f297a6905ddb978888d9272b0d05e6f53309856bcfc", size = 453207, upload-time = "2026-05-18T04:31:04.231Z" }, - { url = "https://files.pythonhosted.org/packages/04/98/97557a812180338cb1abd32e1cffcc4588f59b5f23e0cb006b2ba95ba64a/watchfiles-1.2.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:56d8641cf834c2836922899105bd3ce3d0dfc69291d52edf0b4d0436829b34c0", size = 459273, upload-time = "2026-05-18T04:31:50.377Z" }, - { url = "https://files.pythonhosted.org/packages/e8/a8/b4b08dcb7653b8087c6586f7ce649505900e866bbcfe40dc9587af02e686/watchfiles-1.2.0-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:2581a94056e55d7d0a31a823ea92bf73749c489ca2285bfdc0fbe6b2bb49d50c", size = 489927, upload-time = "2026-05-18T04:31:42.485Z" }, - { url = "https://files.pythonhosted.org/packages/50/94/3dceea03545d2e5ddfd839f0ddd5e1cecbf1697b5a428d5ba11cef6af95d/watchfiles-1.2.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:41bc1199f7523b3f82843c88cbb979180c949caef0342cf90968f178e5d49b01", size = 570476, upload-time = "2026-05-18T04:31:03.071Z" }, - { url = "https://files.pythonhosted.org/packages/cc/f2/d39a5450c3532092b91f81d274360e613c2371bc874a89c7a1a3c5e8d138/watchfiles-1.2.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:7571e4464cb6e434958f867f7f730b8ab0b75e3f8e5eac0499168486ab3c33a8", size = 465650, upload-time = "2026-05-18T04:30:12.701Z" }, - { url = "https://files.pythonhosted.org/packages/22/24/ed72f68cbc1333ca9b9f2200aa048bb6658ae41709bc1caad4310f4bdffd/watchfiles-1.2.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e53a384f76b631c3ae5334ce6a52f0baa3a911eb94a4eac7f160079868b716d5", size = 456398, upload-time = "2026-05-18T04:30:13.784Z" }, - { url = "https://files.pythonhosted.org/packages/0d/64/982ef4a4e5bab5b6e5b6becc8cd5e732f6130a78b855f0abec6439a9a135/watchfiles-1.2.0-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:d20029a60a71a052a24c4db7673bc4de39ab89adbaccbfb5d67987c5d73f424d", size = 465140, upload-time = "2026-05-18T04:31:52.111Z" }, - { url = "https://files.pythonhosted.org/packages/a0/0c/95282abf4ed680b6096010bcfc30c5fa7a041fc5aa5a2ad17a2cc6c75bba/watchfiles-1.2.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:2cb93af48550faf1cea04c303107c8b75833de7013e57ce27d3b8d21d8d0f58c", size = 630259, upload-time = "2026-05-18T04:31:25.676Z" }, - { url = "https://files.pythonhosted.org/packages/30/45/607c1de1530c4bdcf2cf1d1ecc2505ddba5d96bd43ba9f2b0e79876f850f/watchfiles-1.2.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:2995c176de7692b86a2e4c58d9ec718f753150a979cb4a754e2b4ffa38e70906", size = 659859, upload-time = "2026-05-18T04:30:24.333Z" }, - { url = "https://files.pythonhosted.org/packages/fa/08/d9e2e0f9e8e6791d33aefc694ad7eefa7f901f63caff84a81ded38692f9c/watchfiles-1.2.0-cp312-cp312-win32.whl", hash = "sha256:7a2cffd17d27d2ecbb310c2b1d8174f222a5495b1a721894afa88ec11e25b898", size = 275480, upload-time = "2026-05-18T04:30:31.307Z" }, - { url = "https://files.pythonhosted.org/packages/1c/e6/9d42569c0102645cc8cea5d8c7d8a1e9d4ada2cb7f05f75e554b8aa2202a/watchfiles-1.2.0-cp312-cp312-win_amd64.whl", hash = "sha256:f155b3a1b2a5fc89cdc70d47ee5d54e3b75e88efa34982028a35daef9ba00379", size = 288718, upload-time = "2026-05-18T04:32:10.745Z" }, - { url = "https://files.pythonhosted.org/packages/0a/26/88e0dc6ee3898169d7fa22bb6a69cabf2502d2ee25cb8c876d1262d204f8/watchfiles-1.2.0-cp312-cp312-win_arm64.whl", hash = "sha256:8fa585ede612ee9f9e91b18bebf9ba11b9ae29a4e3a0d0cf6fca3e382133f0d5", size = 281026, upload-time = "2026-05-18T04:30:22.23Z" }, - { url = "https://files.pythonhosted.org/packages/d1/4d/70a7feced9f87e2ff26dba42667290f41694fc64646c67261fbb8cab5d5c/watchfiles-1.2.0-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:01ea8d66f0693b9b60a6541c8d10263091ca9a9060d242f3c1f3143f9aad2c98", size = 399730, upload-time = "2026-05-18T04:31:38.162Z" }, - { url = "https://files.pythonhosted.org/packages/31/3a/0da302f2307aee316922806ebd5726c542cbd787c938271cf14a074c7daf/watchfiles-1.2.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:7ba0480b9a74af058f43b337e937a451e109295c420916d68ad24e3dc02f5e44", size = 392842, upload-time = "2026-05-18T04:30:27.051Z" }, - { url = "https://files.pythonhosted.org/packages/db/ef/d5bdb705c224dbc256aa0c1ec47bf4e61ec52558f2afb44a71a1fe4d7015/watchfiles-1.2.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4f34e26a19f91f710c08e0183429f0d1d15df734e6bc78c31e77b9ea9c433658", size = 452989, upload-time = "2026-05-18T04:31:11.945Z" }, - { url = "https://files.pythonhosted.org/packages/71/29/5495f2c1661949ef7a35e4d71111d129cfe7606414a26887a919d0a55406/watchfiles-1.2.0-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b4e77f6a55f858504069abd35d336a637555c09bca453dde1ee1e5ada8a6a1fb", size = 458978, upload-time = "2026-05-18T04:30:52.606Z" }, - { url = "https://files.pythonhosted.org/packages/d5/8c/7f9c07c433811c2fffd93e13fdfb7135de9aab5f2ae41be08960fa0047dc/watchfiles-1.2.0-cp313-cp313-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0cb4d80e212f116474a545c21c912b445f16bb0cef9e6a73a498164223e14e2f", size = 490248, upload-time = "2026-05-18T04:31:36.003Z" }, - { url = "https://files.pythonhosted.org/packages/3c/11/d93632febc52fbc21be90231bb7c17fd5387f46c9076fd40a5f9c2ae6910/watchfiles-1.2.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b974946a10af379d425e2eef5b62f5c6ebeaccf91d45eaad6f5b27ecd4f91aa0", size = 571847, upload-time = "2026-05-18T04:31:10.862Z" }, - { url = "https://files.pythonhosted.org/packages/55/b4/383173e73aabb07ad1d9c7aa859d95437ac46a6d6a1e11005facda0c9d19/watchfiles-1.2.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:86bc13c25a8d1fcd70b51d0ce7c9b65e90de5666fcbfd3e34957cc73ee19aeb5", size = 465974, upload-time = "2026-05-18T04:30:17.006Z" }, - { url = "https://files.pythonhosted.org/packages/a7/6c/89b1a230a78f57c52dd8893adb1f92f94411721b6ec12596c56d98c74356/watchfiles-1.2.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ca148d73dea36c9763aaa351e4d7a51780ec1584217c45276f4fe8239c768b71", size = 454782, upload-time = "2026-05-18T04:30:35.656Z" }, - { url = "https://files.pythonhosted.org/packages/24/62/1732118367cfff0a9fce3bf62ff4bfded09ef5df21d9d446b858b3f70a96/watchfiles-1.2.0-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:c525543d91961c6955b2636b308569e84a1d1c5f5f2932041ab9ef46422f43e3", size = 465182, upload-time = "2026-05-18T04:30:20.846Z" }, - { url = "https://files.pythonhosted.org/packages/28/96/716f7e5f51339bf22963f3345f9f27d7f3b30e2eadc597e257c881dd3c53/watchfiles-1.2.0-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:a204794696ffb8f9b10fba6f7cb5216d42f3b2b71860ccac6b6e42f5f10973b0", size = 629841, upload-time = "2026-05-18T04:31:05.397Z" }, - { url = "https://files.pythonhosted.org/packages/4c/fe/c40783950fd771ccf66ab3ec2722d188a9af1c7f96c6e811f36e40c6e03f/watchfiles-1.2.0-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:10d86db20695afe7997ac9e1717637d6714a8d0220458c33f3d2061f54cec427", size = 658028, upload-time = "2026-05-18T04:31:48.22Z" }, - { url = "https://files.pythonhosted.org/packages/71/72/4508db1856d1d87fcbb3b63f4839bab1b5682cb0e8d224d122263c09654a/watchfiles-1.2.0-cp313-cp313-win32.whl", hash = "sha256:eb283ee99e21ad6443c8cdb06ac5b34b1308c329cbdf03fa02b445363714c799", size = 275183, upload-time = "2026-05-18T04:30:59.57Z" }, - { url = "https://files.pythonhosted.org/packages/f9/36/14b76ca57652e5cc5fd1c11f32a261292c08a0d19a00351013c2549cbfb2/watchfiles-1.2.0-cp313-cp313-win_amd64.whl", hash = "sha256:a0f27f01bee51861392bb6b7c4fdb290b27d1eb194e9e28788d68102a0e898d9", size = 288059, upload-time = "2026-05-18T04:32:07.937Z" }, - { url = "https://files.pythonhosted.org/packages/1b/8d/0a85e395398d8d20fadfe5c5d32c726eee17a519e78fb356f2cf7531bffe/watchfiles-1.2.0-cp313-cp313-win_arm64.whl", hash = "sha256:3651aa7058595e9cfb75d35dd5ada2bf9f48a5b8a0f3562821d3e210c507e077", size = 280186, upload-time = "2026-05-18T04:31:54.484Z" }, - { url = "https://files.pythonhosted.org/packages/37/68/36db056f1fdcc5f07302f56e631774d6835bcd6fa3ace402304621d5f9e5/watchfiles-1.2.0-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:faea288b6f0ab1902ef08f4ca6de005dccf856c4e0c4f21b8c5fce02d90a1b08", size = 399031, upload-time = "2026-05-18T04:30:44.576Z" }, - { url = "https://files.pythonhosted.org/packages/c1/64/01a9d6f66a82a5c101ce939274106cc72759d62427e153f01edd2b9f87c2/watchfiles-1.2.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:01859b11fd9fbca670f4d5da00fbac282cfea9bd67a2125d8b2833a3b5617ea9", size = 391205, upload-time = "2026-05-18T04:30:25.413Z" }, - { url = "https://files.pythonhosted.org/packages/84/2c/0a44fe058cb4bb7b8ede6b6670698bbb7c0400740e378d00022189b7b31d/watchfiles-1.2.0-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:fff610d7bb2256a317bb1e96f0d7862c7aa8076733ee5df0fd41bbe76a24a4f4", size = 451892, upload-time = "2026-05-18T04:32:14.005Z" }, - { url = "https://files.pythonhosted.org/packages/67/a1/351e0d56cd35e6488b5c8b4fb11a809a5bc923e8fe8fed9faf8920be0c89/watchfiles-1.2.0-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b141a4891c995a039cd89e9a49e62df1dc8a559a5d1a6e4c7106d16c12777a55", size = 458867, upload-time = "2026-05-18T04:31:22.279Z" }, - { url = "https://files.pythonhosted.org/packages/d5/7d/9d09605187f1b838998624049fcf8bf47b73c1a3b76901fcac1782f62277/watchfiles-1.2.0-cp313-cp313t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f22943b7770483f6ea0721c6b11d022947a98eb0acae14694de034f4d0d38925", size = 490217, upload-time = "2026-05-18T04:31:43.657Z" }, - { url = "https://files.pythonhosted.org/packages/60/5d/a17a16eccb182f04188cd308ec24b1a71a9b5c4e7098269cf35d9fa56d02/watchfiles-1.2.0-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:1bc6195825b7dcd217968bb1f801a60fd4c16e8eeab5bedc7fe917d7d5995ab4", size = 571458, upload-time = "2026-05-18T04:32:11.875Z" }, - { url = "https://files.pythonhosted.org/packages/d3/3d/4dd457062083ab1938e5dfd45032eb425cee2ac817287ca8ff4356183e5d/watchfiles-1.2.0-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d4a4b147f5dca2a5d325a06a832fb43f345751adfbc63204aec30e0d9ca965a2", size = 464707, upload-time = "2026-05-18T04:30:43.492Z" }, - { url = "https://files.pythonhosted.org/packages/c6/71/ea8c57b128f5383de74d0c7d2d9c57ad7c9a65a930c451bd25d524b295b7/watchfiles-1.2.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4543579a9bdb0c9560039b4ffddbdb39545707659fbc430ce4c10f3f68d557f9", size = 454663, upload-time = "2026-05-18T04:30:16.061Z" }, - { url = "https://files.pythonhosted.org/packages/53/fd/2e812bf938406d7db351f0703ddd3fc6c061cf30d96153a77bc79a943a44/watchfiles-1.2.0-cp313-cp313t-manylinux_2_31_riscv64.whl", hash = "sha256:20aa0e708b920bde876a4aa82dc7dd6ebea228a63a67cda6632c2fc87b787efa", size = 463537, upload-time = "2026-05-18T04:31:44.9Z" }, - { url = "https://files.pythonhosted.org/packages/86/56/d17a7f1dd1bc3035f1072694a551301272f1739c2d8e319c927cb9e29b38/watchfiles-1.2.0-cp313-cp313t-musllinux_1_1_aarch64.whl", hash = "sha256:d413349d565dab74297f2a63e84a097936be69bf8f3b3801f27f380e32040f44", size = 629194, upload-time = "2026-05-18T04:31:14.141Z" }, - { url = "https://files.pythonhosted.org/packages/be/06/f1ff66bf5cae50aa4062779a0ecd0bbaf15e466195719074078947d9a17d/watchfiles-1.2.0-cp313-cp313t-musllinux_1_1_x86_64.whl", hash = "sha256:f28b2725eb8cce327b9b3ab02415c853011dc55c95832fe90de6bc56f5315f72", size = 656194, upload-time = "2026-05-18T04:31:47.14Z" }, -] - [[package]] name = "wcwidth" version = "0.8.3" @@ -5044,15 +4539,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/c4/0e/57f6bb3024a597b2e8ec4aee710ffe62ddc95af2e2bb1ee7a7abdc22c68c/wcwidth-0.8.3-py3-none-any.whl", hash = "sha256:d5b73dba6158a595ec9370350e7f2637bcac8d6c5e4fde34f30fcffb6103a5e4", size = 331669, upload-time = "2026-08-28T18:10:04.909Z" }, ] -[[package]] -name = "websocket-client" -version = "1.9.2" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/d8/cb/a5abcc2891249f393827c650c6296660ce40374ac22d99ab9aea41f9d2a2/websocket_client-1.9.2.tar.gz", hash = "sha256:0fcb57545848be86992e128218fd96dd87a6769ffdb1a968dff79632b85604d0", size = 84110, upload-time = "2026-08-31T14:08:40.964Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/d5/d2/cc4dc1271e464942db7ee278baae2daa99ee77cb2af744025c04da585a3e/websocket_client-1.9.2-py3-none-any.whl", hash = "sha256:e1a673830a9c7bfa47b1cd3d5e4178f4c9651d80a4eab02c9c23a1c3ec6250ce", size = 95786, upload-time = "2026-08-31T14:08:39.899Z" }, -] - [[package]] name = "websockets" version = "16.1.1"