Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ Cosmos DB requests participate in [task history](../../../reference/task_history
## Known Limitations

- **Read-only**: Writes (`INSERT` / `UPDATE` / `DELETE`) are not supported.
- **No filter / projection / limit pushdown**: SQL predicates are evaluated locally by DataFusion. Use a custom `query:` to narrow at the Cosmos side.
- **Partial filter pushdown**: When `query` is left at its default, `SELECT * FROM c`, Spice pushes the projection and supported `WHERE` conditions into the Cosmos DB query. Supported conditions are a column compared with a literal (`=`, `<>`, `<`, `<=`, `>`, `>=`), `IS NULL`, `IS NOT NULL`, `BETWEEN` with literal bounds, an `IN` list of up to 256 non-null literals, `LIKE 'prefix%'` and DataFusion's built-in `starts_with` with a non-empty literal prefix on a string column, a boolean column or its `NOT`, and `AND` and `OR` combinations of these. `NOT IN`, `NOT BETWEEN`, and other functions are evaluated in Spice. A condition pushes down only on a column of type `Utf8`, `Int64`, `Float64`, or `Boolean`. A comparison on a boolean column supports only `=` and `<>` (and `BETWEEN` does not push down on one), a string range comparison (`<`, `<=`, `>`, `>=`) needs an ASCII bound, an integer literal must be smaller than 2^53 in absolute value, and a floating-point literal must be finite: a condition on `NaN`, `Infinity`, or `-Infinity` is evaluated in Spice. Spice evaluates any other condition itself. Each pushed condition reads a superset of the matching documents, and DataFusion filters the rows again, so the returned rows match a query that pushes nothing down. A document that a pushed condition excludes is not read, so a value in it that another column cannot hold no longer fails the query. With a custom `query`, Spice pushes no conditions or projection and reads the documents that query returns. With either kind of query, a `LIMIT` stops the read once enough documents have been read.
- **Schema is frozen at registration**: Mapping changes after startup require a runtime restart.
- **No change feed**: `RefreshMode::Changes` is not wired.
- **Mid-stream retries are not safe**: Retries apply to the schema-inference pass only. Errors during a streaming scan propagate immediately; rely on dataset refresh-level retry instead.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ datasets:
## Limitations

- **Read-only**: Only `SELECT` scans are supported. Writes (`INSERT` / `UPDATE` / `DELETE`) are not implemented.
- **No filter / projection / limit pushdown into Cosmos**: SQL predicates are evaluated locally by DataFusion after the documents have been streamed in. Use the `query` parameter to narrow at the Cosmos side.
- **Partial filter pushdown**: When `query` is left at its default, `SELECT * FROM c`, Spice pushes the projection and supported `WHERE` conditions into the Cosmos DB query. Supported conditions are a column compared with a literal (`=`, `<>`, `<`, `<=`, `>`, `>=`), `IS NULL`, `IS NOT NULL`, `BETWEEN` with literal bounds, an `IN` list of up to 256 non-null literals, `LIKE 'prefix%'` and DataFusion's built-in `starts_with` with a non-empty literal prefix on a string column, a boolean column or its `NOT`, and `AND` and `OR` combinations of these. `NOT IN`, `NOT BETWEEN`, and other functions are evaluated in Spice. A condition pushes down only on a column of type `Utf8`, `Int64`, `Float64`, or `Boolean`. A comparison on a boolean column supports only `=` and `<>` (and `BETWEEN` does not push down on one), a string range comparison (`<`, `<=`, `>`, `>=`) needs an ASCII bound, an integer literal must be smaller than 2^53 in absolute value, and a floating-point literal must be finite: a condition on `NaN`, `Infinity`, or `-Infinity` is evaluated in Spice. Spice evaluates any other condition itself. Each pushed condition reads a superset of the matching documents, and DataFusion filters the rows again, so the returned rows match a query that pushes nothing down. A document that a pushed condition excludes is not read, so a value in it that another column cannot hold no longer fails the query. With a custom `query`, Spice pushes no conditions or projection and reads the documents that query returns. With either kind of query, a `LIMIT` stops the read once enough documents have been read.
- **Schema is frozen at registration**: New fields added in Cosmos after startup are not picked up until the runtime restarts.
- **No change feed**: `acceleration.refresh_mode: changes` is not supported.
- **No native temporal / decimal / binary types**: Cosmos stores everything as JSON. Round-trip these as strings and `CAST` in SQL.
Expand Down
Loading