Skip to content

docs(cosmosdb): filters, projection and limits push down into Cosmos DB - #2298

Merged
lukekim merged 5 commits into
trunkfrom
docs/cosmosdb-pushdown
Oct 2, 2026
Merged

lukekim merged 5 commits into
trunkfrom
docs/cosmosdb-pushdown

Conversation

@spiceemma

Copy link
Copy Markdown
Contributor

Summary

spiceai/spiceai#14419 added filter, projection, and limit pushdown to the Cosmos DB connector. On vNext, the Cosmos DB reference page and deployment guide still listed "No filter / projection / limit pushdown" as a limitation. This PR replaces that bullet on both pages with a description of what pushes down: the projection and supported WHERE conditions when query is the default SELECT * FROM c, and a LIMIT with any query.

Source PRs

Files changed (2)

  • website/docs/components/data-connectors/cosmosdb/index.md
  • website/docs/components/data-connectors/cosmosdb/deployment.md

Propagation: version-specific change, vNext only. git grep -c supports_filters_pushdown v2.3.2 -- crates/data-connectors/connector-cosmosdb returns no hits, so the "No filter / projection / limit pushdown" bullet stays correct in the 8 versioned copies (2.0.x–2.3.x, two pages each). grep -rln "No filter / projection / limit pushdown" website/docs now returns nothing.

The MongoDB and DynamoDB parts of #14419 change which filters push down, but no published page makes a claim about their pushdown, so neither page needs a change. grep -in push returns only an unrelated DynamoDB timestamp line.

Verification

All code is at origin/trunk b5126937a8. These are name/existence claims, each Reproduced by the quoted line:

  • Pushdown only with the default query: crates/data-connectors/connector-cosmosdb/src/cosmosdb/provider.rs: fn pushes_down(&self) -> bool { self.config.query.trim().eq_ignore_ascii_case(super::DEFAULT_QUERY) }, with pub const DEFAULT_QUERY: &str = "SELECT * FROM c"; in cosmosdb/mod.rs. supports_filters_pushdown returns Unsupported for every filter when !self.pushes_down().
  • Projection: pushed_down_query builds SELECT {projected properties} FROM c WHERE ….
  • Inexact (Spice filters again): filter.rs classify: Some(_) => TableProviderFilterPushDown::Inexact, None => TableProviderFilterPushDown::Unsupported.
  • Supported conditions, from Translator::translate: Operator::And; Operator::Or (both sides must translate); column-vs-literal Eq/NotEq/Lt/LtEq/Gt/GtEq; a boolean Expr::Column and Expr::Not over a boolean column only; IsNull; IsNotNull; Between { negated: false }; InList { negated: false } with values.len() > MAX_IN_VALUES (256) or any null refused; Like on a Kind::Utf8 column via like_prefix; DataFusion's own starts_with; _ => None.
  • Limit: CosmosDBExec reads while read < limit for either query mode.

Behavior: Unverified, code inspection only. I did not run the claim that results match a query with no pushdown, because it needs a Cosmos DB account or the emulator. The source PR's differential round-trip suite (crates/runtime/tests/pushdown_roundtrip/) is the runtime evidence.

Dedupe: gh pr list --repo spiceai/docs --state all --search "spiceai/spiceai#14419" → []. Open bot PRs #2133, #2296, and #2297 do not touch these files.

Build: cd website && npm run build exits 0 and ends with [SUCCESS] Generated static files in "build".

Test plan

  • cd website && npm run build passes
  • Versioned-docs propagation checked: vNext only (no Cosmos DB pushdown at v2.3.2)
  • Files updated: 2

Reproduction

Doc said (website/docs/components/data-connectors/cosmosdb/index.md:223 at 6409e742): No filter / projection / limit pushdown into Cosmos: SQL predicates are evaluated locally by DataFusion after the documents have been streamed in.
Code says (connector_cosmosdb::cosmosdb::provider::CosmosDBTableProvider::supports_filters_pushdown at trunk b5126937a8): let translator = Translator::new(&self.schema); Ok(filters.iter().map(|f| translator.classify(f)).collect())
Behavior claim: not run (needs a Cosmos DB account or the emulator), so the correction is Unverified, code inspection only for result equivalence.

@spiceemma
spiceemma requested a review from a team as a code owner October 2, 2026 08:16
Copilot AI balanced review requested due to automatic review settings October 2, 2026 08:16
@spiceemma spiceemma self-assigned this Oct 2, 2026
@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

✅ Pull with Spice Passed

Passing checks:

  • ✅ Title meets minimum length requirement (10 characters)
  • ✅ Has at least one of the required labels: area/blog, area/docs, area/cookbook, dependencies
  • ✅ No banned labels detected
  • ✅ Has at least one assignee: spiceemma

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

🔍 Pull with Spice Failed

Passing checks:

  • ✅ Title meets minimum length requirement (10 characters)
  • ✅ Has at least one of the required labels: area/blog, area/docs, area/cookbook, dependencies
  • ✅ No banned labels detected

Failed checks:

  • ❌ At least one assignee is required for this pull request.

Please address these issues and update your pull request.

1 similar comment
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

🔍 Pull with Spice Failed

Passing checks:

  • ✅ Title meets minimum length requirement (10 characters)
  • ✅ Has at least one of the required labels: area/blog, area/docs, area/cookbook, dependencies
  • ✅ No banned labels detected

Failed checks:

  • ❌ At least one assignee is required for this pull request.

Please address these issues and update your pull request.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Both pages overstate filter support by omitting data-type and literal constraints.

Review effort: Balanced
Findings: 2 Low severity

Open (2)
What changed in this PR

Updates vNext Cosmos DB documentation for pushdown behavior introduced by spiceai/spiceai#14419.

Changes:

  • Documents projection and partial filter pushdown.
  • Documents limit behavior and custom-query restrictions.
File Description
cosmosdb/​index.md Updates connector limitations.
cosmosdb/​deployment.md Updates deployment limitations.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread website/docs/components/data-connectors/cosmosdb/deployment.md Outdated
Comment thread website/docs/components/data-connectors/cosmosdb/index.md Outdated
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

🚀 deployed to https://a138b199.spiceai-org-website.pages.dev

Copilot AI balanced review requested due to automatic review settings October 2, 2026 08:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Both pages overstate starts_with support and query-outcome equivalence.

Review effort: Balanced
Findings: 2 Low severity

Open (2)
Resolved since last review (2)

Comment thread website/docs/components/data-connectors/cosmosdb/deployment.md Outdated
Comment thread website/docs/components/data-connectors/cosmosdb/index.md Outdated
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

🚀 deployed to https://2142c9f8.spiceai-org-website.pages.dev

Copilot AI balanced review requested due to automatic review settings October 2, 2026 08:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The documented filter constraints omit that floating-point literals must be finite.

Review effort: Balanced
Findings: 2 Low severity

Open (2)
Resolved since last review (2)

Comment thread website/docs/components/data-connectors/cosmosdb/deployment.md Outdated
Comment thread website/docs/components/data-connectors/cosmosdb/index.md Outdated
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

🚀 deployed to https://21b59e99.spiceai-org-website.pages.dev

…pice

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 2, 2026 10:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Both pages inaccurately describe Boolean and BETWEEN pushdown support.

Review effort: Balanced
Findings: 2 Low severity

Open (2)
Resolved since last review (2)

Comment thread website/docs/components/data-connectors/cosmosdb/deployment.md Outdated
Comment thread website/docs/components/data-connectors/cosmosdb/index.md Outdated
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

🚀 deployed to https://ed5426fe.spiceai-org-website.pages.dev

…WEEN to literal bounds

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 2, 2026 11:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approved

The documentation matches the referenced connector implementation and correctly remains limited to vNext.

Review effort: Balanced
Findings: None

Resolved since last review (2)

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

🚀 deployed to https://cdbef61d.spiceai-org-website.pages.dev

@lukekim
lukekim merged commit d17171b into trunk Oct 2, 2026
8 checks passed
@lukekim
lukekim deleted the docs/cosmosdb-pushdown branch October 2, 2026 15:08

This branch was successfully deployed

1 active deployment
preview — 3cbac75c Deployed Oct 2, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants