Skip to content

Retain selectivity from supported AND conjuncts #25612

Description

@gabotechs

Describe the bug

Adding an unsupported AND condition discards the useful estimate for the
supported condition. The extra restriction increases the estimate tenfold
because the whole predicate falls back to 20% selectivity.

To Reproduce

From the repository root, with the CLI fix from
PR #25570 applied:

cargo build --profile ci --locked -p datafusion-benchmarks --bin dfbench
cargo install tpchgen-cli --version 1.1.1 --locked # if not already installed
repro_dir=$(mktemp -d)
tpchgen-cli --scale-factor 1 --format parquet \
  --parquet-compression 'ZSTD(1)' --parts 1 --output-dir "$repro_dir/data"
cat > "$repro_dir/repro.sql" <<'SQL'
SET datafusion.execution.target_partitions = 1;
SET datafusion.optimizer.enable_dynamic_filter_pushdown = false;
SELECT p_partkey FROM part WHERE p_size = 15;

SELECT p_partkey FROM part WHERE p_size = 15 AND p_type LIKE '%BRASS%';
SQL
target/ci/dfbench statistics \
  --path "$repro_dir/data" --query_path "$repro_dir/repro.sql"

Observed with tpchgen-cli 1.1.1 at
6c320561b5.
Inspect the SELECT reports; ignore the empty SET reports.

SELECT FilterExec node Estimated rows Actual rows
p_size = 15 0 4,000 3,907
Add AND p_type LIKE '%BRASS%' 0 40,000 747

Expected behavior

Retain supported conjunct estimates and combine unsupported conjuncts
conservatively. Adding an AND condition should not increase estimated rows; OR
requires different treatment.

Additional context

Related:
filter selectivity #14237.

Part of #25610.

No activity

Activity on this issue will appear here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions