feat: mark hash join, TopK and aggregate dynamic filters as optional - #25681

Draft

adriangb wants to merge 17 commits into

apache:mainfrom

pydantic:optional-filter-producers

Commits on Sep 26, 2026

perf: collapse partitioned hash join InList dynamic filters into one IN list

adriangb
and
claude
committed
d4526b1
perf: dedupe collapsed hash join IN lists and add combined bounds

adriangb
and
claude
committed
ef7dde0
perf: push partitioned hash join dynamic filter bounds and membership as separate filters

adriangb
and
claude
committed
a6a6b1c
bench: add ordered-subset hash join dynamic filter benchmark

adriangb
and
claude
committed
4f4c886
fix: rename test to pass the typos check

adriangb
and
claude
committed
72f9ed6
feat: add OptionalFilterPhysicalExpr to mark filters not needed for correctness

adriangb
and
claude
committed
7c39ddd
feat: add filter_stats (FilterCost, Clock) for filter measurements

adriangb
and
claude
committed
3eb4b80
feat: add OptionalFilterGate to pause optional filters that cost more than they save

adriangb
and
claude
committed
4f7a4f7
feat(gate): add a measured overhead for each evaluated row to the cost

adriangb
and
claude
committed
26c2bef
feat(gate): share pauses between the gates of one plan site

adriangb
and
claude
committed
4a3a9e0
feat(gate): decide only on windows of at least MIN_OBSERVED_ROWS rows

adriangb
and
claude
committed
7bdfa6d
feat(gate): use the measured work of the filter producer as the saving

adriangb
and
claude
committed
582645c
feat: mark hash join, TopK and aggregate dynamic filters as optional

adriangb
and
claude
committed
8942765
test: update plan snapshots for optional dynamic filters

adriangb
and
claude
committed
9d7fac7
feat(hash-join): measure the work that a dynamic filter saves for each removed row

adriangb
and
claude
committed
31519e8
fix(hash-join): measure only the work of a probe row without a match

adriangb
and
claude
committed
72cbd47

Commits on Sep 28, 2026

Merge apache/main into optional-filter-producers

adriangb
and
claude
committed
6ae033c

Navigation Menu

feat: mark hash join, TopK and aggregate dynamic filters as optional - #25681

feat: mark hash join, TopK and aggregate dynamic filters as optional#25681
adriangb wants to merge 17 commits into
apache:mainfrom
pydantic:optional-filter-producers

Commits on Sep 26, 2026

perf: collapse partitioned hash join InList dynamic filters into one IN list

perf: dedupe collapsed hash join IN lists and add combined bounds

perf: push partitioned hash join dynamic filter bounds and membership as separate filters

bench: add ordered-subset hash join dynamic filter benchmark

fix: rename test to pass the typos check

feat: add OptionalFilterPhysicalExpr to mark filters not needed for correctness

feat: add filter_stats (FilterCost, Clock) for filter measurements

feat: add OptionalFilterGate to pause optional filters that cost more than they save

feat(gate): add a measured overhead for each evaluated row to the cost

feat(gate): share pauses between the gates of one plan site

feat(gate): decide only on windows of at least MIN_OBSERVED_ROWS rows

feat(gate): use the measured work of the filter producer as the saving

feat: mark hash join, TopK and aggregate dynamic filters as optional

test: update plan snapshots for optional dynamic filters

feat(hash-join): measure the work that a dynamic filter saves for each removed row

fix(hash-join): measure only the work of a probe row without a match

Commits on Sep 28, 2026

Merge apache/main into optional-filter-producers