Skip to content

Commit c620746

Browse files
adriangbclaude
andcommitted
docs: clarify what PushedDown::Yes and PushedDown::No mean
`PushedDown::Yes` means the child applies the predicate exactly, so the parent does not need to evaluate it again. `PushedDown::No` means the parent must still evaluate it. A child that replies `No` can still use the filter in an inexact way, for example for statistics pruning. Document this. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 95bb0a0 commit c620746

1 file changed

Lines changed: 23 additions & 2 deletions

File tree

‎datafusion/physical-plan/src/filter_pushdown.rs‎

Lines changed: 23 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -123,11 +123,32 @@ impl PushedDownPredicate {
123123
}
124124

125125
/// Discriminant for the result of pushing down a filter into a child node.
126+
///
127+
/// This answers one question for the parent: "do I still have to evaluate
128+
/// this filter?". It does not describe how the child uses the filter.
129+
///
130+
/// A child that replies [`PushedDown::No`] may still keep the filter and use
131+
/// it in an inexact way, for example to prune files, row groups or pages with
132+
/// statistics. `ParquetSource` does this when `pushdown_filters` is disabled.
133+
/// There is no separate "inexact" state: whether or not the child uses the
134+
/// filter, the parent must evaluate it, so the parent's action is the same.
135+
///
136+
/// Producers of filters that are not needed for correctness (for example
137+
/// dynamic filters from `HashJoinExec` or TopK) do not use this reply to
138+
/// find out if a consumer exists. They check whether the filter is present
139+
/// in the child subtree instead (see
140+
/// [`plan_contains_expression_id`](crate::execution_plan::plan_contains_expression_id)).
126141
#[derive(Debug, Clone, Copy)]
127142
pub enum PushedDown {
128-
/// The predicate was successfully pushed down into the child node.
143+
/// The child guarantees that it applies the predicate exactly: it never
144+
/// produces a row for which the predicate is not true. The parent does
145+
/// not need to evaluate the predicate again.
129146
Yes,
130-
/// The predicate could not be pushed down into the child node.
147+
/// The child does not guarantee that it applies the predicate exactly,
148+
/// so the parent must still evaluate it.
149+
///
150+
/// The child may ignore the predicate, or it may use it in an inexact
151+
/// way, for example for statistics pruning.
131152
No,
132153
}
133154

0 commit comments

Comments
 (0)