[GitHub] [arrow-datafusion] alamb commented on a change in pull request #561: Fix pruning on not equal predicate

[GitBox <gi...@apache.org>]

alamb commented on a change in pull request #561:
URL: https://github.com/apache/arrow-datafusion/pull/561#discussion_r651282910



##########
File path: datafusion/src/physical_optimizer/pruning.rs
##########
@@ -553,12 +553,14 @@ fn build_predicate_expression(
     let corrected_op = expr_builder.correct_operator(op);
     let statistics_expr = match corrected_op {
         Operator::NotEq => {
-            // column != literal => (min, max) = literal => min > literal || literal > max
+            // column != literal => (min, max) = literal =>
+            // !(min != literal && max != literal) ==>
+            // min != literal || literal != max
             let min_column_expr = expr_builder.min_column_expr()?;
             let max_column_expr = expr_builder.max_column_expr()?;
             min_column_expr
-                .gt(expr_builder.scalar_expr().clone())
-                .or(expr_builder.scalar_expr().clone().gt(max_column_expr))
+                .not_eq(expr_builder.scalar_expr().clone())
+                .or(expr_builder.scalar_expr().clone().not_eq(max_column_expr))

Review comment:
       The way I think about it = predicate is effectively `!(col == min && col == max)` which when you apply Demorgan's law you get `(col != min) || (col != max)`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org