[GitHub] [arrow-datafusion] lvheyang commented on issue #723: ABS() function in WHERE clause gives unexpected results

[GitBox <gi...@apache.org>]

lvheyang commented on issue #723:
URL: https://github.com/apache/arrow-datafusion/issues/723#issuecomment-880027602


   I have reproduced this problem. I found the problem is in datafusion/src/physical_optimizer/pruning.rs, the PruningPredicate. 
   
   In its comment :
   
   ```
       /// A pruning predicate is one that has been rewritten in terms of
       /// the min and max values of column references and that evaluates
       /// to FALSE if the filter predicate would evaluate FALSE *for
       /// every row* whose values fell within the min / max ranges (aka
       /// could be pruned).
   ```
   
   It emphasizes that if the pruning predicate evaluates to FALSE , then the filter should evaluates FALSE for **every row**
   
   In this case, it built the pruning predicate as `abs(c0_min - 251.10794896957802) < 1`, unluckily it's not right. 
   
   c0_min is 107.0090813093981, this pruning predicate would evaluate False.
   
   But for the row with value 251.10794896957802, this predicate evaluate True. 
   
   So it violates the rule in the comment, and this causes the whole row group was omitted, so there is no result when compared with 1 or 111.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org