[GitHub] [arrow-datafusion] Dandandan commented on pull request #436: Remove reundant filters (e.g. c> 5 AND c>5 --> c>5)

[GitBox <gi...@apache.org>]

Dandandan commented on pull request #436:
URL: https://github.com/apache/arrow-datafusion/pull/436#issuecomment-850792952


   Looks like a good start.
   
   I think it's good to think of these rules as simple rewrite rules we can apply to the expression tree.
   
   ```
   p OR p => a
   p AND p => a
   ```
   
   Some weeks ago I implemented some rules in this custom DataFusion optimizer (based on graph rewriting), you can see them here:
   
   https://github.com/Dandandan/datafusion-tokomak/blob/main/src/lib.rs#L44
   There are a lot of other patterns we can add if we do term rewriting in this way.
   
   Also, I think we should make this optimizer run on *any* `Expr`, not just those in `Filter` as this would benefit any `Expr` (in projections, aggregations, joins, etc)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org