You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/05/24 12:46:44 UTC

[GitHub] [arrow-datafusion] e-dard opened a new issue #408: Optimising an already optimised logical plan results in redundant pushed down filters

e-dard opened a new issue #408:
URL: https://github.com/apache/arrow-datafusion/issues/408


   **Describe the bug**
   
   If you run a `LogicalPlan` that has a `TableProviderFilterPushDown` `TableProvider`, then the filter pushdown optimiser rule will keep adding filters to the pushed down table scan without checking if they're already present.
   
   Given the plan:
   
   ```
   Filter: #a Eq Int64(1)
     TableScan: projection=None
   ```
   
   it will be optimised to the following plan the first time it runs through the filter:
   
   ```
   Filter: #a Eq Int64(1)
         TableScan: projection=None, filters=[#a Eq Int64(1)]
   ```
   
   If you run it through the optimiser again then it will not be left as is. Instead it will become:
   
   ```
   Filter: #a Eq Int64(1)
     TableScan: projection=None, filters=[#a Eq Int64(1), #a Eq Int64(1)]
   ```
   
   **To Reproduce**
   
   Run a logical plan that pushes predicates down to the table provider through the plan optimiser twice. All pushed down expression filters will be duplicated in the table  scan filters.
   
   **Expected behavior**
   
   The plan should not change if it's ran through the optimiser multiple times.
   
   **Additional context**
   
   Whilst the current behaviour produces correct plans they are not optimal because they can contain redunant duplicate filters if they're run through the optimiser multiple times.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] alamb closed issue #408: Optimising an already optimised logical plan results in redundant pushed down filters

Posted by GitBox <gi...@apache.org>.

alamb closed issue #408:
URL: https://github.com/apache/arrow-datafusion/issues/408


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org