[jira] [Resolved] (DRILL-3765) Partition prune rule is unnecessary fired multiple times.

["Jinfeng Ni (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DRILL-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jinfeng Ni resolved DRILL-3765.
-------------------------------
    Resolution: Won't Fix

I looked at the Calcite trace file. Seems the multiple rule firing is what Calcite's rule framework is supposed to do. 

The multiple rule firings are actually against different set of RelNode, which are result of other rules firing. For instance, one scan could be the one before Project push down, while the other one could be after the project push down. In other words, there is no redundant rule firing, against the same set of RelNode.  Seems the multiple rule firing in this case is the way it should be.

If we stop the subsequent rule's firing after the first rule firing, the query planner may lose the chance to find the ultimate best plan. 

Therefore, I'm going to mark this as "won't fix".  

> Partition prune rule is unnecessary fired multiple times. 
> ----------------------------------------------------------
>
>                 Key: DRILL-3765
>                 URL: https://issues.apache.org/jira/browse/DRILL-3765
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>            Reporter: Jinfeng Ni
>            Assignee: Jinfeng Ni
>
> It seems that the partition prune rule may be fired multiple times, even after the first rule execution has pushed the filter into the scan operator. Since partition prune has to build the vectors to contain the partition /file / directory information, to invoke the partition prune rule unnecessary may lead to big memory overhead.
> Drill planner should avoid the un-necessary partition prune rule, in order to reduce the chance of hitting OOM exception, while the partition prune rule is executed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)