Q：why Hive storage plug-in optimization rule NOT working?

[Nai Yan <zh...@bigknow.com.cn>]

Greetings,
      We are trying to learn about drill storage plug-in and try to optimize it for our use scenario, however we have found Hive's opt rule "HivePushPartitionFilterIntoScan" NOT really get involved in planning phase. Accorrding to our understanding, Storage plug-in's opt rule will be included in logical/physical planning phase. If I am wrong? 

       In addition to this, we have found even though we commented out this rule clause, partition filter still pushed down. 

       Any help will be appreciated. 

      Thanks! 



Nai Yan
 

Re: Q：why Hive storage plug-in optimization rule NOT working?

[Vitalii Diravka <vi...@apache.org>]

Hi!

*HivePushPartitionFilterIntoSc* is logical optimizer rule and is a part of
"Partition Prune Planning". So it should work fine after matching.
From your words looks like pushing partition filter works earlier, before
that rule. Possibly it may be true, so then it will be nice to refactor
this code and optimize usage of these rules.
It can be done by investigating Drill intermediate logical plans by adding
needed loggers in the *DefaultSqlHandler* class.
Also Jira ticket will be helpful for this task too.

Kind regards
Vitalii


On Fri, Jan 25, 2019 at 4:43 AM Nai Yan <zh...@bigknow.com.cn> wrote:

> Greetings,
>       We are trying to learn about drill storage plug-in and try to
> optimize it for our use scenario, however we have found Hive's opt rule
> "HivePushPartitionFilterIntoScan" NOT really get involved in planning
> phase. Accorrding to our understanding, Storage plug-in's opt rule will be
> included in logical/physical planning phase. If I am wrong?
>
>        In addition to this, we have found even though we commented out
> this rule clause, partition filter still pushed down.
>
>        Any help will be appreciated.
>
>       Thanks!
>
>
>
> Nai Yan
>
>