[jira] [Commented] (HIVE-25941) Long compilation time of complex query due to analysis for materialized view rewrite

["Krisztian Kasa (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-25941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489356#comment-17489356 ] 

Krisztian Kasa commented on HIVE-25941:
---------------------------------------

{{hive.materializedview.rewriting.sql.subquery}} controls whether the algorithms compares the sql text of the query with MV definitions sql texts is enabled or not. Such comparison requires the expanded query text. The algorithm takes each subquery and generates the expanded text of that subquery by using The {{UnparseTranslator}} and {{TokenRewriteStream}}.
A significant time is consumed by {{TokenRewriteStream}} while compiling a query with more than 100 subqueries ( !sample.png! ).

A possible solution is instead of comparing the expanded sql query texts try comparing the AST trees.

> Long compilation time of complex query due to analysis for materialized view rewrite
> ------------------------------------------------------------------------------------
>
>                 Key: HIVE-25941
>                 URL: https://issues.apache.org/jira/browse/HIVE-25941
>             Project: Hive
>          Issue Type: Bug
>          Components: Materialized views
>            Reporter: Krisztian Kasa
>            Assignee: Krisztian Kasa
>            Priority: Major
>         Attachments: sample.png
>
>
> When compiling query the optimizer tries to rewrite the query plan or subtrees of the plan to use materialized view scans.
> If
> {code}
> set hive.materializedview.rewriting.sql.subquery=false;
> {code}
> the compilation succeed in less then 10 sec otherwise it takes several minutes (~ 5min) depending on the hardware.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)