[jira] [Work logged] (HIVE-25941) Long compilation time of complex query due to analysis for materialized view rewrite

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-25941?focusedWorklogId=724372&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-724372 ]

ASF GitHub Bot logged work on HIVE-25941:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 10/Feb/22 09:04
            Start Date: 10/Feb/22 09:04
    Worklog Time Spent: 10m 
      Work Description: kasakrisz opened a new pull request #3014:
URL: https://github.com/apache/hive/pull/3014


   ### What changes were proposed in this pull request?
   1. When rewriting queries automatically to use MVs lookup the MVs by the AST of the subquery should be rewritten.
   2. Store MVs AST in `HiveRelOptMaterialization`
   3. To get the expanded subquery AST required the lookup generate the expanded query test only one by using `UnparseTranslator` and `TokeRewriteStream` and call the parser with the expanded sql query text to get the AST.
   
   ### Why are the changes needed?
   `UnparseTranslator` and `TokeRewriteStream` are called for each subquery and complex queries containing lots of subqueries require more compilation time. By generating the expanded sql query text of the whole query once and comparing the ASTs of the subqueries runs faster.
   
   ### Does this PR introduce _any_ user-facing change?
   No.
   
   ### How was this patch tested?
   ```
   mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=materialized_view_rewrite_by_text.q,materialized_view_rewrite_by_text_3.q,materialized_view_rewrite_by_text_4.q,materialized_view_rewrite_by_text_5.q,materialized_view_rewrite_by_text_6.q,materialized_view_rewrite_by_text_7.q,materialized_view_rewrite_by_text_8.q -pl itests/qtest -Pitests
   
   mvn test -Dtest=TestMaterializedViewsCache -pl ql
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 724372)
    Remaining Estimate: 0h
            Time Spent: 10m

> Long compilation time of complex query due to analysis for materialized view rewrite
> ------------------------------------------------------------------------------------
>
>                 Key: HIVE-25941
>                 URL: https://issues.apache.org/jira/browse/HIVE-25941
>             Project: Hive
>          Issue Type: Bug
>          Components: Materialized views
>            Reporter: Krisztian Kasa
>            Assignee: Krisztian Kasa
>            Priority: Major
>         Attachments: sample.png
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> When compiling query the optimizer tries to rewrite the query plan or subtrees of the plan to use materialized view scans.
> If
> {code}
> set hive.materializedview.rewriting.sql.subquery=false;
> {code}
> the compilation succeed in less then 10 sec otherwise it takes several minutes (~ 5min) depending on the hardware.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)