[jira] [Work logged] (HIVE-24274) Implement Query Text based MaterializedView rewrite

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-24274?focusedWorklogId=516640&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-516640 ]

ASF GitHub Bot logged work on HIVE-24274:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 25/Nov/20 12:22
            Start Date: 25/Nov/20 12:22
    Worklog Time Spent: 10m 
      Work Description: kasakrisz opened a new pull request #1706:
URL: https://github.com/apache/hive/pull/1706


   ### What changes were proposed in this pull request?
   * Add feature: Enable materialized view rewrite of a query if the query text is the same as the query defined in the materialized view.
   * Enable unparsing for all queries in order to generate the expanded query text for comparison.
   * Refactor and extend the `HiveMaterializedViewsRegistry` with the lookup by query text functionality.
   
   ### Why are the changes needed?
   This patch provides an alternative way to rewrite queries using materialized views. Materialized view query definitions has some limitations like can't have `UNION`, `SORT BY` operator. These are enabled when using the text based rewrite.
   
   ### Does this PR introduce _any_ user-facing change?
   In some cases when rewrite was not possible because of the limitations mentioned above. With this patch the rewriting will be executed and it will have an effect of the output of `EXPLAIN`, `EXPLAIN CBO` commands: instead of the original query plan a scan on the materialized view will appear.
   
   ### How was this patch tested?
   ```
   mvn test -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=mv_rewrite_by_text.q,masking_14.q,masking_mv.q,schq_materialized.q,sketches_materialized_view_safety.q -pl itests/qtest -Pitests
   mvn test -Dtest=TestMaterializedViewsCache -pl ql
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 516640)
    Time Spent: 20m  (was: 10m)

> Implement Query Text based MaterializedView rewrite
> ---------------------------------------------------
>
>                 Key: HIVE-24274
>                 URL: https://issues.apache.org/jira/browse/HIVE-24274
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Krisztian Kasa
>            Assignee: Krisztian Kasa
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> Besides the way queries are currently rewritten to use materialized views in Hive this project provides an alternative:
> Compare the query text with the materialized views query text stored. If we found a match the original query's logical plan can be replaced by a scan on the materialized view.
> - Only materialized views which are enabled to rewrite can participate
> - Use existing *HiveMaterializedViewsRegistry* through *Hive* object by adding a lookup method by query text.
> - There might be more than one materialized views which have the same query text. In this case chose the first valid one.
> - Validation can be done by calling *Hive.validateMaterializedViewsFromRegistry()*
> - The scope of this first patch is rewriting queries which entire text can be matched only.
> - Use the expanded query text (fully qualified column and table names) for comparing



--
This message was sent by Atlassian Jira
(v8.3.4#803005)