You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/10/20 04:44:00 UTC
[jira] [Commented] (IMPALA-10723) Allow basic querying and
computing stats on a materialized view
[ https://issues.apache.org/jira/browse/IMPALA-10723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430875#comment-17430875 ]
ASF subversion and git services commented on IMPALA-10723:
----------------------------------------------------------
Commit dc8f375986321f2d7c1ed750e5c02c2e6f8cb4e8 in impala's branch refs/heads/master from Yu-Wen Lai
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=dc8f375 ]
IMPALA-10959: Reload MV as ACID tables
We observed that the event processor is broken after receiving a
partition event for materialized views (MV). This is because we are
treating MV as view in Impala but Hive generates partition events for MV,
which breaks current event processor.
In this patch, we let partition events of MV follow the code path of ACID
tables to reload the view. In the long term, we will need IMPALA-10723 to
treat materialized view as a table.
Tests:
- manually testing
Change-Id: Ibeab8cc53ad47d24df8baba81e1ec6ea4c80a084
Reviewed-on: http://gerrit.cloudera.org:8080/17911
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Vihang Karajgaonkar <vi...@cloudera.com>
> Allow basic querying and computing stats on a materialized view
> ---------------------------------------------------------------
>
> Key: IMPALA-10723
> URL: https://issues.apache.org/jira/browse/IMPALA-10723
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Aman Sinha
> Assignee: Aman Sinha
> Priority: Major
>
> Currently, in Impala, a Materialized View (MV) created via Hive is visible through the metadata catalog and can be queried but the query is expanded to its corresponding view definition. This is incorrect because a materialized view is a regular (physical) table, not a view. Even though Impala does not support either creating MV or automatic rewriting to use MV, querying an MV directly should be allowed.
> Here's the current behavior:
> {noformat}
> [localhost:21050] functional> explain select * from materialized_view;
> Query: explain select * from materialized_view
> +------------------------------------------------------------------------------------+
> | Explain String |
> +------------------------------------------------------------------------------------+
> | Max Per-Host Resource Reservation: Memory=4.00MB Threads=2 |
> | Per-Host Resource Estimates: Memory=10MB |
> | WARNING: The following tables are missing relevant table and/or column statistics. |
> | functional.insert_only_transactional_table |
> | |
> | PLAN-ROOT SINK |
> | | |
> | 01:EXCHANGE [UNPARTITIONED] |
> | | |
> | 00:SCAN HDFS [functional.insert_only_transactional_table] |
> | HDFS partitions=1/1 files=0 size=0B |
> | row-size=4B cardinality=0 |
> +------------------------------------------------------------------------------------+
> {noformat}
> Note that the plan shows the scan of the underlying table instead of the materialized_view table. We should only be scanning the MV (including applying partition pruning, predicate pushdown etc.) and not treating this as a view.
> This JIRA is to enhance the frontend to recognize a materialized view as a table rather than a view. This will further allow commands such as COMPUTE STATS, DROP STATS, SHOW [TABLE | COLUMN] STATS to be run on the MV. One motivation for doing this is to allow an external frontend that supports automatic query rewrites using MVs to leverage the statistics on MVs.
> Since Impala is not creating the MV, we will need to block DML operations on the MV. Further, special handling needs to be done for Ranger authorization policies such that any column masking/row filtering policies defined on the source tables of the MV are taken into consideration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org