[ https://issues.apache.org/jira/browse/IMPALA-11390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Quanlong Huang updated IMPALA-11390:
------------------------------------
Description:
After IMPALA-10723, materialized views are treated as table instead of view in Impala. The DESCRIBE FORMATTED statement therefore is missing the view information.
Results in Impala
{noformat}
[localhost:21050] default> describe formatted functional_orc_def.mv1_alltypes_jointbl;
Query: describe formatted functional_orc_def.mv1_alltypes_jointbl
+------------------------------+------------------------------------------------------------------------------------------+----------------------+
| name | type | comment |
+------------------------------+------------------------------------------------------------------------------------------+----------------------+
| # col_name | data_type | comment |
| | NULL | NULL |
| c1 | smallint | NULL |
| c2 | boolean | NULL |
| c3 | bigint | NULL |
| min_bigint | bigint | NULL |
| min_zip | int | NULL |
| | NULL | NULL |
| # Detailed Table Information | NULL | NULL |
| Database: | functional_orc_def | NULL |
| OwnerType: | USER | NULL |
| Owner: | quanlong | NULL |
| CreateTime: | Sun May 22 11:16:57 CST 2022 | NULL |
| LastAccessTime: | UNKNOWN | NULL |
| Retention: | 0 | NULL |
| Location: | hdfs://localhost:20500/test-warehouse/managed/functional_orc_def.db/mv1_alltypes_jointbl | NULL |
| Table Type: | MATERIALIZED_VIEW | NULL |
| Table Parameters: | NULL | NULL |
| | bucketing_version | 2 |
| | materializedview.engine | tez |
| | numFiles | 1 |
| | totalSize | 555 |
| | transient_lastDdlTime | 1653189458 |
| | NULL | NULL |
| # Storage Information | NULL | NULL |
| SerDe Library: | org.apache.hadoop.hive.ql.io.orc.OrcSerde | NULL |
| InputFormat: | org.apache.hadoop.hive.ql.io.orc.OrcInputFormat | NULL |
| OutputFormat: | org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat | NULL |
| Compressed: | No | NULL |
| Num Buckets: | -1 | NULL |
| Bucket Columns: | [] | NULL |
| Sort Columns: | [] | NULL |
| | NULL | NULL |
| # Constraints | NULL | NULL |
+------------------------------+------------------------------------------------------------------------------------------+----------------------+
{noformat}
Results in Hive (the view definition is shown as Original Query):
{noformat}
+-----------------------------------------------+----------------------------------------------------+----------------------------------------------------+
| col_name | data_type | comment |
+-----------------------------------------------+----------------------------------------------------+----------------------------------------------------+
| c1 | smallint | |
| c2 | boolean | |
| c3 | bigint | |
| min_bigint | bigint | |
| min_zip | int | |
| | NULL | NULL |
| # Detailed Table Information | NULL | NULL |
| Database: | functional_orc_def | NULL |
| OwnerType: | USER | NULL |
| Owner: | quanlong | NULL |
| CreateTime: | Sun May 22 11:16:57 CST 2022 | NULL |
| LastAccessTime: | UNKNOWN | NULL |
| Retention: | 0 | NULL |
| Location: | hdfs://localhost:20500/test-warehouse/managed/functional_orc_def.db/mv1_alltypes_jointbl | NULL |
| Table Type: | MATERIALIZED_VIEW | NULL |
| Table Parameters: | NULL | NULL |
| | bucketing_version | 2 |
| | materializedview.engine | tez |
| | numFiles | 1 |
| | totalSize | 555 |
| | transient_lastDdlTime | 1653189458 |
| | NULL | NULL |
| # Storage Information | NULL | NULL |
| SerDe Library: | org.apache.hadoop.hive.ql.io.orc.OrcSerde | NULL |
| InputFormat: | org.apache.hadoop.hive.ql.io.orc.OrcInputFormat | NULL |
| OutputFormat: | org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat | NULL |
| Compressed: | No | NULL |
| Num Buckets: | -1 | NULL |
| Bucket Columns: | [] | NULL |
| Sort Columns: | [] | NULL |
| | NULL | NULL |
| # Materialized View Information | NULL | NULL |
| Original Query: | SELECT t1.smallint_col c1, t1.bool_col c2, | NULL |
| | | t2.test_id c3, min(t1.bigint_col) min_bigint, min(t2.test_zip) min_zip |
| | | FROM functional_orc_def.alltypes t1 |
| | | JOIN functional_orc_def.jointbl t2 ON (t1.id=t2.alltypes_id) |
| | | group by t1.smallint_col, t1.bool_col, t2.test_id |
| Expanded Query: | SELECT `t1`.`smallint_col` `c1`, `t1`.`bool_col` `c2`, | NULL |
| | | `t2`.`test_id` `c3`, min(`t1`.`bigint_col`) `min_bigint`, min(`t2`.`test_zip`) `min_zip` |
| | | FROM `functional_orc_def`.`alltypes` `t1` |
| | | JOIN `functional_orc_def`.`jointbl` `t2` ON (`t1`.`id`=`t2`.`alltypes_id`) |
| | | group by `t1`.`smallint_col`, `t1`.`bool_col`, `t2`.`test_id` |
| Rewrite Enabled: | Yes | NULL |
| Outdated for Rewriting: | No | NULL |
| | NULL | NULL |
| # Materialized View Source table information | NULL | NULL |
| Table name | I/U/D since last rebuild | NULL |
| hive.functional_orc_def.alltypes | 0/0/0 | NULL |
| hive.functional_orc_def.jointbl | 0/0/0 | NULL |
+-----------------------------------------------+----------------------------------------------------+----------------------------------------------------+{noformat}
CC [~amansinha]
was:
After IMPALA-10723, materialized views are treated as table instead of view in Impala. The DESCRIBE FORMATTED statement therefore is missing the view information.
Results in Impala
{noformat}
[localhost:21050] default> describe formatted functional_orc_def.mv1_alltypes_jointbl;
Query: describe formatted functional_orc_def.mv1_alltypes_jointbl
+------------------------------+------------------------------------------------------------------------------------------+----------------------+
| name | type | comment |
+------------------------------+------------------------------------------------------------------------------------------+----------------------+
| # col_name | data_type | comment |
| | NULL | NULL |
| c1 | smallint | NULL |
| c2 | boolean | NULL |
| c3 | bigint | NULL |
| min_bigint | bigint | NULL |
| min_zip | int | NULL |
| | NULL | NULL |
| # Detailed Table Information | NULL | NULL |
| Database: | functional_orc_def | NULL |
| OwnerType: | USER | NULL |
| Owner: | quanlong | NULL |
| CreateTime: | Sun May 22 11:16:57 CST 2022 | NULL |
| LastAccessTime: | UNKNOWN | NULL |
| Retention: | 0 | NULL |
| Location: | hdfs://localhost:20500/test-warehouse/managed/functional_orc_def.db/mv1_alltypes_jointbl | NULL |
| Table Type: | MATERIALIZED_VIEW | NULL |
| Table Parameters: | NULL | NULL |
| | bucketing_version | 2 |
| | materializedview.engine | tez |
| | numFiles | 1 |
| | totalSize | 555 |
| | transient_lastDdlTime | 1653189458 |
| | NULL | NULL |
| # Storage Information | NULL | NULL |
| SerDe Library: | org.apache.hadoop.hive.ql.io.orc.OrcSerde | NULL |
| InputFormat: | org.apache.hadoop.hive.ql.io.orc.OrcInputFormat | NULL |
| OutputFormat: | org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat | NULL |
| Compressed: | No | NULL |
| Num Buckets: | -1 | NULL |
| Bucket Columns: | [] | NULL |
| Sort Columns: | [] | NULL |
| | NULL | NULL |
| # Constraints | NULL | NULL |
+------------------------------+------------------------------------------------------------------------------------------+----------------------+
{noformat}
Results in Hive
{noformat}
+-----------------------------------------------+----------------------------------------------------+----------------------------------------------------+
| col_name | data_type | comment |
+-----------------------------------------------+----------------------------------------------------+----------------------------------------------------+
| c1 | smallint | |
| c2 | boolean | |
| c3 | bigint | |
| min_bigint | bigint | |
| min_zip | int | |
| | NULL | NULL |
| # Detailed Table Information | NULL | NULL |
| Database: | functional_orc_def | NULL |
| OwnerType: | USER | NULL |
| Owner: | quanlong | NULL |
| CreateTime: | Sun May 22 11:16:57 CST 2022 | NULL |
| LastAccessTime: | UNKNOWN | NULL |
| Retention: | 0 | NULL |
| Location: | hdfs://localhost:20500/test-warehouse/managed/functional_orc_def.db/mv1_alltypes_jointbl | NULL |
| Table Type: | MATERIALIZED_VIEW | NULL |
| Table Parameters: | NULL | NULL |
| | bucketing_version | 2 |
| | materializedview.engine | tez |
| | numFiles | 1 |
| | totalSize | 555 |
| | transient_lastDdlTime | 1653189458 |
| | NULL | NULL |
| # Storage Information | NULL | NULL |
| SerDe Library: | org.apache.hadoop.hive.ql.io.orc.OrcSerde | NULL |
| InputFormat: | org.apache.hadoop.hive.ql.io.orc.OrcInputFormat | NULL |
| OutputFormat: | org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat | NULL |
| Compressed: | No | NULL |
| Num Buckets: | -1 | NULL |
| Bucket Columns: | [] | NULL |
| Sort Columns: | [] | NULL |
| | NULL | NULL |
| # Materialized View Information | NULL | NULL |
| Original Query: | SELECT t1.smallint_col c1, t1.bool_col c2, | NULL |
| | | t2.test_id c3, min(t1.bigint_col) min_bigint, min(t2.test_zip) min_zip |
| | | FROM functional_orc_def.alltypes t1 |
| | | JOIN functional_orc_def.jointbl t2 ON (t1.id=t2.alltypes_id) |
| | | group by t1.smallint_col, t1.bool_col, t2.test_id |
| Expanded Query: | SELECT `t1`.`smallint_col` `c1`, `t1`.`bool_col` `c2`, | NULL |
| | | `t2`.`test_id` `c3`, min(`t1`.`bigint_col`) `min_bigint`, min(`t2`.`test_zip`) `min_zip` |
| | | FROM `functional_orc_def`.`alltypes` `t1` |
| | | JOIN `functional_orc_def`.`jointbl` `t2` ON (`t1`.`id`=`t2`.`alltypes_id`) |
| | | group by `t1`.`smallint_col`, `t1`.`bool_col`, `t2`.`test_id` |
| Rewrite Enabled: | Yes | NULL |
| Outdated for Rewriting: | No | NULL |
| | NULL | NULL |
| # Materialized View Source table information | NULL | NULL |
| Table name | I/U/D since last rebuild | NULL |
| hive.functional_orc_def.alltypes | 0/0/0 | NULL |
| hive.functional_orc_def.jointbl | 0/0/0 | NULL |
+-----------------------------------------------+----------------------------------------------------+----------------------------------------------------+{noformat}
CC [~amansinha]
> Describe formatted statement on materialized view should show the view definition
> ---------------------------------------------------------------------------------
>
> Key: IMPALA-11390
> URL: https://issues.apache.org/jira/browse/IMPALA-11390
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Quanlong Huang
> Priority: Major
>
> After IMPALA-10723, materialized views are treated as table instead of view in Impala. The DESCRIBE FORMATTED statement therefore is missing the view information.
> Results in Impala
> {noformat}
> [localhost:21050] default> describe formatted functional_orc_def.mv1_alltypes_jointbl;
> Query: describe formatted functional_orc_def.mv1_alltypes_jointbl
> +------------------------------+------------------------------------------------------------------------------------------+----------------------+
> | name | type | comment |
> +------------------------------+------------------------------------------------------------------------------------------+----------------------+
> | # col_name | data_type | comment |
> | | NULL | NULL |
> | c1 | smallint | NULL |
> | c2 | boolean | NULL |
> | c3 | bigint | NULL |
> | min_bigint | bigint | NULL |
> | min_zip | int | NULL |
> | | NULL | NULL |
> | # Detailed Table Information | NULL | NULL |
> | Database: | functional_orc_def | NULL |
> | OwnerType: | USER | NULL |
> | Owner: | quanlong | NULL |
> | CreateTime: | Sun May 22 11:16:57 CST 2022 | NULL |
> | LastAccessTime: | UNKNOWN | NULL |
> | Retention: | 0 | NULL |
> | Location: | hdfs://localhost:20500/test-warehouse/managed/functional_orc_def.db/mv1_alltypes_jointbl | NULL |
> | Table Type: | MATERIALIZED_VIEW | NULL |
> | Table Parameters: | NULL | NULL |
> | | bucketing_version | 2 |
> | | materializedview.engine | tez |
> | | numFiles | 1 |
> | | totalSize | 555 |
> | | transient_lastDdlTime | 1653189458 |
> | | NULL | NULL |
> | # Storage Information | NULL | NULL |
> | SerDe Library: | org.apache.hadoop.hive.ql.io.orc.OrcSerde | NULL |
> | InputFormat: | org.apache.hadoop.hive.ql.io.orc.OrcInputFormat | NULL |
> | OutputFormat: | org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat | NULL |
> | Compressed: | No | NULL |
> | Num Buckets: | -1 | NULL |
> | Bucket Columns: | [] | NULL |
> | Sort Columns: | [] | NULL |
> | | NULL | NULL |
> | # Constraints | NULL | NULL |
> +------------------------------+------------------------------------------------------------------------------------------+----------------------+
> {noformat}
> Results in Hive (the view definition is shown as Original Query):
> {noformat}
> +-----------------------------------------------+----------------------------------------------------+----------------------------------------------------+
> | col_name | data_type | comment |
> +-----------------------------------------------+----------------------------------------------------+----------------------------------------------------+
> | c1 | smallint | |
> | c2 | boolean | |
> | c3 | bigint | |
> | min_bigint | bigint | |
> | min_zip | int | |
> | | NULL | NULL |
> | # Detailed Table Information | NULL | NULL |
> | Database: | functional_orc_def | NULL |
> | OwnerType: | USER | NULL |
> | Owner: | quanlong | NULL |
> | CreateTime: | Sun May 22 11:16:57 CST 2022 | NULL |
> | LastAccessTime: | UNKNOWN | NULL |
> | Retention: | 0 | NULL |
> | Location: | hdfs://localhost:20500/test-warehouse/managed/functional_orc_def.db/mv1_alltypes_jointbl | NULL |
> | Table Type: | MATERIALIZED_VIEW | NULL |
> | Table Parameters: | NULL | NULL |
> | | bucketing_version | 2 |
> | | materializedview.engine | tez |
> | | numFiles | 1 |
> | | totalSize | 555 |
> | | transient_lastDdlTime | 1653189458 |
> | | NULL | NULL |
> | # Storage Information | NULL | NULL |
> | SerDe Library: | org.apache.hadoop.hive.ql.io.orc.OrcSerde | NULL |
> | InputFormat: | org.apache.hadoop.hive.ql.io.orc.OrcInputFormat | NULL |
> | OutputFormat: | org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat | NULL |
> | Compressed: | No | NULL |
> | Num Buckets: | -1 | NULL |
> | Bucket Columns: | [] | NULL |
> | Sort Columns: | [] | NULL |
> | | NULL | NULL |
> | # Materialized View Information | NULL | NULL |
> | Original Query: | SELECT t1.smallint_col c1, t1.bool_col c2, | NULL |
> | | | t2.test_id c3, min(t1.bigint_col) min_bigint, min(t2.test_zip) min_zip |
> | | | FROM functional_orc_def.alltypes t1 |
> | | | JOIN functional_orc_def.jointbl t2 ON (t1.id=t2.alltypes_id) |
> | | | group by t1.smallint_col, t1.bool_col, t2.test_id |
> | Expanded Query: | SELECT `t1`.`smallint_col` `c1`, `t1`.`bool_col` `c2`, | NULL |
> | | | `t2`.`test_id` `c3`, min(`t1`.`bigint_col`) `min_bigint`, min(`t2`.`test_zip`) `min_zip` |
> | | | FROM `functional_orc_def`.`alltypes` `t1` |
> | | | JOIN `functional_orc_def`.`jointbl` `t2` ON (`t1`.`id`=`t2`.`alltypes_id`) |
> | | | group by `t1`.`smallint_col`, `t1`.`bool_col`, `t2`.`test_id` |
> | Rewrite Enabled: | Yes | NULL |
> | Outdated for Rewriting: | No | NULL |
> | | NULL | NULL |
> | # Materialized View Source table information | NULL | NULL |
> | Table name | I/U/D since last rebuild | NULL |
> | hive.functional_orc_def.alltypes | 0/0/0 | NULL |
> | hive.functional_orc_def.jointbl | 0/0/0 | NULL |
> +-----------------------------------------------+----------------------------------------------------+----------------------------------------------------+{noformat}
> CC [~amansinha]
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org