You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Venugopal Reddy K (Jira)" <ji...@apache.org> on 2024/06/07 09:17:00 UTC
[jira] [Updated] (IMPALA-13141) Partition transactional table is not updated on alter partition when hms_event_incremental_refresh_transactional_table is disabled
[ https://issues.apache.org/jira/browse/IMPALA-13141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Venugopal Reddy K updated IMPALA-13141:
---------------------------------------
Description:
Partition transactional table is not updated on alter partition when hms_event_incremental_refresh_transactional_table is disabled.
*Observations:*
1. In case of AlterPartitionEvent, this issue occurs when hms_event_incremental_refresh_transactional_table is disabled.
2. In case of BatchPartitionEvent(when. more than 1
AlterPartitionEvent are batched together), this issue occurs without disabling hms_event_incremental_refresh_transactional_table.
*Steps to reproduce:*
1. Create partitioned table and add some partitions from hive:
Note: This step can be from impala too.
{code:java}
0: jdbc:hive2://localhost:11050> create table s(i int, j int, p int);
0: jdbc:hive2://localhost:11050> insert into s values(1,10,100),(2,20,200);
{code}
{code:java}
0: jdbc:hive2://localhost:11050> create table test1(i int, j int) partitioned by(p int) tblproperties ('transactional'='true', 'transactional_properties'='insert_only');
0: jdbc:hive2://localhost:11050> set hive.exec.dynamic.partition.mode=nonstrict;
0: jdbc:hive2://localhost:11050> insert into test partition(p) select * from s;
0: jdbc:hive2://localhost:11050> show partitions test;
+------------+
| partition |
+------------+
| p=100 |
| p=200 |
+------------+
0: jdbc:hive2://localhost:11050> desc formatted test partition(p=100);
+-----------------------------------+----------------------------------------------------+-----------------------+
| col_name | data_type | comment |
+-----------------------------------+----------------------------------------------------+-----------------------+
| i | int | |
| j | int | |
| | NULL | NULL |
| # Partition Information | NULL | NULL |
| # col_name | data_type | comment |
| p | int | |
| | NULL | NULL |
| # Detailed Partition Information | NULL | NULL |
| Partition Value: | [100] | NULL |
| Database: | default | NULL |
| Table: | test | NULL |
| CreateTime: | Fri Jun 07 14:21:17 IST 2024 | NULL |
| LastAccessTime: | UNKNOWN | NULL |
| Location: | hdfs://localhost:20500/test-warehouse/managed/test/p=100 | NULL |
| Partition Parameters: | NULL | NULL |
| | numFiles | 1 |
| | totalSize | 5 |
| | transient_lastDdlTime | 1717750277 |
| | NULL | NULL |
| # Storage Information | NULL | NULL |
| SerDe Library: | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL |
| InputFormat: | org.apache.hadoop.mapred.TextInputFormat | NULL |
| OutputFormat: | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL |
| Compressed: | No | NULL |
| Num Buckets: | -1 | NULL |
| Bucket Columns: | [] | NULL |
| Sort Columns: | [] | NULL |
| Storage Desc Params: | NULL | NULL |
| | serialization.format | 1 |
+-----------------------------------+----------------------------------------------------+-----------------------+
0: jdbc:hive2://localhost:11050>
{code}
2. Check for the partitions and its locations from impala shell:
{code:java}
[localhost:21050] default> show partitions test;
Query: show partitions test
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
| p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location | EC Policy |
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
| 100 | -1 | 1 | 5B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://localhost:20500/test-warehouse/managed/test/p=100 | NONE |
| 200 | -1 | 1 | 5B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://localhost:20500/test-warehouse/managed/test/p=200 | NONE |
| Total | -1 | 2 | 10B | 0B | | | | | |
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
Fetched 3 row(s) in 4.43s
[localhost:21050] default>
[localhost:21050] default> select * from test;
Query: select * from test
Query submitted at: 2024-06-07 14:28:13 (Coordinator: http://kvenureddy-OptiPlex-7000:25000)
Query state can be monitored at: http://kvenureddy-OptiPlex-7000:25000/query_plan?query_id=024dff2843ae25bf:6d5792d100000000
+---+----+-----+
| i | j | p |
+---+----+-----+
| 2 | 20 | 200 |
| 1 | 10 | 100 |
+---+----+-----+
Fetched 2 row(s) in 0.34s
[localhost:21050] default> select * from test where p=100;
Query: select * from test where p=100
Query submitted at: 2024-06-07 14:28:19 (Coordinator: http://kvenureddy-OptiPlex-7000:25000)
Query state can be monitored at: http://kvenureddy-OptiPlex-7000:25000/query_plan?query_id=fa42921d722ecc8a:6e7e545400000000
+---+----+-----+
| i | j | p |
+---+----+-----+
| 1 | 10 | 100 |
+---+----+-----+
Fetched 1 row(s) in 0.11s
[localhost:21050] default>
{code}
3. Alter partition location from hive:
{code:java}
0: jdbc:hive2://localhost:11050> alter table test1 partition(p=100) set location '/dummylocation100';
0: jdbc:hive2://localhost:11050> desc formatted test partition(p=100);
+-----------------------------------+----------------------------------------------------+-----------------------+
| col_name | data_type | comment |
+-----------------------------------+----------------------------------------------------+-----------------------+
| i | int | |
| j | int | |
| | NULL | NULL |
| # Partition Information | NULL | NULL |
| # col_name | data_type | comment |
| p | int | |
| | NULL | NULL |
| # Detailed Partition Information | NULL | NULL |
| Partition Value: | [100] | NULL |
| Database: | default | NULL |
| Table: | test | NULL |
| CreateTime: | Fri Jun 07 14:21:17 IST 2024 | NULL |
| LastAccessTime: | UNKNOWN | NULL |
| Location: | hdfs://localhost:20500/dummylocation100 | NULL |
| Partition Parameters: | NULL | NULL |
| | last_modified_by | kvenureddy |
| | last_modified_time | 1717750893 |
| | numFiles | 1 |
| | totalSize | 5 |
| | transient_lastDdlTime | 1717750893 |
| | NULL | NULL |
| # Storage Information | NULL | NULL |
| SerDe Library: | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL |
| InputFormat: | org.apache.hadoop.mapred.TextInputFormat | NULL |
| OutputFormat: | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL |
| Compressed: | No | NULL |
| Num Buckets: | -1 | NULL |
| Bucket Columns: | [] | NULL |
| Sort Columns: | [] | NULL |
| Storage Desc Params: | NULL | NULL |
| | serialization.format | 1 |
+-----------------------------------+----------------------------------------------------+-----------------------+
0: jdbc:hive2://localhost:11050> {code}
4. Check for the partition location from impala shell. It is not updated.
{code:java}
[localhost:21050] default> show partitions test;
Query: show partitions test
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
| p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location | EC Policy |
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
| 100 | -1 | 1 | 5B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://localhost:20500/test-warehouse/managed/test/p=100 | NONE |
| 200 | -1 | 1 | 5B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://localhost:20500/test-warehouse/managed/test/p=200 | NONE |
| Total | -1 | 2 | 10B | 0B | | | | | |
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
Fetched 3 row(s) in 0.01s
[localhost:21050] default>
[localhost:21050] default> select * from test where p=100;
Query: select * from test where p=100
Query submitted at: 2024-06-07 14:35:06 (Coordinator: http://kvenureddy-OptiPlex-7000:25000)
Query state can be monitored at: http://kvenureddy-OptiPlex-7000:25000/query_plan?query_id=7042fda005972a2b:a82fd60700000000
+---+----+-----+
| i | j | p |
+---+----+-----+
| 1 | 10 | 100 |
+---+----+-----+
Fetched 1 row(s) in 0.11s
[localhost:21050] default>
{code}
was:
Partition transactional table is not updated on alter partition when hms_event_incremental_refresh_transactional_table is disabled.
*Observations:*
1. In case of AlterPartitionEvent, this issue occurs when hms_event_incremental_refresh_transactional_table is disabled.
2. In case of BatchPartitionEvent(when. more than 1
AlterPartitionEvent are batched together), this issue occurs without disabling hms_event_incremental_refresh_transactional_table.
*Steps to reproduce:*
1. Create partitioned table and add some partitions from hive:
Note: This step can be from impala too.
{code:java}
0: jdbc:hive2://localhost:11050> create table s(i int, j int, p int);
0: jdbc:hive2://localhost:11050> insert into s values(1,10,100),(2,20,200);
{code}
{code:java}
0: jdbc:hive2://localhost:11050> create table test1(i int, j int) partitioned by(p int) tblproperties ('transactional'='true', 'transactional_properties'='insert_only');
0: jdbc:hive2://localhost:11050> set hive.exec.dynamic.partition.mode=nonstrict;
0: jdbc:hive2://localhost:11050> insert into test partition(p) select * from s;
0: jdbc:hive2://localhost:11050> show partitions test;
+------------+
| partition |
+------------+
| p=100 |
| p=200 |
+------------+
0: jdbc:hive2://localhost:11050> desc formatted test partition(p=100);
+-----------------------------------+----------------------------------------------------+-----------------------+
| col_name | data_type | comment |
+-----------------------------------+----------------------------------------------------+-----------------------+
| i | int | |
| j | int | |
| | NULL | NULL |
| # Partition Information | NULL | NULL |
| # col_name | data_type | comment |
| p | int | |
| | NULL | NULL |
| # Detailed Partition Information | NULL | NULL |
| Partition Value: | [100] | NULL |
| Database: | default | NULL |
| Table: | test | NULL |
| CreateTime: | Fri Jun 07 14:21:17 IST 2024 | NULL |
| LastAccessTime: | UNKNOWN | NULL |
| Location: | hdfs://localhost:20500/test-warehouse/managed/test/p=100 | NULL |
| Partition Parameters: | NULL | NULL |
| | numFiles | 1 |
| | totalSize | 5 |
| | transient_lastDdlTime | 1717750277 |
| | NULL | NULL |
| # Storage Information | NULL | NULL |
| SerDe Library: | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL |
| InputFormat: | org.apache.hadoop.mapred.TextInputFormat | NULL |
| OutputFormat: | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL |
| Compressed: | No | NULL |
| Num Buckets: | -1 | NULL |
| Bucket Columns: | [] | NULL |
| Sort Columns: | [] | NULL |
| Storage Desc Params: | NULL | NULL |
| | serialization.format | 1 |
+-----------------------------------+----------------------------------------------------+-----------------------+
0: jdbc:hive2://localhost:11050>
{code}
2. Check for the partitions and its locations from impala shell:
{code:java}
[localhost:21050] default> show partitions test;
Query: show partitions test
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
| p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location | EC Policy |
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
| 100 | -1 | 1 | 5B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://localhost:20500/test-warehouse/managed/test/p=100 | NONE |
| 200 | -1 | 1 | 5B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://localhost:20500/test-warehouse/managed/test/p=200 | NONE |
| Total | -1 | 2 | 10B | 0B | | | | | |
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
Fetched 3 row(s) in 4.43s
[localhost:21050] default>
[localhost:21050] default> select * from test;
Query: select * from test
Query submitted at: 2024-06-07 14:28:13 (Coordinator: http://kvenureddy-OptiPlex-7000:25000)
Query state can be monitored at: http://kvenureddy-OptiPlex-7000:25000/query_plan?query_id=024dff2843ae25bf:6d5792d100000000
+---+----+-----+
| i | j | p |
+---+----+-----+
| 2 | 20 | 200 |
| 1 | 10 | 100 |
+---+----+-----+
Fetched 2 row(s) in 0.34s
[localhost:21050] default> select * from test where p=100;
Query: select * from test where p=100
Query submitted at: 2024-06-07 14:28:19 (Coordinator: http://kvenureddy-OptiPlex-7000:25000)
Query state can be monitored at: http://kvenureddy-OptiPlex-7000:25000/query_plan?query_id=fa42921d722ecc8a:6e7e545400000000
+---+----+-----+
| i | j | p |
+---+----+-----+
| 1 | 10 | 100 |
+---+----+-----+
Fetched 1 row(s) in 0.11s
[localhost:21050] default>
{code}
3. Alter partition location from hive:
{code:java}
0: jdbc:hive2://localhost:11050> alter table test1 partition(p=100) set location '/dummylocation100';
0: jdbc:hive2://localhost:11050> desc formatted test partition(p=100);
+-----------------------------------+----------------------------------------------------+-----------------------+
| col_name | data_type | comment |
+-----------------------------------+----------------------------------------------------+-----------------------+
| i | int | |
| j | int | |
| | NULL | NULL |
| # Partition Information | NULL | NULL |
| # col_name | data_type | comment |
| p | int | |
| | NULL | NULL |
| # Detailed Partition Information | NULL | NULL |
| Partition Value: | [100] | NULL |
| Database: | default | NULL |
| Table: | test | NULL |
| CreateTime: | Fri Jun 07 14:21:17 IST 2024 | NULL |
| LastAccessTime: | UNKNOWN | NULL |
| Location: | hdfs://localhost:20500/dummylocation100 | NULL |
| Partition Parameters: | NULL | NULL |
| | last_modified_by | kvenureddy |
| | last_modified_time | 1717750893 |
| | numFiles | 1 |
| | totalSize | 5 |
| | transient_lastDdlTime | 1717750893 |
| | NULL | NULL |
| # Storage Information | NULL | NULL |
| SerDe Library: | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL |
| InputFormat: | org.apache.hadoop.mapred.TextInputFormat | NULL |
| OutputFormat: | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL |
| Compressed: | No | NULL |
| Num Buckets: | -1 | NULL |
| Bucket Columns: | [] | NULL |
| Sort Columns: | [] | NULL |
| Storage Desc Params: | NULL | NULL |
| | serialization.format | 1 |
+-----------------------------------+----------------------------------------------------+-----------------------+
0: jdbc:hive2://localhost:11050> {code}
4. Check for the partition location from impala shell. It is not updated.
{code:java}
[localhost:21050] default> show partitions test;
Query: show partitions test
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
| p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location | EC Policy |
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
| 100 | -1 | 1 | 5B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://localhost:20500/test-warehouse/managed/test/p=100 | NONE |
| 200 | -1 | 1 | 5B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://localhost:20500/test-warehouse/managed/test/p=200 | NONE |
| Total | -1 | 2 | 10B | 0B | | | | | |
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
Fetched 3 row(s) in 0.01s
[localhost:21050] default>
[localhost:21050] default> select * from test where p=100;
Query: select * from test where p=100
Query submitted at: 2024-06-07 14:35:06 (Coordinator: http://kvenureddy-OptiPlex-7000:25000)
Query state can be monitored at: http://kvenureddy-OptiPlex-7000:25000/query_plan?query_id=7042fda005972a2b:a82fd60700000000
+---+----+-----+
| i | j | p |
+---+----+-----+
| 1 | 10 | 100 |
+---+----+-----+
Fetched 1 row(s) in 0.11s
[localhost:21050] default>
{code}
> Partition transactional table is not updated on alter partition when hms_event_incremental_refresh_transactional_table is disabled
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-13141
> URL: https://issues.apache.org/jira/browse/IMPALA-13141
> Project: IMPALA
> Issue Type: Bug
> Reporter: Venugopal Reddy K
> Priority: Major
>
> Partition transactional table is not updated on alter partition when hms_event_incremental_refresh_transactional_table is disabled.
> *Observations:*
> 1. In case of AlterPartitionEvent, this issue occurs when hms_event_incremental_refresh_transactional_table is disabled.
> 2. In case of BatchPartitionEvent(when. more than 1
> AlterPartitionEvent are batched together), this issue occurs without disabling hms_event_incremental_refresh_transactional_table.
> *Steps to reproduce:*
> 1. Create partitioned table and add some partitions from hive:
> Note: This step can be from impala too.
> {code:java}
> 0: jdbc:hive2://localhost:11050> create table s(i int, j int, p int);
> 0: jdbc:hive2://localhost:11050> insert into s values(1,10,100),(2,20,200);
> {code}
> {code:java}
> 0: jdbc:hive2://localhost:11050> create table test1(i int, j int) partitioned by(p int) tblproperties ('transactional'='true', 'transactional_properties'='insert_only');
> 0: jdbc:hive2://localhost:11050> set hive.exec.dynamic.partition.mode=nonstrict;
> 0: jdbc:hive2://localhost:11050> insert into test partition(p) select * from s;
> 0: jdbc:hive2://localhost:11050> show partitions test;
> +------------+
> | partition |
> +------------+
> | p=100 |
> | p=200 |
> +------------+
> 0: jdbc:hive2://localhost:11050> desc formatted test partition(p=100);
> +-----------------------------------+----------------------------------------------------+-----------------------+
> | col_name | data_type | comment |
> +-----------------------------------+----------------------------------------------------+-----------------------+
> | i | int | |
> | j | int | |
> | | NULL | NULL |
> | # Partition Information | NULL | NULL |
> | # col_name | data_type | comment |
> | p | int | |
> | | NULL | NULL |
> | # Detailed Partition Information | NULL | NULL |
> | Partition Value: | [100] | NULL |
> | Database: | default | NULL |
> | Table: | test | NULL |
> | CreateTime: | Fri Jun 07 14:21:17 IST 2024 | NULL |
> | LastAccessTime: | UNKNOWN | NULL |
> | Location: | hdfs://localhost:20500/test-warehouse/managed/test/p=100 | NULL |
> | Partition Parameters: | NULL | NULL |
> | | numFiles | 1 |
> | | totalSize | 5 |
> | | transient_lastDdlTime | 1717750277 |
> | | NULL | NULL |
> | # Storage Information | NULL | NULL |
> | SerDe Library: | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL |
> | InputFormat: | org.apache.hadoop.mapred.TextInputFormat | NULL |
> | OutputFormat: | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL |
> | Compressed: | No | NULL |
> | Num Buckets: | -1 | NULL |
> | Bucket Columns: | [] | NULL |
> | Sort Columns: | [] | NULL |
> | Storage Desc Params: | NULL | NULL |
> | | serialization.format | 1 |
> +-----------------------------------+----------------------------------------------------+-----------------------+
> 0: jdbc:hive2://localhost:11050>
> {code}
> 2. Check for the partitions and its locations from impala shell:
> {code:java}
> [localhost:21050] default> show partitions test;
> Query: show partitions test
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
> | p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location | EC Policy |
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
> | 100 | -1 | 1 | 5B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://localhost:20500/test-warehouse/managed/test/p=100 | NONE |
> | 200 | -1 | 1 | 5B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://localhost:20500/test-warehouse/managed/test/p=200 | NONE |
> | Total | -1 | 2 | 10B | 0B | | | | | |
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
> Fetched 3 row(s) in 4.43s
> [localhost:21050] default>
> [localhost:21050] default> select * from test;
> Query: select * from test
> Query submitted at: 2024-06-07 14:28:13 (Coordinator: http://kvenureddy-OptiPlex-7000:25000)
> Query state can be monitored at: http://kvenureddy-OptiPlex-7000:25000/query_plan?query_id=024dff2843ae25bf:6d5792d100000000
> +---+----+-----+
> | i | j | p |
> +---+----+-----+
> | 2 | 20 | 200 |
> | 1 | 10 | 100 |
> +---+----+-----+
> Fetched 2 row(s) in 0.34s
> [localhost:21050] default> select * from test where p=100;
> Query: select * from test where p=100
> Query submitted at: 2024-06-07 14:28:19 (Coordinator: http://kvenureddy-OptiPlex-7000:25000)
> Query state can be monitored at: http://kvenureddy-OptiPlex-7000:25000/query_plan?query_id=fa42921d722ecc8a:6e7e545400000000
> +---+----+-----+
> | i | j | p |
> +---+----+-----+
> | 1 | 10 | 100 |
> +---+----+-----+
> Fetched 1 row(s) in 0.11s
> [localhost:21050] default>
> {code}
> 3. Alter partition location from hive:
> {code:java}
> 0: jdbc:hive2://localhost:11050> alter table test1 partition(p=100) set location '/dummylocation100';
> 0: jdbc:hive2://localhost:11050> desc formatted test partition(p=100);
> +-----------------------------------+----------------------------------------------------+-----------------------+
> | col_name | data_type | comment |
> +-----------------------------------+----------------------------------------------------+-----------------------+
> | i | int | |
> | j | int | |
> | | NULL | NULL |
> | # Partition Information | NULL | NULL |
> | # col_name | data_type | comment |
> | p | int | |
> | | NULL | NULL |
> | # Detailed Partition Information | NULL | NULL |
> | Partition Value: | [100] | NULL |
> | Database: | default | NULL |
> | Table: | test | NULL |
> | CreateTime: | Fri Jun 07 14:21:17 IST 2024 | NULL |
> | LastAccessTime: | UNKNOWN | NULL |
> | Location: | hdfs://localhost:20500/dummylocation100 | NULL |
> | Partition Parameters: | NULL | NULL |
> | | last_modified_by | kvenureddy |
> | | last_modified_time | 1717750893 |
> | | numFiles | 1 |
> | | totalSize | 5 |
> | | transient_lastDdlTime | 1717750893 |
> | | NULL | NULL |
> | # Storage Information | NULL | NULL |
> | SerDe Library: | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL |
> | InputFormat: | org.apache.hadoop.mapred.TextInputFormat | NULL |
> | OutputFormat: | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL |
> | Compressed: | No | NULL |
> | Num Buckets: | -1 | NULL |
> | Bucket Columns: | [] | NULL |
> | Sort Columns: | [] | NULL |
> | Storage Desc Params: | NULL | NULL |
> | | serialization.format | 1 |
> +-----------------------------------+----------------------------------------------------+-----------------------+
> 0: jdbc:hive2://localhost:11050> {code}
> 4. Check for the partition location from impala shell. It is not updated.
> {code:java}
> [localhost:21050] default> show partitions test;
> Query: show partitions test
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
> | p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location | EC Policy |
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
> | 100 | -1 | 1 | 5B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://localhost:20500/test-warehouse/managed/test/p=100 | NONE |
> | 200 | -1 | 1 | 5B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://localhost:20500/test-warehouse/managed/test/p=200 | NONE |
> | Total | -1 | 2 | 10B | 0B | | | | | |
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------+-----------+
> Fetched 3 row(s) in 0.01s
> [localhost:21050] default>
> [localhost:21050] default> select * from test where p=100;
> Query: select * from test where p=100
> Query submitted at: 2024-06-07 14:35:06 (Coordinator: http://kvenureddy-OptiPlex-7000:25000)
> Query state can be monitored at: http://kvenureddy-OptiPlex-7000:25000/query_plan?query_id=7042fda005972a2b:a82fd60700000000
> +---+----+-----+
> | i | j | p |
> +---+----+-----+
> | 1 | 10 | 100 |
> +---+----+-----+
> Fetched 1 row(s) in 0.11s
> [localhost:21050] default>
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org