[jira] [Created] (IMPALA-8734) ALTER TABLE ... SET TBLPROPERTIES doesn't propagate until Invalidate Metadata

["Vincent Tran (JIRA)" <ji...@apache.org>]

Vincent Tran created IMPALA-8734:
------------------------------------

             Summary: ALTER TABLE ... SET TBLPROPERTIES doesn't propagate until Invalidate Metadata
                 Key: IMPALA-8734
                 URL: https://issues.apache.org/jira/browse/IMPALA-8734
             Project: IMPALA
          Issue Type: Improvement
            Reporter: Vincent Tran


More particularly, if one needs to change the data representation of NULL in the tblproperties, an IM is needed for table to be readable with the new `serialization.null.format`

To setup, create a table and dfs -put some data files into hdfs with some null strings:
{noformat}
[localhost:21000] default> create table v.t1(c1 string);
Query: create table v.t1(c1 string)
+-------------------------+
| summary |
+-------------------------+
| Table has been created. |
+-------------------------+
Fetched 1 row(s) in 0.51s
[localhost:21000] default> show create table v.t1;
Query: show create table v.t1
+----------------------------------------------------------+
| result |
+----------------------------------------------------------+
| CREATE TABLE v.t1 ( |
| c1 STRING |
| ) |
| STORED AS TEXTFILE |
| LOCATION 'hdfs://localhost:20500/test-warehouse/v.db/t1' |
| |
+----------------------------------------------------------+
Fetched 1 row(s) in 4.38s
[localhost:21000] default> !printf '\nnull\n' > f;
--------
Executed in 0.00s
[localhost:21000] default> !hdfs dfs -put f /test-warehouse/v.db/t1/f1;
--------
Executed in 1.98s{noformat}
 

Repro:
{noformat}
[localhost:21000] default> select * from v.t1;
Query: select * from v.t1
Query submitted at: 2019-06-27 12:48:13 (Coordinator: http://blackbox.vpc.cloudera.com:25000)
Query progress can be monitored at: http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=8b4532852b10613e:e9e4847500000000
+------+
| c1 |
+------+
| |
| null |
+------+
Fetched 2 row(s) in 4.77s
[localhost:21000] default> alter table v.t1 set tblproperties("serialization.null.format"="null");
Query: alter table v.t1 set tblproperties("serialization.null.format"="null")
+----------------+
| summary |
+----------------+
| Updated table. |
+----------------+
Fetched 1 row(s) in 0.05s
[localhost:21000] default> select * from v.t1;
Query: select * from v.t1
Query submitted at: 2019-06-27 12:48:39 (Coordinator: http://blackbox.vpc.cloudera.com:25000)
Query progress can be monitored at: http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=e54749def520fa97:9800728300000000
+------+
| c1 |
+------+
| |
| null |
+------+
Fetched 2 row(s) in 0.11s
[localhost:21000] default> invalidate metadata v.t1;
Query: invalidate metadata v.t1
Query submitted at: 2019-06-27 12:48:50 (Coordinator: http://blackbox.vpc.cloudera.com:25000)
Query progress can be monitored at: http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=9d41e48bc8d5bad3:8020514f00000000
Fetched 0 row(s) in 0.01s
[localhost:21000] default> select * from v.t1;
Query: select * from v.t1
Query submitted at: 2019-06-27 12:48:51 (Coordinator: http://blackbox.vpc.cloudera.com:25000)
Query progress can be monitored at: http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=0446c2c65af3400a:3e3a484900000000
+------+
| c1 |
+------+
| |
| NULL |
+------+
Fetched 2 row(s) in 4.48s
[localhost:21000] default>{noformat}
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)