[jira] [Resolved] (IMPALA-8734) ALTER TABLE ... SET TBLPROPERTIES doesn't propagate until Invalidate Metadata

["Fredy Wijaya (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Fredy Wijaya resolved IMPALA-8734.
----------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 3.3.0

> ALTER TABLE ... SET TBLPROPERTIES doesn't propagate until Invalidate Metadata
> -----------------------------------------------------------------------------
>
>                 Key: IMPALA-8734
>                 URL: https://issues.apache.org/jira/browse/IMPALA-8734
>             Project: IMPALA
>          Issue Type: Improvement
>    Affects Versions: Impala 3.3.0
>            Reporter: Vincent Tran
>            Assignee: Fredy Wijaya
>            Priority: Major
>             Fix For: Impala 3.3.0
>
>
> More particularly, if one needs to change the data representation of NULL in the tblproperties, an INVALIDATE METADATA is needed for table to be readable with the new `serialization.null.format`
> To setup, create a table and dfs -put some data files into hdfs with some null strings:
> {noformat}
> [localhost:21000] default> create table v.t1(c1 string);
> Query: create table v.t1(c1 string)
> +-------------------------+
> | summary |
> +-------------------------+
> | Table has been created. |
> +-------------------------+
> Fetched 1 row(s) in 0.51s
> [localhost:21000] default> show create table v.t1;
> Query: show create table v.t1
> +----------------------------------------------------------+
> | result |
> +----------------------------------------------------------+
> | CREATE TABLE v.t1 ( |
> | c1 STRING |
> | ) |
> | STORED AS TEXTFILE |
> | LOCATION 'hdfs://localhost:20500/test-warehouse/v.db/t1' |
> | |
> +----------------------------------------------------------+
> Fetched 1 row(s) in 4.38s
> [localhost:21000] default> !printf '\nnull\n' > f;
> --------
> Executed in 0.00s
> [localhost:21000] default> !hdfs dfs -put f /test-warehouse/v.db/t1/f1;
> --------
> Executed in 1.98s{noformat}
>  
> Repro:
> {noformat}
> [localhost:21000] default> select * from v.t1;
> Query: select * from v.t1
> Query submitted at: 2019-06-27 12:48:13 (Coordinator: http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at: http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=8b4532852b10613e:e9e4847500000000
> +------+
> | c1 |
> +------+
> | |
> | null |
> +------+
> Fetched 2 row(s) in 4.77s
> [localhost:21000] default> alter table v.t1 set tblproperties("serialization.null.format"="null");
> Query: alter table v.t1 set tblproperties("serialization.null.format"="null")
> +----------------+
> | summary |
> +----------------+
> | Updated table. |
> +----------------+
> Fetched 1 row(s) in 0.05s
> [localhost:21000] default> select * from v.t1;
> Query: select * from v.t1
> Query submitted at: 2019-06-27 12:48:39 (Coordinator: http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at: http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=e54749def520fa97:9800728300000000
> +------+
> | c1 |
> +------+
> | |
> | null |
> +------+
> Fetched 2 row(s) in 0.11s
> [localhost:21000] default> invalidate metadata v.t1;
> Query: invalidate metadata v.t1
> Query submitted at: 2019-06-27 12:48:50 (Coordinator: http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at: http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=9d41e48bc8d5bad3:8020514f00000000
> Fetched 0 row(s) in 0.01s
> [localhost:21000] default> select * from v.t1;
> Query: select * from v.t1
> Query submitted at: 2019-06-27 12:48:51 (Coordinator: http://blackbox.vpc.cloudera.com:25000)
> Query progress can be monitored at: http://blackbox.vpc.cloudera.com:25000/query_plan?query_id=0446c2c65af3400a:3e3a484900000000
> +------+
> | c1 |
> +------+
> | |
> | NULL |
> +------+
> Fetched 2 row(s) in 4.48s
> [localhost:21000] default>{noformat}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)