[jira] [Resolved] (CARBONDATA-4282) Issues with table having complex columns related to long string, SI, local dictionary

["Ajantha Bhat (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CARBONDATA-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ajantha Bhat resolved CARBONDATA-4282.
--------------------------------------
    Fix Version/s: 2.3.0
       Resolution: Fixed

> Issues with table having complex columns related to long string, SI, local dictionary
> -------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-4282
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-4282
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: SHREELEKHYA GAMPA
>            Priority: Major
>             Fix For: 2.3.0
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> *1. Insert/load fails after alter add complex column if table contains long string columns.*
>   [Steps] :-  
> DROP TABLE IF EXISTS alter_com;
> CREATE TABLE alter_com(intfield int,EDUCATED string ,rankk string ) STORED AS carbondata TBLPROPERTIES('inverted_index'='intField','sort_columns'='intField','TABLE_BLOCKSIZE'= '256 MB','TABLE_BLOCKLET_SIZE'='8','SORT_SCOPE'='no_sort','COLUMN_META_CACHE'='rankk','carbon.column.compressor'='gzip','long_string_columns'='rankk','table_page_size_inmb'='1');
> insert into alter_com values(1,'cse','xi'); select * from alter_com limit 1;
> ALTER TABLE alter_com ADD COLUMNS(map1 Map<short,int>, map2 Map<long,double>, map3 Map<decimal(3,2),string>, map4 Map<char(5),varchar(50)>, map5 Map<boolean,date>,map6 Map<string,timestamp>,map7 map<string,array<string>>, map8 map<string,struct<d:int, s:struct<im:string>>>); 
> ALTER TABLE alter_com SET TBLPROPERTIES('long_string_columns'='EDUCATED');
> insert into alter_com values(1,'ece','x', map(1,2),map(3,2.34), map(1.23,'hello'),map('abc','def'), map(true,'2017-02-01'),map('time','2018-02-01 02:00:00.0'),map('ph',array(1,2)), map('a',named_struct('d',23,'s',named_struct('im','sh'))));
> [Expected Result] :- insert/load should be success after alter add map column ,if table contains long string columns
> *2. create index on array of complex column (map/struct) throws null pointer exception instead of correct error message.*
> [Steps] :-
> drop table if exists strarmap1; create table strarmap1(id int,name string,str struct<intval:int,tm:timestamp,map1:map<int,string>>,arr array<map<string,timestamp>>) stored as carbondata tblproperties('inverted_index'='name','sort_columns'='name','TABLE_BLOCKSIZE'= '256 MB','TABLE_BLOCKLET_SIZE'='8','CACHE_LEVEL'='BLOCKLET'); load data inpath 'hdfs://hacluster/chetan/strarmap1.csv' into table strarmap1 options('fileheader'='id,name,str,arr','COMPLEX_DELIMITER_LEVEL_3'='#','COMPLEX_DELIMITER_LEVEL_2'='$','COMPLEX_DELIMITER_LEVEL_1'='&','BAD_RECORDS_ACTION'='FORCE'); CREATE INDEX index2 ON TABLE strarmap1 (arr) as 'carbondata' properties('sort_scope'='global_sort','global_sort_partitions'='3');
> [Expected Result] :- create index on array of map(string,timestamp) should thrown correct validation error message.
> [Actual Issue]:- create index on array of map(string,timestamp) throws null pointer exception instead of correct error message
> *3. alter table property local dictionary inlcude/exclude with newly added map column is failing.*
> [Steps] :-
> drop table if exists strarmap1; create table strarmap1(id int,name string,str struct<intval:int,tm:timestamp,map1:map<int,string>>,arr array<map<string,timestamp>>) stored as carbondata tblproperties('inverted_index'='name','sort_columns'='name','local_dictionary_enable'='false','local_dictionary_include'='map1','local_dictionary_exclude'='str,arr','local_dictionary_threshold'='1000');
> ALTER TABLE strarmap1 ADD COLUMNS(map1 Map<short,int>, map2 Map<long,double>, map3 Map<decimal(3,2),string>, map4 Map<char(5),varchar(50)>, map5 Map<boolean,date>,map6 Map<string,timestamp>,map7 map<string,array<string>>, map8 map<string,struct<d:int, s:struct<im:string>>>); ALTER TABLE strarmap1 SET TBLPROPERTIES('LOCAL_DICTIONARY_ENABLE'='true','local_dictionary_include'='map4','local_dictionary_threshold'='1000');
> [Expected Result] :- alter table property local dictionary inlcude/exclude with newly added map column should be success
> [Actual Issue]:- alter table property local dictionary inlcude/exclude with newly added map column is failing



--
This message was sent by Atlassian Jira
(v8.3.4#803005)