[jira] [Created] (HIVE-13515) Semantics of TypeInfoUtils.areColTypesCompatible changed which also changed the semantics of hive.metastore.disallow.incompatible.col.type.changes of

["Matt McCline (JIRA)" <ji...@apache.org>]

Matt McCline created HIVE-13515:
-----------------------------------

             Summary: Semantics of TypeInfoUtils.areColTypesCompatible changed which also changed the semantics of hive.metastore.disallow.incompatible.col.type.changes of 
                 Key: HIVE-13515
                 URL: https://issues.apache.org/jira/browse/HIVE-13515
             Project: Hive
          Issue Type: Bug
          Components: Hive
    Affects Versions: 2.1.0
            Reporter: Matt McCline
            Assignee: Ashutosh Chauhan
            Priority: Critical



I had this test case in my new Schema Evolution patch (not checked in yet) that verifies STRING_GROUP to DECIMAL conversion:

{code}
-- SUBSECTION: ALTER TABLE CHANGE COLUMNS for STRING_GROUP -> DECIMAL: (STRING, CHAR, VARCHAR)
--
CREATE TABLE table_change_string_group_decimal(insert_num int, c1 STRING, c2 CHAR(50), c3 VARCHAR(50), b STRING);

insert into table table_change_string_group_decimal
    values (1, '753.7028', '753.7028', '753.7028', 'original'),
           (2, '-3651.672121', '-3651.672121', '-3651.672121', 'original'),
           (3, '-29.0764', '-29.0764', '-29.0764', 'original'),
           (4, '-10.3', '-10.3', '-10.3', 'original');

-- Table-Non-Cascade CHANGE COLUMNS ...
alter table table_change_string_group_decimal replace columns (insert_num int, c1 decimal(38,18), c2 decimal(38,18), c3 decimal(38,18), b STRING);

insert into table table_change_string_group_decimal
     values (5, 30.774, 30.774, 30.774, 'new'),
            (6, 20.31, 20.31, 20.31, 'new'),
            (7, 46114.284799488, 46114.284799488, 46114.284799488, 'new'),
            (8, -66475.561431, -66475.561431, -66475.561431, 'new');

insert into table table_change_string_group_decimal
     values (9, 17808.963785, 17808.963785, 17808.963785, 'new'),
            (10, 9250340.75 , 9250340.75 , 9250340.75 , 'new');

explain
select insert_num,c1,c2,c3,b from table_change_string_group_decimal order by insert_num;

select insert_num,c1,c2,c3,b from table_change_string_group_decimal order by insert_num;

drop table table_change_string_group_decimal;
{code}

that now fails on master (I suspect because of HIVE-13380 Decimal should have lower precedence than double in type hierachy) when:

{code}
set hive.metastore.disallow.incompatible.col.type.changes=true;
{code}

with this call stack:
{code}
Caused by: InvalidOperationException(message:The following columns have types incompatible with the existing columns in their respective positions :
c1,c2,c3)
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.throwExceptionIfIncompatibleColTypeChange(MetaStoreUtils.java:599)
	at org.apache.hadoop.hive.metastore.HiveAlterHandler.alterTable(HiveAlterHandler.java:132)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table_core(HiveMetaStore.java:3502)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table_with_environment_context(HiveMetaStore.java:3473)
{code}

Key point is MetaStoreUtils.throwExceptionIfIncompatibleColTypeChange is calling TypeInfoUtils.areColTypesCompatible which changed

-    // Allow implicit String to Decimal conversion
-    if (fromPg == PrimitiveGrouping.STRING_GROUP && to == PrimitiveCategory.DECIMAL) {
-      return true;
-    }

Will not current customers that have scripts with hive.metastore.disallow.incompatible.col.type.changes=true that do Schema Evolution get bitten?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)