You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Alexander Behm (JIRA)" <ji...@apache.org> on 2016/08/16 06:28:20 UTC
[jira] [Commented] (HIVE-6590) Hive does not work properly with
boolean partition columns (wrong results and inserts to incorrect HDFS
path)
[ https://issues.apache.org/jira/browse/HIVE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422294#comment-15422294 ]
Alexander Behm commented on HIVE-6590:
--------------------------------------
A related and severe issue is that ALTER TABLE DROP PARTITION may drop partitions that were not specified in the ALTER statement. This could lead to accidental data loss.
Reproduction:
{code}
CREATE TABLE broken (c int) PARTITIONED BY (b1 BOOLEAN, s STRING, b2 BOOLEAN, i INT);
# Insert a few variants of 'false' partition-key values.
INSERT INTO TABLE broken PARTITION(b1=false,s='a',b2=false,i=0) VALUES(1);
INSERT INTO TABLE broken PARTITION(b1=FALSE,s='a',b2=false,i=0) VALUES(3);
INSERT INTO TABLE broken PARTITION(b1=false,s='a',b2=False,i=0) VALUES(5);
INSERT INTO TABLE broken PARTITION(b1=false,s='a',b2=FalsE,i=0) VALUES(7);
# Insert a few variants of 'true' partition-key values.
INSERT INTO TABLE broken PARTITION(b1=true,s='a',b2=true,i=0) VALUES(2);
INSERT INTO TABLE broken PARTITION(b1=TRUE,s='a',b2=true,i=0) VALUES(4);
INSERT INTO TABLE broken PARTITION(b1=true,s='a',b2=True,i=0) VALUES(6);
INSERT INTO TABLE broken PARTITION(b1=true,s='a',b2=TruE,i=0) VALUES(8);
# Insert a few variants of mixed 'true'/'false' partition-key values.
INSERT INTO TABLE broken PARTITION(b1=false,s='a',b2=true,i=0) VALUES(100);
INSERT INTO TABLE broken PARTITION(b1=FALSE,s='a',b2=TRUE,i=0) VALUES(1000);
INSERT INTO TABLE broken PARTITION(b1=true,s='a',b2=false,i=0) VALUES(10000);
INSERT INTO TABLE broken PARTITION(b1=tRUe,s='a',b2=fALSe,i=0) VALUES(100000);
# Very broken partition drop.
hive> ALTER TABLE broken DROP PARTITION(b1=true,s='a',b2=true,i=0);
Dropped the partition b1=false/s=a/b2=false/i=0
Dropped the partition b1=false/s=a/b2=False/i=0
Dropped the partition b1=false/s=a/b2=FalsE/i=0
Dropped the partition b1=FALSE/s=a/b2=false/i=0
Dropped the partition b1=false/s=a/b2=true/i=0
Dropped the partition b1=FALSE/s=a/b2=TRUE/i=0
Dropped the partition b1=true/s=a/b2=false/i=0
Dropped the partition b1=tRUe/s=a/b2=fALSe/i=0
Dropped the partition b1=true/s=a/b2=true/i=0
Dropped the partition b1=true/s=a/b2=True/i=0
Dropped the partition b1=true/s=a/b2=TruE/i=0
Dropped the partition b1=TRUE/s=a/b2=true/i=0
OK
Time taken: 1.387 seconds
{code}
> Hive does not work properly with boolean partition columns (wrong results and inserts to incorrect HDFS path)
> -------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-6590
> URL: https://issues.apache.org/jira/browse/HIVE-6590
> Project: Hive
> Issue Type: Bug
> Components: Database/Schema, Metastore
> Affects Versions: 0.10.0
> Reporter: Lenni Kuff
>
> Hive does not work properly with boolean partition columns. Queries return wrong results and also insert to incorrect HDFS paths.
> {code}
> create table bool_part(int_col int) partitioned by(bool_col boolean);
> # This works, creating 3 unique partitions!
> ALTER TABLE bool_table ADD PARTITION (bool_col=FALSE);
> ALTER TABLE bool_table ADD PARTITION (bool_col=false);
> ALTER TABLE bool_table ADD PARTITION (bool_col=False);
> {code}
> The first problem is that Hive cannot filter on a bool partition key column. "select * from bool_part" returns the correct results, but if you apply a filter on the bool partition key column hive won't return any results.
> The second problem is that Hive seems to just call "toString()" on the boolean literal value. This means you can end up with multiple partitions (FALSE, false, FaLSE, etc) mapping to the literal value 'FALSE'. For example, if you can add three partition in have for the same logic value "false" doing:
> ALTER TABLE bool_table ADD PARTITION (bool_col=FALSE) -> /test-warehouse/bool_table/bool_col=FALSE/
> ALTER TABLE bool_table ADD PARTITION (bool_col=false) -> /test-warehouse/bool_table/bool_col=false/
> ALTER TABLE bool_table ADD PARTITION (bool_col=False) -> /test-warehouse/bool_table/bool_col=False/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)