You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Ádám Szita (Jira)" <ji...@apache.org> on 2022/04/29 10:30:00 UTC
[jira] [Updated] (HIVE-26189) Iceberg metadata query throws exceptions after partition evolution

     [ https://issues.apache.org/jira/browse/HIVE-26189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ádám Szita updated HIVE-26189:
------------------------------
    Description: 
The following test case surfaced two issues with metadata table queries:
{code:java}
CREATE EXTERNAL TABLE `partev`( `id` int, `ts` timestamp, `ts2` timestamp)  STORED BY ICEBERG STORED AS ORC;

ALTER TABLE partev SET PARTITION SPEC (id);
INSERT INTO partev VALUES (1, current_timestamp(), current_timestamp());
INSERT INTO partev VALUES (2, current_timestamp(), current_timestamp());

ALTER TABLE partev SET PARTITION SPEC (year(ts));
INSERT INTO partev VALUES (10, current_timestamp(), current_timestamp());

ALTER TABLE partev SET PARTITION SPEC (month(ts));
INSERT INTO partev VALUES (100, current_timestamp(), current_timestamp());

ALTER TABLE partev SET PARTITION SPEC (day(ts));
INSERT INTO partev VALUES (1000, current_timestamp(), current_timestamp());

ALTER TABLE partev SET PARTITION SPEC (hour(ts));
INSERT INTO partev VALUES (10000, current_timestamp(), current_timestamp());

ALTER TABLE partev SET PARTITION SPEC (bucket(2,id));
INSERT INTO partev VALUES (100000, current_timestamp(), current_timestamp());

select * from default.partev.partitions;

ALTER TABLE partev SET PARTITION SPEC (id, year(ts2));
INSERT INTO partev VALUES (200000, current_timestamp(), current_timestamp());

select * from default.partev.partitions;
 {code}
 * NPE for removed partition columns from new specs
 * for day partition transforms: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.time.LocalDate

  was:
The following test case surfaced two issues with metadata table queries:
{code:java}
CREATE EXTERNAL TABLE `partev`( `id` int, `ts` timestamp, `ts2` timestamp)  STORED BY ICEBERG STORED AS ORC;

ALTER TABLE partev SET PARTITION SPEC (id);
INSERT INTO partev VALUES (1, current_timestamp(), current_timestamp());
INSERT INTO partev VALUES (2, current_timestamp(), current_timestamp());

ALTER TABLE partev SET PARTITION SPEC (year(ts));
INSERT INTO partev VALUES (10, current_timestamp(), current_timestamp());

ALTER TABLE partev SET PARTITION SPEC (month(ts));
INSERT INTO partev VALUES (100, current_timestamp(), current_timestamp());

ALTER TABLE partev SET PARTITION SPEC (day(ts));
INSERT INTO partev VALUES (1000, current_timestamp(), current_timestamp());

ALTER TABLE partev SET PARTITION SPEC (hour(ts));
INSERT INTO partev VALUES (10000, current_timestamp(), current_timestamp());

ALTER TABLE partev SET PARTITION SPEC (bucket(2,id));
INSERT INTO partev VALUES (100000, current_timestamp(), current_timestamp());

select * from default.partev.partitions;

ALTER TABLE partev SET PARTITION SPEC (id, year(ts2));
INSERT INTO partev VALUES (200000, current_timestamp(), current_timestamp());

select * from default.partev.partitions;
 {code}
NPE for removed partition columns from new specs, and class cast exceptions for day transform (Integer to LocalDate)


> Iceberg metadata query throws exceptions after partition evolution
> ------------------------------------------------------------------
>
>                 Key: HIVE-26189
>                 URL: https://issues.apache.org/jira/browse/HIVE-26189
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Ádám Szita
>            Priority: Major
>
> The following test case surfaced two issues with metadata table queries:
> {code:java}
> CREATE EXTERNAL TABLE `partev`( `id` int, `ts` timestamp, `ts2` timestamp)  STORED BY ICEBERG STORED AS ORC;
> ALTER TABLE partev SET PARTITION SPEC (id);
> INSERT INTO partev VALUES (1, current_timestamp(), current_timestamp());
> INSERT INTO partev VALUES (2, current_timestamp(), current_timestamp());
> ALTER TABLE partev SET PARTITION SPEC (year(ts));
> INSERT INTO partev VALUES (10, current_timestamp(), current_timestamp());
> ALTER TABLE partev SET PARTITION SPEC (month(ts));
> INSERT INTO partev VALUES (100, current_timestamp(), current_timestamp());
> ALTER TABLE partev SET PARTITION SPEC (day(ts));
> INSERT INTO partev VALUES (1000, current_timestamp(), current_timestamp());
> ALTER TABLE partev SET PARTITION SPEC (hour(ts));
> INSERT INTO partev VALUES (10000, current_timestamp(), current_timestamp());
> ALTER TABLE partev SET PARTITION SPEC (bucket(2,id));
> INSERT INTO partev VALUES (100000, current_timestamp(), current_timestamp());
> select * from default.partev.partitions;
> ALTER TABLE partev SET PARTITION SPEC (id, year(ts2));
> INSERT INTO partev VALUES (200000, current_timestamp(), current_timestamp());
> select * from default.partev.partitions;
>  {code}
>  * NPE for removed partition columns from new specs
>  * for day partition transforms: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.time.LocalDate



--
This message was sent by Atlassian Jira
(v8.20.7#820007)