You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Stamatis Zampetakis (Jira)" <ji...@apache.org> on 2021/08/10 09:18:00 UTC
[jira] [Resolved] (HIVE-25410) CommonMergeJoinOperator fails when a
join key is ARRAY with arbitrary size
[ https://issues.apache.org/jira/browse/HIVE-25410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stamatis Zampetakis resolved HIVE-25410.
----------------------------------------
Resolution: Fixed
Fixed in [9b5feec4dc475d6ee38c6d11130fb19d351a718d|https://github.com/apache/hive/commit/9b5feec4dc475d6ee38c6d11130fb19d351a718d]. Thanks for the PR [~okumin]!
> CommonMergeJoinOperator fails when a join key is ARRAY with arbitrary size
> --------------------------------------------------------------------------
>
> Key: HIVE-25410
> URL: https://issues.apache.org/jira/browse/HIVE-25410
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Reporter: okumin
> Assignee: okumin
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Thanks to HIVE-24883, CommonMergeJoinOperator can handle ARRAY or STRUCT types as a JOIN key.
> There are corner cases where CommonMergeJoinOperator fails with `ArrayIndexOutOfBoundsException`.
>
> This is a simple case.
> {code:java}
> SET hive.auto.convert.join=false;
> CREATE TABLE table_list_types (id int, key array<int>);
> INSERT INTO table_list_types VALUES (1, array(1, 2)), (2, array(1, 2)), (3, array(1, 2, 3)), (4, array(1, 2, 3));
> SELECT * FROM table_list_types t1 INNER JOIN table_list_types t2 ON t1.key = t2.key; {code}
> With 69c97c26ac68a245f4d327cc2f7b3a2333f8fa84, the following error happened.
> {code:java}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 2
> at org.apache.hadoop.hive.ql.exec.HiveStructComparator.compare(HiveStructComparator.java:57)
> at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.compareKey(CommonMergeJoinOperator.java:629)
> at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.compareKeys(CommonMergeJoinOperator.java:597)
> at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.processKey(CommonMergeJoinOperator.java:566)
> at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.process(CommonMergeJoinOperator.java:249)
> at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:370)
> ... 26 more {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)