You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Alessandro Solimando (Jira)" <ji...@apache.org> on 2021/12/02 18:39:00 UTC
[jira] [Updated] (HIVE-25766) java.util.NoSuchElementException in HiveFilterProjectTransposeRule if predicate has no InputRef
[ https://issues.apache.org/jira/browse/HIVE-25766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alessandro Solimando updated HIVE-25766:
----------------------------------------
Description:
The issue can be reproduced with the following query:
{code:java}
create table test1 (s string);
create table test2 (m string);
EXPLAIN CBO SELECT c.m
FROM (
SELECT substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) as m
FROM test1
WHERE substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) = '2') c
JOIN test2 d ON c.m = d.m;
{code}
It fails with the following exception:
{noformat}
java.util.NoSuchElementException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1447)
at java.util.HashMap$KeyIterator.next(HashMap.java:1469)
at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.check(HiveFilterProjectTransposeRule.java:348)
at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:306)
at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:303)
at org.apache.calcite.rel.SingleRel.childrenAccept(SingleRel.java:72)
at org.apache.calcite.rel.RelVisitor.visit(RelVisitor.java:44)
at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:316)
at org.apache.calcite.rel.RelVisitor.go(RelVisitor.java:61)
at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.isRedundantIsNotNull(HiveFilterProjectTransposeRule.java:276)
at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.onMatch(HiveFilterProjectTransposeRule.java:191){noformat}
The current implementation, while checking if the predicate to be transposed is redundant or not, it expects at least one InputRef, but the predicate can have none as in this case.
was:
The issue can be reproduced with the following query:
{code:java}
create table test1 (s string);
create table test2 (m int);
EXPLAIN
SELECT c.m
FROM (
SELECT cast(substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) AS int) as m
FROM test1
WHERE cast(substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) AS int) = 2) c
JOIN test2 d ON c.m = d.m; {code}
It fails with the following exception:
{noformat}
java.util.NoSuchElementException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1447)
at java.util.HashMap$KeyIterator.next(HashMap.java:1469)
at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.check(HiveFilterProjectTransposeRule.java:348)
at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:306)
at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:303)
at org.apache.calcite.rel.SingleRel.childrenAccept(SingleRel.java:72)
at org.apache.calcite.rel.RelVisitor.visit(RelVisitor.java:44)
at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:316)
at org.apache.calcite.rel.RelVisitor.go(RelVisitor.java:61)
at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.isRedundantIsNotNull(HiveFilterProjectTransposeRule.java:276)
at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.onMatch(HiveFilterProjectTransposeRule.java:191){noformat}
The current implementation, while checking if the predicate to be transposed is redundant or not, it expects at least one InputRef, but the predicate can have none as in this case.
> java.util.NoSuchElementException in HiveFilterProjectTransposeRule if predicate has no InputRef
> -----------------------------------------------------------------------------------------------
>
> Key: HIVE-25766
> URL: https://issues.apache.org/jira/browse/HIVE-25766
> Project: Hive
> Issue Type: Bug
> Components: CBO, Query Planning
> Affects Versions: 4.0.0
> Reporter: Alessandro Solimando
> Assignee: Alessandro Solimando
> Priority: Major
>
> The issue can be reproduced with the following query:
> {code:java}
> create table test1 (s string);
> create table test2 (m string);
> EXPLAIN CBO SELECT c.m
> FROM (
> SELECT substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) as m
> FROM test1
> WHERE substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) = '2') c
> JOIN test2 d ON c.m = d.m;
> {code}
> It fails with the following exception:
> {noformat}
> java.util.NoSuchElementException
> at java.util.HashMap$HashIterator.nextNode(HashMap.java:1447)
> at java.util.HashMap$KeyIterator.next(HashMap.java:1469)
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.check(HiveFilterProjectTransposeRule.java:348)
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:306)
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:303)
> at org.apache.calcite.rel.SingleRel.childrenAccept(SingleRel.java:72)
> at org.apache.calcite.rel.RelVisitor.visit(RelVisitor.java:44)
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:316)
> at org.apache.calcite.rel.RelVisitor.go(RelVisitor.java:61)
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.isRedundantIsNotNull(HiveFilterProjectTransposeRule.java:276)
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.onMatch(HiveFilterProjectTransposeRule.java:191){noformat}
> The current implementation, while checking if the predicate to be transposed is redundant or not, it expects at least one InputRef, but the predicate can have none as in this case.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)