[jira] [Commented] (HIVE-5556) Pushdown join conditions

["cw (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14184986#comment-14184986 ] 

cw commented on HIVE-5556:
--------------------------

Hi [~rhbutani]. I found a SQL runned well on hive0.11 but now get a IndexOutOfBoundsException on hive0.13. Here is the code that would cause the exception.

CREATE TABLE `test_1`( k string, v string );
CREATE TABLE `test_2`( k string, v string );

explain select * from
(
    SELECT 
        k tk,
        k tv
    FROM 
        test_1
)sub1
 
JOIN
(
    select
        k tk,
        v tv
    from test_1
)sub2 ON (sub1.tk=sub2.tk AND sub1.tk=sub2.tv)
 
JOIN
(
    select
        k tk
    from test_2
)sub3 ON sub1.tk=sub3.tk;

The stack of java heap is:
java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
    at java.util.ArrayList.rangeCheck(ArrayList.java:604)
    at java.util.ArrayList.get(ArrayList.java:382)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.mergeJoins(SemanticAnalyzer.java:7403)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.mergeJoinTree(SemanticAnalyzer.java:7616)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:8946)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9220)
    at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
    at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:64)
    at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:422)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:322)
    at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:978)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1043)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:904)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)


I noticed this patch edited SemanticAnalyzer.java with the change below:
-    if ((targetCondn == null) || (nodeCondn.size() != targetCondn.size())) {
-      return -1;
+    if ( targetCondn == null ) {
+      return new ObjectPair(-1, null);
+    }

Then I changed the if statement as below and found the problem is gone.
if ( targetCondn == null || (nodeCondn.size() != targetCondn.size())) {
    return new ObjectPair(-1, null);
}

Would you please help to check this problem?

> Pushdown join conditions
> ------------------------
>
>                 Key: HIVE-5556
>                 URL: https://issues.apache.org/jira/browse/HIVE-5556
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Query Processor
>            Reporter: Harish Butani
>            Assignee: Harish Butani
>              Labels: TODOC13
>             Fix For: 0.13.0
>
>         Attachments: HIVE-5556.1.patch, HIVE-5556.2.patch
>
>
> See details in HIVE-5555



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)