[jira] [Resolved] (HIVE-3326) plan for multiple mapjoin followed by a normal join is wrong

["Navis (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Navis resolved HIVE-3326.
-------------------------

    Resolution: Cannot Reproduce

Cannot reproduce the situation by HIVE-3784
                
> plan for multiple mapjoin followed by a normal join is wrong
> ------------------------------------------------------------
>
>                 Key: HIVE-3326
>                 URL: https://issues.apache.org/jira/browse/HIVE-3326
>             Project: Hive
>          Issue Type: Bug
>          Components: SQL
>         Environment: OS X 10.8; java 1.6.0_33
>            Reporter: Zhang Xinyu
>            Assignee: Navis
>         Attachments: HIVE-3326.D8091.1.patch, patch.diff
>
>
> example queries:
> {code}
> create table yudi(c1 int, c2 int, c3 int, c4 int);
> create table wangmu(c1 int, c2 int, c3 int, c4 int);
> select /*+mapjoin(b,c)*/ * from yudi a join yudi b on a.c1=b.c1 join wangmu c on b.c2=c.c2 join yudi d on a.c3=d.c3;
> {code}
> in explain mode, I got this:
> {code}
> hive> explain select /*+mapjoin(b,c)*/ * from yudi a join yudi b on a.c1=b.c1 join wangmu c on b.c2=c.c2 join yudi d on a.c3=d.c3;
> OK
> STAGE DEPENDENCIES:
>   Stage-8 is a root stage
>   Stage-2 depends on stages: Stage-8
>   Stage-7 depends on stages: Stage-2
>   Stage-3 depends on stages: Stage-7
>   Stage-1 depends on stages: Stage-3
> STAGE PLANS:
>   Stage: Stage-8
>     Map Reduce Local Work
>       Alias -> Map Local Tables:
>         b
>         <Not Important>
>   Stage: Stage-2
>     Map Reduce
>       Alias -> Map Operator Tree:
>         a
>         <Not Important>
>       Local Work:
>         Map Reduce Local Work
>   Stage: Stage-7
>     Map Reduce Local Work
>       Alias -> Map Local Tables:
>         c
>         <Not Important>
>   Stage: Stage-3
>     Map Reduce
>       Alias -> Map Operator Tree:
>            file:/var/folders/4w/3_nk1cwd4pd023mzx64p3r480000gn/T/dukezhang/hive_2012-08-01_14-01-37_152_5814747325029961632/-mr-10002
>         <Not Important>
>       Local Work:
>         Map Reduce Local Work
>   Stage: Stage-1
>     Map Reduce
>       Alias -> Map Operator Tree:
>         d
>           TableScan
>         file:/var/folders/4w/3_nk1cwd4pd023mzx64p3r480000gn/T/dukezhang/hive_2012-08-01_14-01-37_152_5814747325029961632/-mr-10002
>           Select Operator
>       Reduce Operator Tree:
>       <Not Important>
> {code}
> You see, mapper of Stage-1 should read from Stage-3, maybe '.../-mr-10003', not Stage-2(result in '.../-mr-10002').
> To resolve this bug, I found these codes(GenMapRedUtils.java, about line 431):
> {code:title=GenMapRedUtils.java}
> if (oldMapJoin == null) {
>   if (opProcCtx.getParseCtx().getListMapJoinOpsNoReducer().contains(mjOp)
>       || local || (oldTask != null) && (parTasks != null)) {
>     taskTmpDir = mjCtx.getTaskTmpDir();
>     tt_desc = mjCtx.getTTDesc();
>     rootOp = mjCtx.getRootMapJoinOp();
>     }
> } else {
>   GenMRMapJoinCtx oldMjCtx = opProcCtx.getMapJoinCtx(oldMapJoin);
>   assert oldMjCtx != null;
>   taskTmpDir = oldMjCtx.getTaskTmpDir();
>   tt_desc = oldMjCtx.getTTDesc();
>   rootOp = oldMjCtx.getRootMapJoinOp();
> }
> {code}
> my query goes into 'else' block and gets wrong taskTmpDir. I hack them to let query go into 'if' block, and it works.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira