You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Sergey Shelukhin (JIRA)" <ji...@apache.org> on 2016/11/24 00:24:58 UTC
[jira] [Comment Edited] (HIVE-15278) PTF+SortMergeJoin = NPE
[ https://issues.apache.org/jira/browse/HIVE-15278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15691760#comment-15691760 ]
Sergey Shelukhin edited comment on HIVE-15278 at 11/24/16 12:24 AM:
--------------------------------------------------------------------
The patch incommensurate with the amount of pain wrought by the debugging session :)
was (Author: sershe):
The patch incommensurate with the amount of pain put into it :)
> PTF+SortMergeJoin = NPE
> -----------------------
>
> Key: HIVE-15278
> URL: https://issues.apache.org/jira/browse/HIVE-15278
> Project: Hive
> Issue Type: Bug
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Attachments: HIVE-15278.patch
>
>
> Manifests as
> {noformat}
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.exec.persistence.PTFRowContainer.first(PTFRowContainer.java:115)
> at org.apache.hadoop.hive.ql.exec.PTFPartition.iterator(PTFPartition.java:114)
> at org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.finishPartition(PTFOperator.java:340)
> at org.apache.hadoop.hive.ql.exec.PTFOperator.process(PTFOperator.java:114)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:838)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
> at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343)
> ... 29 more
> {noformat}
> It's actually a somewhat subtle ordering problem in sortmerge - as it stands, it calls different branches of the tree in closeOp after they themselves have already been closed. Other operators that clean stuff up in close may result in different errors. The common pattern is
> {noformat}
> 1125 at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)
> 1126 at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:274)
> 1127 at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchOneRow(CommonMergeJoinOperator.java:404)
> ...
> 1131 at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinFinalLeftData(CommonMergeJoinOperator.java:428)
> 1132 at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.closeOp(CommonMergeJoinOperator.java:388)
> 1133 at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)
> ...
> 1139 at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:294)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)