You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org> on 2010/10/01 07:45:33 UTC
[jira] Commented: (HIVE-1678) NPE in MapJoin
[ https://issues.apache.org/jira/browse/HIVE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916789#action_12916789 ]
Amareshwari Sriramadasu commented on HIVE-1678:
-----------------------------------------------
The following query :
select /*+MAPJOIN(src, myinput1) */ count(srcpart.key) from srcpart join src on (srcpart.value=src.value) join myinput1 on (srcpart.key=myinput1.key);
launches three MapReduce jobs in which third one fails with NPE.
The task logs :
{noformat}
2010-10-01 11:07:02,324 INFO ExecMapper:
<MAP>Id =11
<Children>
<SEL>Id =0
<Children>
<MAPJOIN>Id =1
<Children>
<FS>Id =2
<Parent>Id = 1 null<\Parent>
<\FS>
<\Children>
<Parent>Id = 0 nullId = 3
<TS>Id =3
<Children>null
<\Children>
<\TS><\Parent>
<\MAPJOIN>
<\Children>
<Parent>Id = 11 null<\Parent>
<\SEL>
<\Children>
<\MAP>
2010-10-01 11:07:02,324 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initializing Self 11 MAP
2010-10-01 11:07:02,324 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing Self 0 SEL
2010-10-01 11:07:02,327 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: SELECT struct<_col0:int>
2010-10-01 11:07:02,332 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Operator 0 SEL initialized
2010-10-01 11:07:02,332 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing children of 0 SEL
2010-10-01 11:07:02,332 INFO org.apache.hadoop.hive.ql.exec.MapJoinOperator: Initializing child 1 MAPJOIN
2010-10-01 11:07:02,332 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initialization Done 0 SEL
2010-10-01 11:07:02,332 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initialization Done 11 MAP
2010-10-01 11:07:02,334 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 11 forwarding 1 rows
2010-10-01 11:07:02,334 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 0 forwarding 1 rows
2010-10-01 11:07:02,338 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"_col0":238}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:483)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:329)
at org.apache.hadoop.mapred.Child$4.run(Child.java:219)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1021)
at org.apache.hadoop.mapred.Child.main(Child.java:213)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:177)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:459)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:710)
at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:459)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:710)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:464)
... 9 more
{noformat}
>From the logs we can see that MapJoinOperator is not intialized at all.
> NPE in MapJoin
> ---------------
>
> Key: HIVE-1678
> URL: https://issues.apache.org/jira/browse/HIVE-1678
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Amareshwari Sriramadasu
>
> The query with two map joins and a group by fails with following NPE:
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:177)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:464)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.