You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Nandor Kollar (JIRA)" <ji...@apache.org> on 2016/11/14 08:41:58 UTC
[jira] [Commented] (PIG-5057) IndexOutOfBoundsException when pig
reducer processOnePackageOutput
[ https://issues.apache.org/jira/browse/PIG-5057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663167#comment-15663167 ]
Nandor Kollar commented on PIG-5057:
------------------------------------
Could you please add a test case too which demonstrates the problem?
> IndexOutOfBoundsException when pig reducer processOnePackageOutput
> ------------------------------------------------------------------
>
> Key: PIG-5057
> URL: https://issues.apache.org/jira/browse/PIG-5057
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.14.0, 0.15.0
> Environment: pig-0.14/pig-0.15 cdh5.3.2
> Reporter: shenxianqiang
> Priority: Minor
> Attachments: PIG-5057.patch
>
>
> When runing a pig job,the reducer throw out of bands exception,which leads the job failed.
> {quote}
> 2016-11-14 15:31:04,752 WARN [main] org.apache.hadoop.io.compress.LzoCodec: org.apache.hadoop.io.compress.LzoCodec is deprecated. You should use com.hadoop.compression.lzo.LzoCodec instead to generate LZO compressed data.
> 2016-11-14 15:32:08,300 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.rangeCheck(ArrayList.java:635)
> at java.util.ArrayList.get(ArrayList.java:411)
> at org.apache.pig.data.DefaultTuple.get(DefaultTuple.java:117)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.Packager.getValueTuple(Packager.java:234)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage$PeekedBag$1.next(POPackage.java:435)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage$PeekedBag$1.next(POPackage.java:408)
> at org.apache.pig.data.DefaultAbstractBag.addAll(DefaultAbstractBag.java:151)
> at org.apache.pig.data.DefaultAbstractBag.addAll(DefaultAbstractBag.java:137)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.Packager.attachInput(Packager.java:125)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage.getNextTuple(POPackage.java:283)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:431)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:422)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:269)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1892)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)