You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Rohini Palaniswamy (JIRA)" <ji...@apache.org> on 2015/01/26 19:11:35 UTC
[jira] [Comment Edited] (TEZ-1999) IndexOutOfBoundsException during
merge
[ https://issues.apache.org/jira/browse/TEZ-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14292170#comment-14292170 ]
Rohini Palaniswamy edited comment on TEZ-1999 at 1/26/15 6:10 PM:
------------------------------------------------------------------
Example:
MergeJoin_8 in pig e2e:
{code}
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSecondaryKeyComparator.compare(PigSecondaryKeyComparator.java:61)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.compare(TezMerger.java:578)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.compareKeyWithNextTopKey(TezMerger.java:526)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.adjustPriorityQueue(TezMerger.java:504)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.next(TezMerger.java:543)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger.writeFile(TezMerger.java:204)
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager.finalMerge(MergeManager.java:885)
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager.close(MergeManager.java:496)
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$RunShuffleCallable.call(Shuffle.java:364)
{code}
Rank_3:
{code}
Caused by: java.lang.IndexOutOfBoundsException
at java.nio.ByteBuffer.wrap(ByteBuffer.java:371)
at org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compare(BinInterSedes.java:726)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTupleSortComparator.compare(PigTupleSortComparator.java:100)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.compare(TezMerger.java:578)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.compareKeyWithNextTopKey(TezMerger.java:526)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.adjustPriorityQueue(TezMerger.java:508)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.next(TezMerger.java:543)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger.writeFile(TezMerger.java:204)
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager.finalMerge(MergeManager.java:885)
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager.close(MergeManager.java:496)
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$RunShuffleCallable.call(Shuffle.java:364)
... 6 more
{code}
was (Author: rohini):
Example:
MergeJoin_8 in pig e2e:
{code}
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSecondaryKeyComparator.compare(PigSecondaryKeyComparator.java:61)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.compare(TezMerger.java:578)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.compareKeyWithNextTopKey(TezMerger.java:526)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.adjustPriorityQueue(TezMerger.java:504)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.next(TezMerger.java:543)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger.writeFile(TezMerger.java:204)
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager.finalMerge(MergeManager.java:885)
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager.close(MergeManager.java:496)
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$RunShuffleCallable.call(Shuffle.java:364)
{code}
Rank_3:
Caused by: java.lang.IndexOutOfBoundsException
at java.nio.ByteBuffer.wrap(ByteBuffer.java:371)
at org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compare(BinInterSedes.java:726)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTupleSortComparator.compare(PigTupleSortComparator.java:100)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.compare(TezMerger.java:578)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.compareKeyWithNextTopKey(TezMerger.java:526)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.adjustPriorityQueue(TezMerger.java:508)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.next(TezMerger.java:543)
at org.apache.tez.runtime.library.common.sort.impl.TezMerger.writeFile(TezMerger.java:204)
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager.finalMerge(MergeManager.java:885)
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager.close(MergeManager.java:496)
at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$RunShuffleCallable.call(Shuffle.java:364)
... 6 more
> IndexOutOfBoundsException during merge
> --------------------------------------
>
> Key: TEZ-1999
> URL: https://issues.apache.org/jira/browse/TEZ-1999
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
>
> This is most likely due to TEZ-1963
> b2 hits IndexOutOfBoundsException in
> public int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2) {
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)