You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Siddharth Seth (JIRA)" <ji...@apache.org> on 2014/04/11 19:40:35 UTC
[jira] [Assigned] (TEZ-1040) TezMerger hangs when io.sort.mb value
is 1
[ https://issues.apache.org/jira/browse/TEZ-1040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth reassigned TEZ-1040:
-----------------------------------
Assignee: Siddharth Seth
> TezMerger hangs when io.sort.mb value is 1
> ------------------------------------------
>
> Key: TEZ-1040
> URL: https://issues.apache.org/jira/browse/TEZ-1040
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
> Assignee: Siddharth Seth
>
> TestCombiner.testMultiCombinerUse() in pig test suite sets io.sort.mb to 1 for testing purposes. Tez is stuck at a loop while running the task with 0.4.0-incubating and 0.5.0-incubating-SNAPSHOT in org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.computeBytesInMerges. It runs fine with 0.3.0-incubating.
> {code}
> at java.lang.Long.valueOf(Long.java:557)
> at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.computeBytesInMerges(TezMerger.java:814)
> at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:571)
> at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:541)
> at org.apache.tez.runtime.library.common.sort.impl.TezMerger.merge(TezMerger.java:164)
> at org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.mergeParts(DefaultSorter.java:1078)
> at org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.flush(DefaultSorter.java:633)
> at org.apache.tez.runtime.library.output.OnFileSortedOutput.close(OnFileSortedOutput.java:134)
> - locked <7f03a5b90> (a org.apache.tez.runtime.library.output.OnFileSortedOutput)
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.close(LogicalIOProcessorRuntimeTask.java:331)
> at org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:567)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:394)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553)
> {code}
> To reproduce:
> ant clean test -Dtestcase=TestCombiner -Dexectype=tez -Dhadoopversion=23 -Dtest.output=true -logfile /tmp/log
--
This message was sent by Atlassian JIRA
(v6.2#6252)