You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Jonathan Eagles (JIRA)" <ji...@apache.org> on 2014/08/07 03:25:11 UTC
[jira] [Updated] (TEZ-1385) Disk Direct fails for MapOutput when
trying to use OnDiskMerger
[ https://issues.apache.org/jira/browse/TEZ-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated TEZ-1385:
---------------------------------
Description:
Stack Trace
2014-08-06 20:08:52,869 [DiskToDiskMerger [scope_75]] INFO org.apache.tez.runtime.library.common.shuffle.impl.MergeManager - attempt_1407373728912_0001_1_02_000000_0_1000 8 Finished merging 10 map output files on disk of total-size 24291. Local output file is /tmp/temp751138072/.tez/application_1407373728912_0001/localmode-local-dir/tmp/tem p751138072/.tez/application_1407373728912_0001/localmode-local-dir/output/attempt_1407373728912_0001_1_01_000000_0_10006/file.out.merged of size 24001
2014-08-06 20:08:52,870 [TezTaskEventRouter[attempt_1407373728912_0001_1_02_000000_0]] WARN org.apache.tez.runtime.LogicalIOProcessorRuntimeTask - Failed to handle event
java.lang.ClassCastException: org.apache.hadoop.fs.Path cannot be cast to org.apache.hadoop.io.FileChunkPath
at org.apache.hadoop.io.FileChunkPath.compareTo(FileChunkPath.java:72)
at java.util.TreeMap.put(TreeMap.java:545)
at java.util.TreeSet.add(TreeSet.java:238)
at org.apache.tez.runtime.library.common.shuffle.impl.MergeManager.closeOnDiskFile(MergeManager.java:428)
at org.apache.tez.runtime.library.common.shuffle.impl.MapOutput.commit(MapOutput.java:221)
at org.apache.tez.runtime.library.common.shuffle.impl.ShuffleScheduler.copySucceeded(ShuffleScheduler.java:171)
at org.apache.tez.runtime.library.common.shuffle.impl.ShuffleInputEventHandler.processDataMovementEvent(ShuffleInputEventHandler.java:135)
at org.apache.tez.runtime.library.common.shuffle.impl.ShuffleInputEventHandler.handleEvent(ShuffleInputEventHandler.java:84)
at org.apache.tez.runtime.library.common.shuffle.impl.ShuffleInputEventHandler.handleEvents(ShuffleInputEventHandler.java:77)
at org.apache.tez.runtime.library.common.shuffle.impl.Shuffle.handleEvents(Shuffle.java:229)
at org.apache.tez.runtime.library.input.ShuffledMergedInput.handleEvents(ShuffledMergedInput.java:252)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.handleEvent(LogicalIOProcessorRuntimeTask.java:605)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.access$1100(LogicalIOProcessorRuntimeTask.java:90)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$1.run(LogicalIOProcessorRuntimeTask.java:668)
at java.lang.Thread.run(Thread.java:695)
> Disk Direct fails for MapOutput when trying to use OnDiskMerger
> ----------------------------------------------------------------
>
> Key: TEZ-1385
> URL: https://issues.apache.org/jira/browse/TEZ-1385
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Jonathan Eagles
> Priority: Critical
>
> Stack Trace
> 2014-08-06 20:08:52,869 [DiskToDiskMerger [scope_75]] INFO org.apache.tez.runtime.library.common.shuffle.impl.MergeManager - attempt_1407373728912_0001_1_02_000000_0_1000 8 Finished merging 10 map output files on disk of total-size 24291. Local output file is /tmp/temp751138072/.tez/application_1407373728912_0001/localmode-local-dir/tmp/tem p751138072/.tez/application_1407373728912_0001/localmode-local-dir/output/attempt_1407373728912_0001_1_01_000000_0_10006/file.out.merged of size 24001
> 2014-08-06 20:08:52,870 [TezTaskEventRouter[attempt_1407373728912_0001_1_02_000000_0]] WARN org.apache.tez.runtime.LogicalIOProcessorRuntimeTask - Failed to handle event
> java.lang.ClassCastException: org.apache.hadoop.fs.Path cannot be cast to org.apache.hadoop.io.FileChunkPath
> at org.apache.hadoop.io.FileChunkPath.compareTo(FileChunkPath.java:72)
> at java.util.TreeMap.put(TreeMap.java:545)
> at java.util.TreeSet.add(TreeSet.java:238)
> at org.apache.tez.runtime.library.common.shuffle.impl.MergeManager.closeOnDiskFile(MergeManager.java:428)
> at org.apache.tez.runtime.library.common.shuffle.impl.MapOutput.commit(MapOutput.java:221)
> at org.apache.tez.runtime.library.common.shuffle.impl.ShuffleScheduler.copySucceeded(ShuffleScheduler.java:171)
> at org.apache.tez.runtime.library.common.shuffle.impl.ShuffleInputEventHandler.processDataMovementEvent(ShuffleInputEventHandler.java:135)
> at org.apache.tez.runtime.library.common.shuffle.impl.ShuffleInputEventHandler.handleEvent(ShuffleInputEventHandler.java:84)
> at org.apache.tez.runtime.library.common.shuffle.impl.ShuffleInputEventHandler.handleEvents(ShuffleInputEventHandler.java:77)
> at org.apache.tez.runtime.library.common.shuffle.impl.Shuffle.handleEvents(Shuffle.java:229)
> at org.apache.tez.runtime.library.input.ShuffledMergedInput.handleEvents(ShuffledMergedInput.java:252)
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.handleEvent(LogicalIOProcessorRuntimeTask.java:605)
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.access$1100(LogicalIOProcessorRuntimeTask.java:90)
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$1.run(LogicalIOProcessorRuntimeTask.java:668)
> at java.lang.Thread.run(Thread.java:695)
--
This message was sent by Atlassian JIRA
(v6.2#6252)