You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by "Markus Weimer (JIRA)" <ji...@apache.org> on 2015/05/30 18:01:17 UTC
[jira] [Updated] (REEF-308) Out of order state transition messages
from the .NET Evaluator
[ https://issues.apache.org/jira/browse/REEF-308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Markus Weimer updated REEF-308:
-------------------------------
Summary: Out of order state transition messages from the .NET Evaluator (was: Race condition and clock skew issues in Tasks reporting back to JobDriver)
> Out of order state transition messages from the .NET Evaluator
> --------------------------------------------------------------
>
> Key: REEF-308
> URL: https://issues.apache.org/jira/browse/REEF-308
> Project: REEF
> Issue Type: Bug
> Components: REEF Driver, REEF-Runtime-HDInsight, REEF.NET Driver
> Affects Versions: 0.11
> Environment: REEF .NET on HDInsight
> Reporter: Andrew Chung
> Assignee: Andrew Chung
>
> Race condition and clock skew issues in Tasks reporting back to JobDriver.
> Stacktrace:
> {noformat}
> May 03, 2015 5:16:54 PM org.apache.reef.runtime.common.launch.REEFErrorHandler onNext
> SEVERE: Uncaught exception.
> java.lang.RuntimeException: Received an message of state RUNNING, not INIT or FAILED for Task 894e954e-a1a1-412a-85c6-d515956d6140_635662701583621790 which we haven't heard from before.
> at org.apache.reef.runtime.common.driver.evaluator.EvaluatorManager.onTaskStatusMessage(EvaluatorManager.java:459)
> at org.apache.reef.runtime.common.driver.evaluator.EvaluatorManager.onEvaluatorHeartbeatMessage(EvaluatorManager.java:341)
> at org.apache.reef.runtime.common.driver.evaluator.EvaluatorHeartbeatHandler.onNext(EvaluatorHeartbeatHandler.java:61)
> at org.apache.reef.runtime.common.driver.evaluator.EvaluatorHeartbeatHandler.onNext(EvaluatorHeartbeatHandler.java:36)
> at org.apache.reef.wake.remote.impl.HandlerContainer.onNext(HandlerContainer.java:145)
> at org.apache.reef.wake.remote.impl.HandlerContainer.onNext(HandlerContainer.java:37)
> at org.apache.reef.wake.remote.impl.OrderedPullEventHandler.onNext(OrderedRemoteReceiverStage.java:164)
> at org.apache.reef.wake.remote.impl.OrderedPullEventHandler.onNext(OrderedRemoteReceiverStage.java:146)
> at org.apache.reef.wake.impl.ThreadPoolStage$1.run(ThreadPoolStage.java:180)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> May 03, 2015 5:16:54 PM org.apache.reef.runtime.common.launch.REEFErrorHandler onNext
> SEVERE: Caught an exception from Wake we cannot send upstream because there is no upstream
> May 03, 2015 5:16:58 PM org.apache.reef.runtime.common.launch.REEFErrorHandler onNext
> SEVERE: Uncaught exception.
> java.lang.RuntimeException: Received an message of state RUNNING, not INIT or FAILED for Task 894e954e-a1a1-412a-85c6-d515956d6140_635662701583621790 which we haven't heard from before.
> at org.apache.reef.runtime.common.driver.evaluator.EvaluatorManager.onTaskStatusMessage(EvaluatorManager.java:459)
> at org.apache.reef.runtime.common.driver.evaluator.EvaluatorManager.onEvaluatorHeartbeatMessage(EvaluatorManager.java:341)
> at org.apache.reef.runtime.common.driver.evaluator.EvaluatorHeartbeatHandler.onNext(EvaluatorHeartbeatHandler.java:61)
> at org.apache.reef.runtime.common.driver.evaluator.EvaluatorHeartbeatHandler.onNext(EvaluatorHeartbeatHandler.java:36)
> at org.apache.reef.wake.remote.impl.HandlerContainer.onNext(HandlerContainer.java:145)
> at org.apache.reef.wake.remote.impl.HandlerContainer.onNext(HandlerContainer.java:37)
> at org.apache.reef.wake.remote.impl.OrderedPullEventHandler.onNext(OrderedRemoteReceiverStage.java:164)
> at org.apache.reef.wake.remote.impl.OrderedPullEventHandler.onNext(OrderedRemoteReceiverStage.java:146)
> at org.apache.reef.wake.impl.ThreadPoolStage$1.run(ThreadPoolStage.java:180)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)