You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by "Andrew Chung (JIRA)" <ji...@apache.org> on 2015/05/06 02:20:03 UTC
[jira] [Created] (REEF-308) Race condition and clock skew issues in
Tasks reporting back to JobDriver
Andrew Chung created REEF-308:
---------------------------------
Summary: Race condition and clock skew issues in Tasks reporting back to JobDriver
Key: REEF-308
URL: https://issues.apache.org/jira/browse/REEF-308
Project: REEF
Issue Type: Bug
Components: REEF-Runtime-HDInsight, REEF.NET Driver
Affects Versions: 0.11
Environment: REEF .NET on HDInsight
Reporter: Andrew Chung
Race condition and clock skew issues in Tasks reporting back to JobDriver.
Stacktrace:
May 03, 2015 5:16:54 PM org.apache.reef.runtime.common.launch.REEFErrorHandler onNext
SEVERE: Uncaught exception.
java.lang.RuntimeException: Received an message of state RUNNING, not INIT or FAILED for Task 894e954e-a1a1-412a-85c6-d515956d6140_635662701583621790 which we haven't heard from before.
at org.apache.reef.runtime.common.driver.evaluator.EvaluatorManager.onTaskStatusMessage(EvaluatorManager.java:459)
at org.apache.reef.runtime.common.driver.evaluator.EvaluatorManager.onEvaluatorHeartbeatMessage(EvaluatorManager.java:341)
at org.apache.reef.runtime.common.driver.evaluator.EvaluatorHeartbeatHandler.onNext(EvaluatorHeartbeatHandler.java:61)
at org.apache.reef.runtime.common.driver.evaluator.EvaluatorHeartbeatHandler.onNext(EvaluatorHeartbeatHandler.java:36)
at org.apache.reef.wake.remote.impl.HandlerContainer.onNext(HandlerContainer.java:145)
at org.apache.reef.wake.remote.impl.HandlerContainer.onNext(HandlerContainer.java:37)
at org.apache.reef.wake.remote.impl.OrderedPullEventHandler.onNext(OrderedRemoteReceiverStage.java:164)
at org.apache.reef.wake.remote.impl.OrderedPullEventHandler.onNext(OrderedRemoteReceiverStage.java:146)
at org.apache.reef.wake.impl.ThreadPoolStage$1.run(ThreadPoolStage.java:180)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
May 03, 2015 5:16:54 PM org.apache.reef.runtime.common.launch.REEFErrorHandler onNext
SEVERE: Caught an exception from Wake we cannot send upstream because there is no upstream
May 03, 2015 5:16:58 PM org.apache.reef.runtime.common.launch.REEFErrorHandler onNext
SEVERE: Uncaught exception.
java.lang.RuntimeException: Received an message of state RUNNING, not INIT or FAILED for Task 894e954e-a1a1-412a-85c6-d515956d6140_635662701583621790 which we haven't heard from before.
at org.apache.reef.runtime.common.driver.evaluator.EvaluatorManager.onTaskStatusMessage(EvaluatorManager.java:459)
at org.apache.reef.runtime.common.driver.evaluator.EvaluatorManager.onEvaluatorHeartbeatMessage(EvaluatorManager.java:341)
at org.apache.reef.runtime.common.driver.evaluator.EvaluatorHeartbeatHandler.onNext(EvaluatorHeartbeatHandler.java:61)
at org.apache.reef.runtime.common.driver.evaluator.EvaluatorHeartbeatHandler.onNext(EvaluatorHeartbeatHandler.java:36)
at org.apache.reef.wake.remote.impl.HandlerContainer.onNext(HandlerContainer.java:145)
at org.apache.reef.wake.remote.impl.HandlerContainer.onNext(HandlerContainer.java:37)
at org.apache.reef.wake.remote.impl.OrderedPullEventHandler.onNext(OrderedRemoteReceiverStage.java:164)
at org.apache.reef.wake.remote.impl.OrderedPullEventHandler.onNext(OrderedRemoteReceiverStage.java:146)
at org.apache.reef.wake.impl.ThreadPoolStage$1.run(ThreadPoolStage.java:180)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)