You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2016/07/25 16:37:20 UTC
[jira] [Commented] (SPARK-16708)
ExecutorAllocationManager.numRunningTasks can be negative when stage retry
[ https://issues.apache.org/jira/browse/SPARK-16708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392245#comment-15392245 ]
Sean Owen commented on SPARK-16708:
-----------------------------------
Looks like a duplicate of SPARK-10141
> ExecutorAllocationManager.numRunningTasks can be negative when stage retry
> --------------------------------------------------------------------------
>
> Key: SPARK-16708
> URL: https://issues.apache.org/jira/browse/SPARK-16708
> Project: Spark
> Issue Type: Bug
> Affects Versions: 1.6.0
> Reporter: Hong Shen
>
> When a task fetch failed, the stage will complete and retry, when the stage complete, ExecutorAllocationManager.numRunningTasks will be set 0, here is the code:
> {code}
> override def onStageCompleted(stageCompleted: SparkListenerStageCompleted): Unit = {
> val stageId = stageCompleted.stageInfo.stageId
> allocationManager.synchronized {
> stageIdToNumTasks -= stageId
> stageIdToTaskIndices -= stageId
> stageIdToExecutorPlacementHints -= stageId
> // Update the executor placement hints
> updateExecutorPlacementHints()
> // If this is the last stage with pending tasks, mark the scheduler queue as empty
> // This is needed in case the stage is aborted for any reason
> if (stageIdToNumTasks.isEmpty) {
> allocationManager.onSchedulerQueueEmpty()
> if (numRunningTasks != 0) {
> logWarning("No stages are running, but numRunningTasks != 0")
> numRunningTasks = 0
> }
> }
> }
> }
> {code}
> But when the stage's running task finished, numRunningTasks will minus 1, so numRunningTasks be negative, it can cause maxNeeded be negative.
> {code}
> override def onTaskEnd(taskEnd: SparkListenerTaskEnd): Unit = {
> val executorId = taskEnd.taskInfo.executorId
> val taskId = taskEnd.taskInfo.taskId
> val taskIndex = taskEnd.taskInfo.index
> val stageId = taskEnd.stageId
> allocationManager.synchronized {
> numRunningTasks -= 1
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org