[jira] [Created] (SPARK-4020) Failed executor not properly removed if it has not run tasks

["Andrew Or (JIRA)" <ji...@apache.org>]

Andrew Or created SPARK-4020:
--------------------------------

             Summary: Failed executor not properly removed if it has not run tasks
                 Key: SPARK-4020
                 URL: https://issues.apache.org/jira/browse/SPARK-4020
             Project: Spark
          Issue Type: Bug
            Reporter: Andrew Or
            Assignee: Andrew Or


If an executor has not run tasks, then the DAG scheduler won't remove the block manager that belongs to this executor. This is because we don't add the executor to `activeExecutorIds` in `TaskSchedulerImpl` until we have successfully scheduled a task on the executor.

The motivation for fixing this is to avoid having the `BlockManagerMasterActor` to remove the associated block manager through a timeout when it can just do so right away.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org