[jira] [Created] (SPARK-21219) Task retry occurs on same executor due to race condition with blacklisting

["Eric Vandenberg (JIRA)" <ji...@apache.org>]

Eric Vandenberg created SPARK-21219:
---------------------------------------

             Summary: Task retry occurs on same executor due to race condition with blacklisting
                 Key: SPARK-21219
                 URL: https://issues.apache.org/jira/browse/SPARK-21219
             Project: Spark
          Issue Type: Bug
          Components: Scheduler
    Affects Versions: 2.1.1
            Reporter: Eric Vandenberg
            Priority: Minor


When a task fails it is added into the pending task list and corresponding black list policy is enforced (ie, specifying if it can/can't run on a particular node/executor/etc.)  Unfortunately the ordering is such that retrying the task could assign the task to the same executor, which, incidentally could be shutting down and immediately fail the retry.   Instead the black list state should be updated and then the task assigned, ensuring that the black list policy is properly enforced.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org