You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Saisai Shao (JIRA)" <ji...@apache.org> on 2015/07/30 07:33:04 UTC

[jira] [Created] (SPARK-9468) Avoid scheduling tasks on preemption executors

Saisai Shao created SPARK-9468:
----------------------------------

             Summary: Avoid scheduling tasks on preemption executors
                 Key: SPARK-9468
                 URL: https://issues.apache.org/jira/browse/SPARK-9468
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core, YARN
            Reporter: Saisai Shao
            Priority: Minor


Currently Spark SchedulerBackend has no mechanism to be notified of preemption of executors by cluster manager, this will make SchedulerBackend keep scheduling tasks on the to-be-preempted executors and finally be failed. Also this cumulative failure will possibly lead to job failure.

So Spark SchedulerBackend should be notified and avoid scheduling tasks on preemption nodes to avoid unnecessary failure. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org