You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Yuchen Huo (Jira)" <ji...@apache.org> on 2019/11/20 20:50:00 UTC

[jira] [Created] (SPARK-29976) Allow speculation even if there is only one task

Yuchen Huo created SPARK-29976:
----------------------------------

             Summary: Allow speculation even if there is only one task
                 Key: SPARK-29976
                 URL: https://issues.apache.org/jira/browse/SPARK-29976
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 3.0.0
            Reporter: Yuchen Huo


In the current speculative execution implementation if there is only one task in the stage then no speculative run would be conducted. However, there might be cases where an executor have some problem in writing to its disk and just hang forever. In this case, if the single task stage get assigned to the problematic executor then the whole job would hang forever. It would be better if we could run the task on another executor if this happens. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org