You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Andrew Or (JIRA)" <ji...@apache.org> on 2015/02/10 02:34:35 UTC

[jira] [Closed] (SPARK-5698) Dynamic allocation: do not allow user to request a negative delta

     [ https://issues.apache.org/jira/browse/SPARK-5698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Or closed SPARK-5698.
----------------------------
          Resolution: Fixed
       Fix Version/s: 1.2.2
                      1.3.0
    Target Version/s: 1.3.0, 1.2.2  (was: 1.3.0)

> Dynamic allocation: do not allow user to request a negative delta
> -----------------------------------------------------------------
>
>                 Key: SPARK-5698
>                 URL: https://issues.apache.org/jira/browse/SPARK-5698
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core, YARN
>    Affects Versions: 1.2.0
>            Reporter: Andrew Or
>            Assignee: Andrew Or
>            Priority: Critical
>             Fix For: 1.3.0, 1.2.2
>
>
> If the user calls the following in series:
> sc.requestExecutors(-5)
> sc.killExecutor(...)
> Then we might crash the ApplicationMaster. Why?
> Well, if we request a negative number of additional executors, then the YarnAllocator will lower the target number of executors it is trying to achieve by 5. This might shoot the target number past 0. Then, if so, when we try to kill an executor it will fail the assertion that the target number must be >=0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org