You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 04:22:09 UTC

[jira] [Updated] (SPARK-9873) Cap the amount of executors launched in Mesos fine grain mode

     [ https://issues.apache.org/jira/browse/SPARK-9873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon updated SPARK-9873:
--------------------------------
    Labels: bulk-closed  (was: )

> Cap the amount of executors launched in Mesos fine grain mode
> -------------------------------------------------------------
>
>                 Key: SPARK-9873
>                 URL: https://issues.apache.org/jira/browse/SPARK-9873
>             Project: Spark
>          Issue Type: Improvement
>          Components: Mesos
>            Reporter: Timothy Chen
>            Priority: Major
>              Labels: bulk-closed
>
> Currently in fine grain mode as long as there is resources available that matches the scheduler requirement Spark will try to use the resources offered by mesos, which means to excessive resource usage that can lead to other frameworks not able to get their fair share.
> We should add a option to cap the number of executors launched, so the combination of spark.task.cpus and spark.mesos.executor.max is the total amount of cores it will grab.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org