[jira] [Resolved] (SPARK-14822) Add lazy executor startup to Mesos Scheduler

["Hyukjin Kwon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-14822.
----------------------------------
    Resolution: Incomplete

> Add lazy executor startup to Mesos Scheduler
> --------------------------------------------
>
>                 Key: SPARK-14822
>                 URL: https://issues.apache.org/jira/browse/SPARK-14822
>             Project: Spark
>          Issue Type: Task
>          Components: Mesos
>            Reporter: Michael Gummelt
>            Priority: Major
>              Labels: bulk-closed
>
> As we deprecate fine-grained mode, we need to make sure we have alternative solutions for its benefits.
> Its two benefits are:
> 0. lazy executor startup
>   In fine-grained mode, executors are brought up only as tasks are scheduled.  This means that a user doesn't have to set {{spark.cores.max}} to ensure that the app doesn't consume all resources in the cluster.
> 1. relinquishing cores
>   As Spark tasks terminate, the mesos task it was bound to terminates as well, thus relinquishing the cores assigned to it.
> I'd like to add {{0.}} to coarse-grained mode, possibly enabled with a configuration param.  If https://issues.apache.org/jira/browse/MESOS-1279 ever happens, we can add {{1.}} as well.
> cc [~tnachen] [~dragos] [~skonto] [~andrewor14]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org