[jira] [Resolved] (SPARK-10293) Add support for oversubscription in Mesos

["Hyukjin Kwon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-10293.
----------------------------------
    Resolution: Incomplete

> Add support for oversubscription in Mesos
> -----------------------------------------
>
>                 Key: SPARK-10293
>                 URL: https://issues.apache.org/jira/browse/SPARK-10293
>             Project: Spark
>          Issue Type: Story
>          Components: Mesos
>            Reporter: Chris Bannister
>            Priority: Major
>              Labels: bulk-closed
>
> Currently when running Spark on Mesos each executor will use all the CPU resources offered to it. This can lead to cases where a Spark executor is using all the CPU resources on a single slave but is underutilising the CPU allocated to it.
> Mesos added support in 0.23 for oversubscription, where frameworks can be offered slack resources for CPU resources already allocated. So that if a task is allocated 10 cpus but is only using 1, 9 revokable offers will be made to other frameworks. If the original task starts using its allocated CPU then Mesos will preempt the revokable task, killing it.
> From a cluster usage perspective it would be very useful to be able to specify that some jobs are revokable and can be ran in slack resources, and that they should be rescheduled without affecting the job status (ie not count towards job failure) when a task is revoked.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org