You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Timothy Chen (JIRA)" <ji...@apache.org> on 2014/12/24 23:31:13 UTC
[jira] [Commented] (SPARK-4940) Document or Support more evenly
distributing cores for Mesos mode
[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258570#comment-14258570 ]
Timothy Chen commented on SPARK-4940:
-------------------------------------
Potentially I think there are two ways to help distribute the allocation:
1) Use static reservation in mesos side, to reserve resources just for Spark which gurantees the coarse grained mode will use them. The downside is that when Spark doesn't need these resources it can't be shared with other frameworks.
2) In the Spark scheduler side we can perhaps have a minimum and maximum cpu allocation count, so that besides just requiring 1 cpu we also ask to have the required cores to be within a range, so it's much more evenly allocated.
> Document or Support more evenly distributing cores for Mesos mode
> -----------------------------------------------------------------
>
> Key: SPARK-4940
> URL: https://issues.apache.org/jira/browse/SPARK-4940
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Reporter: Timothy Chen
>
> Currently in Coarse grain mode the spark scheduler simply takes all the resources it can on each node, but can cause uneven distribution based on resources available on each slave.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org