You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Eron Wright (JIRA)" <ji...@apache.org> on 2016/01/26 02:37:39 UTC
[jira] [Commented] (SPARK-4940) Support more evenly distributing
cores for Mesos mode
[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15116478#comment-15116478 ]
Eron Wright commented on SPARK-4940:
-------------------------------------
Another benefit of maximizing the # of allocated executors is improved data locality, assuming that HDFS is running on most/all slaves. In other words, executor allocation isn't data-aware, but Spark task allocation is.
> Support more evenly distributing cores for Mesos mode
> -----------------------------------------------------
>
> Key: SPARK-4940
> URL: https://issues.apache.org/jira/browse/SPARK-4940
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Reporter: Timothy Chen
> Attachments: mesos-config-difference-3nodes-vs-2nodes.png
>
>
> Currently in Coarse grain mode the spark scheduler simply takes all the resources it can on each node, but can cause uneven distribution based on resources available on each slave.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org