You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2019/10/08 05:44:17 UTC
[jira] [Resolved] (SPARK-9775) Query Mesos for number of CPUs to
set default parallelism
[ https://issues.apache.org/jira/browse/SPARK-9775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-9775.
---------------------------------
Resolution: Incomplete
> Query Mesos for number of CPUs to set default parallelism
> ---------------------------------------------------------
>
> Key: SPARK-9775
> URL: https://issues.apache.org/jira/browse/SPARK-9775
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Affects Versions: 1.4.1
> Reporter: Peder Ås
> Priority: Minor
> Labels: bulk-closed
>
> As highlighted in a TODO on line 400 of MesosSchedulerBackend.scala (at least on 3ca995b7) we should query the Mesos master and set the default parallelism based on the number of CPUs available in the cluster (and multiply by two or three?)
> See code in question [here (gitweb)|https://git-wip-us.apache.org/repos/asf?p=spark.git;a=blob;f=core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala;h=3f63ec1c;hb=HEAD#l400].
> This task should also update the documentation [here (gitweb)|https://git-wip-us.apache.org/repos/asf?p=spark.git;a=blob;f=docs/configuration.md;h=c60dd1;hb=HEAD#l789] to highlight the fact.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org