You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by an...@apache.org on 2015/07/09 22:21:26 UTC
spark git commit: [SPARK-8953] SPARK_EXECUTOR_CORES is not read in
SparkSubmit
Repository: spark
Updated Branches:
refs/heads/master 7ce3b818f -> 930fe9535
[SPARK-8953] SPARK_EXECUTOR_CORES is not read in SparkSubmit
The configuration ```SPARK_EXECUTOR_CORES``` won't put into ```SparkConf```, so it has no effect to the dynamic executor allocation.
Author: xutingjun <xu...@huawei.com>
Closes #7322 from XuTingjun/SPARK_EXECUTOR_CORES and squashes the following commits:
2cafa89 [xutingjun] make SPARK_EXECUTOR_CORES has effect to dynamicAllocation
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/930fe953
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/930fe953
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/930fe953
Branch: refs/heads/master
Commit: 930fe95350f8865e2af2d7afa5b717210933cd43
Parents: 7ce3b81
Author: xutingjun <xu...@huawei.com>
Authored: Thu Jul 9 13:21:10 2015 -0700
Committer: Andrew Or <an...@databricks.com>
Committed: Thu Jul 9 13:21:14 2015 -0700
----------------------------------------------------------------------
.../main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/930fe953/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala b/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
index 73ab183..6e3c0b2 100644
--- a/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
@@ -162,6 +162,7 @@ private[deploy] class SparkSubmitArguments(args: Seq[String], env: Map[String, S
.orNull
executorCores = Option(executorCores)
.orElse(sparkProperties.get("spark.executor.cores"))
+ .orElse(env.get("SPARK_EXECUTOR_CORES"))
.orNull
totalExecutorCores = Option(totalExecutorCores)
.orElse(sparkProperties.get("spark.cores.max"))
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org