You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by an...@apache.org on 2015/06/10 22:30:19 UTC
spark git commit: [SPARK-8290] spark class command builder need read
SPARK_JAVA_OPTS and SPARK_DRIVER_MEMORY properly
Repository: spark
Updated Branches:
refs/heads/master 80043e9e7 -> cb871c44c
[SPARK-8290] spark class command builder need read SPARK_JAVA_OPTS and SPARK_DRIVER_MEMORY properly
SPARK_JAVA_OPTS was missed in reconstructing the launcher part, we should add it back so process launched by spark-class could read it properly. And so does `SPARK_DRIVER_MEMORY`.
The missing part is [here](https://github.com/apache/spark/blob/1c30afdf94b27e1ad65df0735575306e65d148a1/bin/spark-class#L97).
Author: WangTaoTheTonic <wa...@huawei.com>
Author: Tao Wang <wa...@huawei.com>
Closes #6741 from WangTaoTheTonic/SPARK-8290 and squashes the following commits:
bd89f0f [Tao Wang] make sure the memory setting is right too
e313520 [WangTaoTheTonic] spark class command builder need read SPARK_JAVA_OPTS
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/cb871c44
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/cb871c44
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/cb871c44
Branch: refs/heads/master
Commit: cb871c44c38a4c1575ed076389f14641afafad7d
Parents: 80043e9
Author: WangTaoTheTonic <wa...@huawei.com>
Authored: Wed Jun 10 13:30:16 2015 -0700
Committer: Andrew Or <an...@databricks.com>
Committed: Wed Jun 10 13:30:16 2015 -0700
----------------------------------------------------------------------
.../java/org/apache/spark/launcher/SparkClassCommandBuilder.java | 3 +++
1 file changed, 3 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/cb871c44/launcher/src/main/java/org/apache/spark/launcher/SparkClassCommandBuilder.java
----------------------------------------------------------------------
diff --git a/launcher/src/main/java/org/apache/spark/launcher/SparkClassCommandBuilder.java b/launcher/src/main/java/org/apache/spark/launcher/SparkClassCommandBuilder.java
index d80abf2..de85720 100644
--- a/launcher/src/main/java/org/apache/spark/launcher/SparkClassCommandBuilder.java
+++ b/launcher/src/main/java/org/apache/spark/launcher/SparkClassCommandBuilder.java
@@ -93,6 +93,9 @@ class SparkClassCommandBuilder extends AbstractCommandBuilder {
toolsDir.getAbsolutePath(), className);
javaOptsKeys.add("SPARK_JAVA_OPTS");
+ } else {
+ javaOptsKeys.add("SPARK_JAVA_OPTS");
+ memKey = "SPARK_DRIVER_MEMORY";
}
List<String> cmd = buildJavaCommand(extraClassPath);
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org