You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by tg...@apache.org on 2016/04/14 17:29:23 UTC
spark git commit: [SPARK-14572][DOC] Update config docs to allow -Xms
in extraJavaOptions
Repository: spark
Updated Branches:
refs/heads/master 3cf3db17b -> f83ba454a
[SPARK-14572][DOC] Update config docs to allow -Xms in extraJavaOptions
## What changes were proposed in this pull request?
The configuration docs are updated to reflect the changes introduced with [SPARK-12384](https://issues.apache.org/jira/browse/SPARK-12384). This allows the user to specify initial heap memory settings through the extraJavaOptions for executor, driver and am.
## How was this patch tested?
The changes are tested in [SPARK-12384](https://issues.apache.org/jira/browse/SPARK-12384). This is just documenting the changes made.
Author: Dhruve Ashar <dh...@gmail.com>
Closes #12333 from dhruve/doc/SPARK-14572.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/f83ba454
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/f83ba454
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/f83ba454
Branch: refs/heads/master
Commit: f83ba454a507bec0cc389d9a382cd71add7f17c1
Parents: 3cf3db1
Author: Dhruve Ashar <dh...@gmail.com>
Authored: Thu Apr 14 10:29:14 2016 -0500
Committer: Tom Graves <tg...@yahoo-inc.com>
Committed: Thu Apr 14 10:29:14 2016 -0500
----------------------------------------------------------------------
docs/configuration.md | 11 +++++++----
docs/running-on-yarn.md | 4 +++-
2 files changed, 10 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/f83ba454/docs/configuration.md
----------------------------------------------------------------------
diff --git a/docs/configuration.md b/docs/configuration.md
index 937852f..16d5be6 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -225,11 +225,14 @@ Apart from these, the following properties are also available, and may be useful
<td>(none)</td>
<td>
A string of extra JVM options to pass to the driver. For instance, GC settings or other logging.
+ Note that it is illegal to set maximum heap size (-Xmx) settings with this option. Maximum heap
+ size settings can be set with <code>spark.driver.memory</code> in the cluster mode and through
+ the <code>--driver-memory</code> command line option in the client mode.
<br /><em>Note:</em> In client mode, this config must not be set through the <code>SparkConf</code>
directly in your application, because the driver JVM has already started at that point.
Instead, please set this through the <code>--driver-java-options</code> command line option or in
- your default properties file.</td>
+ your default properties file.
</td>
</tr>
<tr>
@@ -269,9 +272,9 @@ Apart from these, the following properties are also available, and may be useful
<td>(none)</td>
<td>
A string of extra JVM options to pass to executors. For instance, GC settings or other logging.
- Note that it is illegal to set Spark properties or heap size settings with this option. Spark
- properties should be set using a SparkConf object or the spark-defaults.conf file used with the
- spark-submit script. Heap size settings can be set with spark.executor.memory.
+ Note that it is illegal to set Spark properties or maximum heap size (-Xmx) settings with this
+ option. Spark properties should be set using a SparkConf object or the spark-defaults.conf file
+ used with the spark-submit script. Maximum heap size settings can be set with spark.executor.memory.
</td>
</tr>
<tr>
http://git-wip-us.apache.org/repos/asf/spark/blob/f83ba454/docs/running-on-yarn.md
----------------------------------------------------------------------
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md
index ddc75a7..09701ab 100644
--- a/docs/running-on-yarn.md
+++ b/docs/running-on-yarn.md
@@ -342,7 +342,9 @@ If you need a reference to the proper location to put log files in the YARN so t
<td>(none)</td>
<td>
A string of extra JVM options to pass to the YARN Application Master in client mode.
- In cluster mode, use <code>spark.driver.extraJavaOptions</code> instead.
+ In cluster mode, use <code>spark.driver.extraJavaOptions</code> instead. Note that it is illegal
+ to set maximum heap size (-Xmx) settings with this option. Maximum heap size settings can be set
+ with <code>spark.yarn.am.memory</code>
</td>
</tr>
<tr>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org