You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by va...@apache.org on 2015/09/17 18:52:59 UTC
spark git commit: [SPARK-10660] Doc describe error in the "Running
Spark on YARN" page
Repository: spark
Updated Branches:
refs/heads/master 2a508df20 -> c88bb5df9
[SPARK-10660] Doc describe error in the "Running Spark on YARN" page
In the Configuration section, the **spark.yarn.driver.memoryOverhead** and **spark.yarn.am.memoryOverhead**‘s default value should be "driverMemory * 0.10, with minimum of 384" and "AM memory * 0.10, with minimum of 384" respectively. Because from Spark 1.4.0, the **MEMORY_OVERHEAD_FACTOR** is set to 0.1.0, not 0.07.
Author: yangping.wu <wy...@163.com>
Closes #8797 from 397090770/SparkOnYarnDocError.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c88bb5df
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c88bb5df
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c88bb5df
Branch: refs/heads/master
Commit: c88bb5df94f9696677c3a429472114bc66f32a52
Parents: 2a508df
Author: yangping.wu <wy...@163.com>
Authored: Thu Sep 17 09:52:40 2015 -0700
Committer: Marcelo Vanzin <va...@cloudera.com>
Committed: Thu Sep 17 09:52:40 2015 -0700
----------------------------------------------------------------------
docs/running-on-yarn.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/c88bb5df/docs/running-on-yarn.md
----------------------------------------------------------------------
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md
index d124432..3a961d2 100644
--- a/docs/running-on-yarn.md
+++ b/docs/running-on-yarn.md
@@ -211,14 +211,14 @@ If you need a reference to the proper location to put log files in the YARN so t
</tr>
<tr>
<td><code>spark.yarn.driver.memoryOverhead</code></td>
- <td>driverMemory * 0.07, with minimum of 384 </td>
+ <td>driverMemory * 0.10, with minimum of 384 </td>
<td>
The amount of off heap memory (in megabytes) to be allocated per driver in cluster mode. This is memory that accounts for things like VM overheads, interned strings, other native overheads, etc. This tends to grow with the container size (typically 6-10%).
</td>
</tr>
<tr>
<td><code>spark.yarn.am.memoryOverhead</code></td>
- <td>AM memory * 0.07, with minimum of 384 </td>
+ <td>AM memory * 0.10, with minimum of 384 </td>
<td>
Same as <code>spark.yarn.driver.memoryOverhead</code>, but for the Application Master in client mode.
</td>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org