You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2015/02/06 10:39:19 UTC
spark git commit: [SPARK-5157][YARN] Configure more JVM options
properly when we use ConcMarkSweepGC for AM.
Repository: spark
Updated Branches:
refs/heads/master f6ba813af -> 24dbc50b9
[SPARK-5157][YARN] Configure more JVM options properly when we use ConcMarkSweepGC for AM.
When we set `SPARK_USE_CONC_INCR_GC`, ConcurrentMarkSweepGC works on the AM.
Actually, if ConcurrentMarkSweepGC is set for the JVM, following JVM options are set automatically and implicitly.
* MaxTenuringThreshold=0
* SurvivorRatio=1024
Those can not be proper value for most cases.
See also http://www.oracle.com/technetwork/java/tuning-139912.html
Author: Kousuke Saruta <sa...@oss.nttdata.co.jp>
Closes #3956 from sarutak/SPARK-5157 and squashes the following commits:
c15da4e [Kousuke Saruta] Set more JVM options for AM when enabling CMS
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/24dbc50b
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/24dbc50b
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/24dbc50b
Branch: refs/heads/master
Commit: 24dbc50b934bfcd9fc3fd91cfd9966b75b250ad7
Parents: f6ba813
Author: Kousuke Saruta <sa...@oss.nttdata.co.jp>
Authored: Fri Feb 6 09:39:12 2015 +0000
Committer: Sean Owen <so...@cloudera.com>
Committed: Fri Feb 6 09:39:12 2015 +0000
----------------------------------------------------------------------
yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala | 2 ++
1 file changed, 2 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/24dbc50b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
----------------------------------------------------------------------
diff --git a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
index 91e8574..e700509 100644
--- a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
+++ b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
@@ -418,6 +418,8 @@ private[spark] class Client(
// In our expts, using (default) throughput collector has severe perf ramifications in
// multi-tenant machines
javaOpts += "-XX:+UseConcMarkSweepGC"
+ javaOpts += "-XX:MaxTenuringThreshold=31"
+ javaOpts += "-XX:SurvivorRatio=8"
javaOpts += "-XX:+CMSIncrementalMode"
javaOpts += "-XX:+CMSIncrementalPacing"
javaOpts += "-XX:CMSIncrementalDutyCycleMin=0"
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org