[jira] [Commented] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60

["Rick Branson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14084927#comment-14084927 ] 

Rick Branson commented on CASSANDRA-7432:
-----------------------------------------

Very happy with this in production. Enabling -XX:+CMSParallelInitialMarkEnabled and -XX:+CMSEdenChunksRecordAlways has *drastically* improved the STW portions of CMS for our nodes with 4GB+ young gen. Not seeing any downsides to enabling these options either.

We're also running a few more options that have trade-offs which could be considered:

-XX:+CMSScavengeBeforeRemark which has helped reduce the pause times of STW portions of CMS remark with large eden spaces, since it's dependent on how much live garbage is in the eden space. If CMS is ran infrequently (every few minutes or so) I can't see how this option could hurt people even with small eden spaces.

-XX:CMSWaitDuration=10000. This is how long the CMS will wait for a young gen collection before running the initial mark. It waits for a collection because the stop-the-world initial mark time depends on how many live objects are in eden space. The default is 2000ms, which seems low to me. The trade-off is that if it waits too long, CMS could run out of time and a promotion failure could occur. This seems unlikely though at 10 seconds in any even remotely healthy scenario.

> Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
> --------------------------------------------------------------------
>
>                 Key: CASSANDRA-7432
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7432
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Packaging
>            Reporter: graham sanderson
>            Assignee: Brandon Williams
>             Fix For: 1.2.19
>
>         Attachments: 7432.txt
>
>
> The new flags in question are as follows:
> {code}
> -XX:+CMSParallelInitialMarkEnabled
> -XX:+CMSEdenChunksRecordAlways
> {code}
> Given we already have
> {code}
> JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" 
> JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" 
> JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" 
> JVM_OPTS="$JVM_OPTS -XX:+UseTLAB"
> if [ "$JVM_ARCH" = "64-Bit" ] ; then
>     JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark"
> fi
> {code}
> The assumption would be that people are at least running on large number CPU cores/threads
> I would therefore recommend defaulting these flags if available - the only two possible downsides for {{+CMSEdenChunksRecordAlways}}:
> 1) There is a new very short (probably un-contended) lock in the "slow" (non TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't detected this timing wise - this is the "slow" path after all
> 2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the default) *and* you call {{System.gc()}} then  {{+CMSEdenChunksRecordAlways}} will expose you to a possible seg fault: (see
> [http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809])



--
This message was sent by Atlassian JIRA
(v6.2#6252)