You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by bl...@apache.org on 2017/12/12 09:55:47 UTC
[09/10] cassandra git commit: Merge branch cassandra-3.0 into
cassandra-3.11
Merge branch cassandra-3.0 into cassandra-3.11
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a7c45be9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a7c45be9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a7c45be9
Branch: refs/heads/trunk
Commit: a7c45be936bf8aa04b657529964c41b0d82c50e2
Parents: 88a41fb dd187d1
Author: Benjamin Lerer <b....@gmail.com>
Authored: Tue Dec 12 10:38:33 2017 +0100
Committer: Benjamin Lerer <b....@gmail.com>
Committed: Tue Dec 12 10:42:50 2017 +0100
----------------------------------------------------------------------
CHANGES.txt | 2 +-
NEWS.txt | 8 +-
bin/cassandra | 19 +-
conf/cassandra-env.ps1 | 914 ++++++++++---------
conf/cassandra-env.sh | 12 +
.../apache/cassandra/service/StartupChecks.java | 74 ++
.../org/apache/cassandra/utils/HeapUtils.java | 93 +-
.../cassandra/utils/JVMStabilityInspector.java | 24 +-
.../utils/JVMStabilityInspectorTest.java | 28 +-
9 files changed, 632 insertions(+), 542 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index 6e9a0bd,20ccc4b..116c7e9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,4 +1,14 @@@
-3.0.16
+3.11.2
+ * Fix imbalanced disks when replacing node with same address with JBOD (CASSANDRA-14084)
+ * Reload compaction strategies when disk boundaries are invalidated (CASSANDRA-13948)
+ * Remove OpenJDK log warning (CASSANDRA-13916)
+ * Prevent compaction strategies from looping indefinitely (CASSANDRA-14079)
+ * Cache disk boundaries (CASSANDRA-13215)
+ * Add asm jar to build.xml for maven builds (CASSANDRA-11193)
+ * Round buffer size to powers of 2 for the chunk cache (CASSANDRA-13897)
+ * Update jackson JSON jars (CASSANDRA-13949)
+ * Avoid locks when checking LCS fanout and if we should defrag (CASSANDRA-13930)
+Merged from 3.0:
- 3.0.16
* Fix SStable ordering by max timestamp in SinglePartitionReadCommand (CASSANDRA-14010)
* Accept role names containing forward-slash (CASSANDRA-14088)
* Optimize CRC check chance probability calculations (CASSANDRA-14094)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/NEWS.txt
----------------------------------------------------------------------
diff --cc NEWS.txt
index 9812b22,1bbe1aa..0c32278
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -18,7 -18,13 +18,13 @@@ using the provided 'sstableupgrade' too
Upgrading
---------
- - Nothing specific to this release, but please see previous upgrading sections.
- - Cassandra is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
- rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
- are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
- options are used. See CASSANDRA-13006 for more details.
- - Cassandra is not logging anymore by default an Heap histogram on OutOfMemoryError. To enable that behavior
- set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property to 'true'. See CASSANDRA-13006
- for more details.
++ - Cassandra is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
++ rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
++ are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
++ options are used. See CASSANDRA-13006 for more details
++ - Cassandra is not logging anymore by default an Heap histogram on OutOfMemoryError. To enable that behavior
++ set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property to 'true'. See CASSANDRA-13006
++ for more details.
Materialized Views
-------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/bin/cassandra
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/conf/cassandra-env.ps1
----------------------------------------------------------------------
diff --cc conf/cassandra-env.ps1
index 2df7054,405ed92..49d03ce
--- a/conf/cassandra-env.ps1
+++ b/conf/cassandra-env.ps1
@@@ -393,70 -393,98 +393,80 @@@ Function SetCassandraEnvironmen
$env:JVM_OPTS="$env:JVM_OPTS -XX:HeapDumpPath=$env:CASSANDRA_HEAPDUMP_DIR\cassandra-$unixTimestamp-pid$pid.hprof"
}
- if ($env:JVM_VERSION.CompareTo("1.8.0") -eq -1 -or [convert]::ToInt32($env:JVM_PATCH_VERSION) -lt 40)
- {
- echo "Cassandra 3.0 and later require Java 8u40 or later."
- exit
- }
-
- # Specifies the default port over which Cassandra will be available for
- # JMX connections.
- $JMX_PORT="7199"
-
- # store in env to check if it's avail in verification
- $env:JMX_PORT=$JMX_PORT
-
- # Configure the following for JEMallocAllocator and if jemalloc is not available in the system
- # library path.
- # set LD_LIBRARY_PATH=<JEMALLOC_HOME>/lib/
- # $env:JVM_OPTS="$env:JVM_OPTS -Djava.library.path=<JEMALLOC_HOME>/lib/"
-
- # jmx: metrics and administration interface
- #
- # add this if you're having trouble connecting:
- # $env:JVM_OPTS="$env:JVM_OPTS -Djava.rmi.server.hostname=<public name>"
- #
- # see
- # https://blogs.oracle.com/jmxetc/entry/troubleshooting_connection_problems_in_jconsole
- # for more on configuring JMX through firewalls, etc. (Short version:
- # get it working with no firewall first.)
- #
- # Due to potential security exploits, Cassandra ships with JMX accessible
- # *only* from localhost. To enable remote JMX connections, uncomment lines below
- # with authentication and ssl enabled. See https://wiki.apache.org/cassandra/JmxSecurity
- #
- #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT"
- #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT"
- #
- # JMX SSL options
- #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.ssl=true"
- #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.ssl.need.client.auth=true"
- #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.ssl.enabled.protocols=<enabled-protocols>"
- #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.ssl.enabled.cipher.suites=<enabled-cipher-suites>"
- #$env:JVM_OPTS="$env:JVM_OPTS -Djavax.net.ssl.keyStore=C:/keystore"
- #$env:JVM_OPTS="$env:JVM_OPTS -Djavax.net.ssl.keyStorePassword=<keystore-password>"
- #$env:JVM_OPTS="$env:JVM_OPTS -Djavax.net.ssl.trustStore=C:/truststore"
- #$env:JVM_OPTS="$env:JVM_OPTS -Djavax.net.ssl.trustStorePassword=<truststore-password>"
- #
- # JMX auth options
- #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.authenticate=true"
- ## Basic file based authn & authz
- #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.password.file=C:/jmxremote.password"
- #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.access.file=C:/jmxremote.access"
-
- ## Custom auth settings which can be used as alternatives to JMX's out of the box auth utilities.
- ## JAAS login modules can be used for authentication by uncommenting these two properties.
- ## Cassandra ships with a LoginModule implementation - org.apache.cassandra.auth.CassandraLoginModule -
- ## which delegates to the IAuthenticator configured in cassandra.yaml
- #$env:JVM_OPTS="$env:JVM_OPTS -Dcassandra.jmx.remote.login.config=CassandraLogin"
- #$env:JVM_OPTS="$env:JVM_OPTS -Djava.security.auth.login.config=C:/cassandra-jaas.config"
-
- ## Cassandra also ships with a helper for delegating JMX authz calls to the configured IAuthorizer,
- ## uncomment this to use it. Requires one of the two authentication options to be enabled
- #$env:JVM_OPTS="$env:JVM_OPTS -Dcassandra.jmx.authorizer=org.apache.cassandra.auth.jmx.AuthorizationProxy"
-
- # Default JMX setup, bound to local loopback address only
- $env:JVM_OPTS="$env:JVM_OPTS -Dcassandra.jmx.local.port=$JMX_PORT"
-
- $env:JVM_OPTS="$env:JVM_OPTS $env:JVM_EXTRA_OPTS"
- }
- if ($env:JVM_VERSION.CompareTo("1.8.0") -eq -1 -or [convert]::ToInt32($env:JVM_PATCH_VERSION) -lt 40)
- {
- echo "Cassandra 3.0 and later require Java 8u40 or later."
- exit
- }
-
- # enable assertions. disabling this in production will give a modest
- # performance benefit (around 5%).
- $env:JVM_OPTS = "$env:JVM_OPTS -ea"
-
- # Specifies the default port over which Cassandra will be available for
- # JMX connections.
- $JMX_PORT="7199"
-
- # store in env to check if it's avail in verification
- $env:JMX_PORT=$JMX_PORT
-
- # enable thread priorities, primarily so we can give periodic tasks
- # a lower priority to avoid interfering with client workload
- $env:JVM_OPTS="$env:JVM_OPTS -XX:+UseThreadPriorities"
- # allows lowering thread priority without being root on linux - probably
- # not necessary on Windows but doesn't harm anything.
- # see http://tech.stolsvik.com/2010/01/linux-java-thread-priorities-workar
- $env:JVM_OPTS="$env:JVM_OPTS -XX:ThreadPriorityPolicy=42"
-
- $env:JVM_OPTS="$env:JVM_OPTS -XX:+HeapDumpOnOutOfMemoryError"
-
+ # stop the jvm on OutOfMemoryError as it can result in some data corruption
+ # uncomment the preferred option
+ # ExitOnOutOfMemoryError and CrashOnOutOfMemoryError require a JRE greater or equals to 1.7 update 101 or 1.8 update 92
+ # $env:JVM_OPTS="$env:JVM_OPTS -XX:+ExitOnOutOfMemoryError"
+ # $env:JVM_OPTS="$env:JVM_OPTS -XX:+CrashOnOutOfMemoryError"
+ $env:JVM_OPTS="$env:JVM_OPTS -XX:OnOutOfMemoryError=""taskkill /F /PID %p"""
+
+ # print an heap histogram on OutOfMemoryError
+ # $env:JVM_OPTS="$env:JVM_OPTS -Dcassandra.printHeapHistogramOnOutOfMemoryError=true"
+
- # Per-thread stack size.
- $env:JVM_OPTS="$env:JVM_OPTS -Xss256k"
-
- # Larger interned string table, for gossip's benefit (CASSANDRA-6410)
- $env:JVM_OPTS="$env:JVM_OPTS -XX:StringTableSize=1000003"
-
- # Make sure all memory is faulted and zeroed on startup.
- # This helps prevent soft faults in containers and makes
- # transparent hugepage allocation more effective.
- #$env:JVM_OPTS="$env:JVM_OPTS -XX:+AlwaysPreTouch"
-
- # Biased locking does not benefit Cassandra.
- $env:JVM_OPTS="$env:JVM_OPTS -XX:-UseBiasedLocking"
++ if ($env:JVM_VERSION.CompareTo("1.8.0") -eq -1 -or [convert]::ToInt32($env:JVM_PATCH_VERSION) -lt 40)
++ {
++ echo "Cassandra 3.0 and later require Java 8u40 or later."
++ exit
++ }
+
- # Enable thread-local allocation blocks and allow the JVM to automatically
- # resize them at runtime.
- $env:JVM_OPTS="$env:JVM_OPTS -XX:+UseTLAB -XX:+ResizeTLAB"
++ # Specifies the default port over which Cassandra will be available for
++ # JMX connections.
++ $JMX_PORT="7199"
+
- # http://www.evanjones.ca/jvm-mmap-pause.html
- $env:JVM_OPTS="$env:JVM_OPTS -XX:+PerfDisableSharedMem"
++ # store in env to check if it's avail in verification
++ $env:JMX_PORT=$JMX_PORT
+
+ # Configure the following for JEMallocAllocator and if jemalloc is not available in the system
+ # library path.
+ # set LD_LIBRARY_PATH=<JEMALLOC_HOME>/lib/
+ # $env:JVM_OPTS="$env:JVM_OPTS -Djava.library.path=<JEMALLOC_HOME>/lib/"
+
- # uncomment to have Cassandra JVM listen for remote debuggers/profilers on port 1414
- # $env:JVM_OPTS="$env:JVM_OPTS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=1414"
-
- # Prefer binding to IPv4 network intefaces (when net.ipv6.bindv6only=1). See
- # http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6342561 (short version:
- # comment out this entry to enable IPv6 support).
- $env:JVM_OPTS="$env:JVM_OPTS -Djava.net.preferIPv4Stack=true"
-
+ # jmx: metrics and administration interface
+ #
+ # add this if you're having trouble connecting:
+ # $env:JVM_OPTS="$env:JVM_OPTS -Djava.rmi.server.hostname=<public name>"
+ #
+ # see
+ # https://blogs.oracle.com/jmxetc/entry/troubleshooting_connection_problems_in_jconsole
+ # for more on configuring JMX through firewalls, etc. (Short version:
+ # get it working with no firewall first.)
+ #
+ # Due to potential security exploits, Cassandra ships with JMX accessible
+ # *only* from localhost. To enable remote JMX connections, uncomment lines below
+ # with authentication and ssl enabled. See https://wiki.apache.org/cassandra/JmxSecurity
+ #
+ #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT"
- #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.ssl=false"
++ #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT"
++ #
++ # JMX SSL options
++ #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.ssl=true"
++ #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.ssl.need.client.auth=true"
++ #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.ssl.enabled.protocols=<enabled-protocols>"
++ #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.ssl.enabled.cipher.suites=<enabled-cipher-suites>"
++ #$env:JVM_OPTS="$env:JVM_OPTS -Djavax.net.ssl.keyStore=C:/keystore"
++ #$env:JVM_OPTS="$env:JVM_OPTS -Djavax.net.ssl.keyStorePassword=<keystore-password>"
++ #$env:JVM_OPTS="$env:JVM_OPTS -Djavax.net.ssl.trustStore=C:/truststore"
++ #$env:JVM_OPTS="$env:JVM_OPTS -Djavax.net.ssl.trustStorePassword=<truststore-password>"
++ #
++ # JMX auth options
+ #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.authenticate=true"
++ ## Basic file based authn & authz
+ #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.password.file=C:/jmxremote.password"
- $env:JVM_OPTS="$env:JVM_OPTS -Dcassandra.jmx.local.port=$JMX_PORT -XX:+DisableExplicitGC"
++ #$env:JVM_OPTS="$env:JVM_OPTS -Dcom.sun.management.jmxremote.access.file=C:/jmxremote.access"
+
- $env:JVM_OPTS="$env:JVM_OPTS $env:JVM_EXTRA_OPTS"
++ ## Custom auth settings which can be used as alternatives to JMX's out of the box auth utilities.
++ ## JAAS login modules can be used for authentication by uncommenting these two properties.
++ ## Cassandra ships with a LoginModule implementation - org.apache.cassandra.auth.CassandraLoginModule -
++ ## which delegates to the IAuthenticator configured in cassandra.yaml
++ #$env:JVM_OPTS="$env:JVM_OPTS -Dcassandra.jmx.remote.login.config=CassandraLogin"
++ #$env:JVM_OPTS="$env:JVM_OPTS -Djava.security.auth.login.config=C:/cassandra-jaas.config"
++
++ ## Cassandra also ships with a helper for delegating JMX authz calls to the configured IAuthorizer,
++ ## uncomment this to use it. Requires one of the two authentication options to be enabled
++ #$env:JVM_OPTS="$env:JVM_OPTS -Dcassandra.jmx.authorizer=org.apache.cassandra.auth.jmx.AuthorizationProxy"
+
- #$env:JVM_OPTS="$env:JVM_OPTS -XX:+UnlockCommercialFeatures -XX:+FlightRecorder"
++ # Default JMX setup, bound to local loopback address only
++ $env:JVM_OPTS="$env:JVM_OPTS -Dcassandra.jmx.local.port=$JMX_PORT"
++
++ $env:JVM_OPTS="$env:JVM_OPTS $env:JVM_EXTRA_OPTS"
+ }
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/conf/cassandra-env.sh
----------------------------------------------------------------------
diff --cc conf/cassandra-env.sh
index 5a02f79,ffc773f..189576b
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@@ -214,6 -247,30 +214,18 @@@ if [ "x$CASSANDRA_HEAPDUMP_DIR" != "x"
JVM_OPTS="$JVM_OPTS -XX:HeapDumpPath=$CASSANDRA_HEAPDUMP_DIR/cassandra-`date +%s`-pid$$.hprof"
fi
+ # stop the jvm on OutOfMemoryError as it can result in some data corruption
+ # uncomment the preferred option
+ # ExitOnOutOfMemoryError and CrashOnOutOfMemoryError require a JRE greater or equals to 1.7 update 101 or 1.8 update 92
+ # For OnOutOfMemoryError we cannot use the JVM_OPTS variables because bash commands split words
+ # on white spaces without taking quotes into account
+ # JVM_OPTS="$JVM_OPTS -XX:+ExitOnOutOfMemoryError"
+ # JVM_OPTS="$JVM_OPTS -XX:+CrashOnOutOfMemoryError"
+ JVM_ON_OUT_OF_MEMORY_ERROR_OPT="-XX:OnOutOfMemoryError=kill -9 %p"
+
+ # print an heap histogram on OutOfMemoryError
+ # JVM_OPTS="$JVM_OPTS -Dcassandra.printHeapHistogramOnOutOfMemoryError=true"
+
-# uncomment to have Cassandra JVM listen for remote debuggers/profilers on port 1414
-# JVM_OPTS="$JVM_OPTS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=1414"
-
-# uncomment to have Cassandra JVM log internal method compilation (developers only)
-# JVM_OPTS="$JVM_OPTS -XX:+UnlockDiagnosticVMOptions -XX:+LogCompilation"
-# JVM_OPTS="$JVM_OPTS -XX:+UnlockCommercialFeatures -XX:+FlightRecorder"
-
-# Prefer binding to IPv4 network intefaces (when net.ipv6.bindv6only=1). See
-# http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6342561 (short version:
-# comment out this entry to enable IPv6 support).
-JVM_OPTS="$JVM_OPTS -Djava.net.preferIPv4Stack=true"
-
# jmx: metrics and administration interface
#
# add this if you're having trouble connecting:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/src/java/org/apache/cassandra/service/StartupChecks.java
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/src/java/org/apache/cassandra/utils/HeapUtils.java
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/src/java/org/apache/cassandra/utils/JVMStabilityInspector.java
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7c45be9/test/unit/org/apache/cassandra/utils/JVMStabilityInspectorTest.java
----------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org