You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by bl...@apache.org on 2017/12/12 09:55:48 UTC
[10/10] cassandra git commit: Merge branch cassandra-3.11 into trunk
Merge branch cassandra-3.11 into trunk
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/17e602b7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/17e602b7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/17e602b7
Branch: refs/heads/trunk
Commit: 17e602b7d248c61634c31c93b7d244f35cfcefd3
Parents: 8e95534 a7c45be
Author: Benjamin Lerer <b....@gmail.com>
Authored: Tue Dec 12 10:52:23 2017 +0100
Committer: Benjamin Lerer <b....@gmail.com>
Committed: Tue Dec 12 10:52:23 2017 +0100
----------------------------------------------------------------------
CHANGES.txt | 2 +-
NEWS.txt | 13 +
bin/cassandra | 19 +-
conf/cassandra-env.ps1 | 914 ++++++++++---------
conf/cassandra-env.sh | 12 +
.../apache/cassandra/service/StartupChecks.java | 74 ++
.../org/apache/cassandra/utils/HeapUtils.java | 93 +-
.../cassandra/utils/JVMStabilityInspector.java | 24 +-
.../utils/JVMStabilityInspectorTest.java | 28 +-
9 files changed, 638 insertions(+), 541 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/17e602b7/CHANGES.txt
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/17e602b7/NEWS.txt
----------------------------------------------------------------------
diff --cc NEWS.txt
index 7bb9152,0c32278..259507e
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -13,103 -13,27 +13,116 @@@ restore snapshots created with the prev
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.
+4.0
+===
+
+New features
+------------
+ - An experimental option to compare all merkle trees together has been added - for example, in
+ a 3 node cluster with 2 replicas identical and 1 out-of-date, with this option enabled, the
+ out-of-date replica will only stream a single copy from up-to-date replica. Enable it by adding
+ "-os" to nodetool repair. See CASSANDRA-3200.
+ - The currentTimestamp, currentDate, currentTime and currentTimeUUID functions have been added.
+ See CASSANDRA-13132
+ - Support for arithmetic operations between `timestamp`/`date` and `duration` has been added.
+ See CASSANDRA-11936
+ - Support for arithmetic operations on number has been added. See CASSANDRA-11935
+ - Preview expected streaming required for a repair (nodetool repair --preview), and validate the
+ consistency of repaired data between nodes (nodetool repair --validate). See CASSANDRA-13257
+ - Support for selecting Map values and Set elements has been added for SELECT queries. See CASSANDRA-7396
+ - Change-Data-Capture has been modified to make CommitLogSegments available
+ immediately upon creation via hard-linking the files. This means that incomplete
+ segments will be available in cdc_raw rather than fully flushed. See documentation
+ and CASSANDRA-12148 for more detail.
+ - The initial build of materialized views can be parallelized. The number of concurrent builder
+ threads is specified by the property `cassandra.yaml:concurrent_materialized_view_builders`.
+ This property can be modified at runtime through both JMX and the new `setconcurrentviewbuilders`
+ and `getconcurrentviewbuilders` nodetool commands. See CASSANDRA-12245 for more details.
+ - There is now a binary full query log based on Chronicle Queue that can be controlled using
+ nodetool enablefullquerylog, disablefullquerylog, and resetfullquerylog. The log
+ contains all queries invoked, approximate time they were invoked, any parameters necessary
+ to bind wildcard values, and all query options. A human readable version of the log can be
+ dumped or tailed using the new bin/fqltool utility. The full query log is designed to be safe
+ to use in production and limits utilization of heap memory and disk space with limits
+ you can specify when enabling the log.
+ See nodetool and fqltool help text for more information.
+ - SSTableDump now supports the -l option to output each partition as it's own json object
+ See CASSANDRA-13848 for more detail
+
+Upgrading
+---------
+ - Cassandra 4.0 removed support for COMPACT STORAGE tables. All Compact Tables
+ have to be migrated using `ALTER ... DROP COMPACT STORAGE` statement in 3.0/3.11.
+ Cassandra starting 4.0 will not start if flags indicate that the table is non-CQL.
+ Syntax for creating compact tables is also deprecated.
+ - Support for legacy auth tables in the system_auth keyspace (users,
+ permissions, credentials) and the migration code has been removed. Migration
+ of these legacy auth tables must have been completed before the upgrade to
+ 4.0 and the legacy tables must have been removed. See the 'Upgrading' section
+ for version 2.2 for migration instructions.
+ - Cassandra 4.0 removed support for the deprecated Thrift interface. Amongst
+ other things, this implies the removal of all yaml options related to thrift
+ ('start_rpc', rpc_port, ...).
+ - Cassandra 4.0 removed support for any pre-3.0 format. This means you
+ cannot upgrade from a 2.x version to 4.0 directly, you have to upgrade to
+ a 3.0.x/3.x version first (and run upgradesstable). In particular, this
+ mean Cassandra 4.0 cannot load or read pre-3.0 sstables in any way: you
+ will need to upgrade those sstable in 3.0.x/3.x first.
+ - Upgrades from 3.0.x or 3.x are supported since 3.0.13 or 3.11.0, previous
+ versions will causes issues during rolling upgrades (CASSANDRA-13274).
+ - Cassandra will no longer allow invalid keyspace replication options, such
+ as invalid datacenter names for NetworkTopologyStrategy. Operators MUST
+ add new nodes to a datacenter before they can set set ALTER or CREATE
+ keyspace replication policies using that datacenter. Existing keyspaces
+ will continue to operate, but CREATE and ALTER will validate that all
+ datacenters specified exist in the cluster.
+ - Cassandra 4.0 fixes a problem with incremental repair which caused repaired
+ data to be inconsistent between nodes. The fix changes the behavior of both
+ full and incremental repairs. For full repairs, data is no longer marked
+ repaired. For incremental repairs, anticompaction is run at the beginning
+ of the repair, instead of at the end. If incremental repair was being used
+ prior to upgrading, a full repair should be run after upgrading to resolve
+ any inconsistencies.
+ - Config option index_interval has been removed (it was deprecated since 2.0)
+ - Deprecated repair JMX APIs are removed.
+ - The version of snappy-java has been upgraded to 1.1.2.6
+ - the miniumum value for internode message timeouts is 10ms. Previously, any
+ positive value was allowed. See cassandra.yaml entries like
+ read_request_timeout_in_ms for more details.
+ - Cassandra 4.0 allows a single port to be used for both secure and insecure
+ connections between cassandra nodes (CASSANDRA-10404). See the yaml for
+ specific property changes, and see the security doc for full details.
+ - Due to the parallelization of the initial build of materialized views,
+ the per token range view building status is stored in the new table
+ `system.view_builds_in_progress`. The old table `system.views_builds_in_progress`
+ is no longer used and can be removed. See CASSANDRA-12245 for more details.
+ - Config option commitlog_sync_batch_window_in_ms has been deprecated as it's
+ documentation has been incorrect and the setting itself near useless.
+ Batch mode remains a valid commit log mode, however.
+ - There is a new commit log mode, group, which is similar to batch mode
+ but blocks for up to a configurable number of milliseconds between disk flushes.
+
+Materialized Views
+-------------------
+ - Following a discussion regarding concerns about the design and safety of Materialized Views, the C* development
+ community no longer recommends them for production use, and considers them experimental. Warnings messages will
+ now be logged when they are created. (See https://www.mail-archive.com/dev@cassandra.apache.org/msg11511.html)
+ - An 'enable_materialized_views' flag has been added to cassandra.yaml to allow operators to prevent creation of
+ views
+
+ 3.11.2
+ ======
+
+ Upgrading
+ ---------
+ - Cassandra is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
+ rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
+ are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
+ options are used. See CASSANDRA-13006 for more details
+ - Cassandra is not logging anymore by default an Heap histogram on OutOfMemoryError. To enable that behavior
+ set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property to 'true'. See CASSANDRA-13006
+ for more details.
+
-Materialized Views
--------------------
- - Following a discussion regarding concerns about the design and safety of Materialized Views, the C* development
- community no longer recommends them for production use, and considers them experimental. Warnings messages will
- now be logged when they are created. (See https://www.mail-archive.com/dev@cassandra.apache.org/msg11511.html)
- - An 'enable_materialized_views' flag has been added to cassandra.yaml to allow operators to prevent creation of
- views
-
3.11.1
======
http://git-wip-us.apache.org/repos/asf/cassandra/blob/17e602b7/conf/cassandra-env.ps1
----------------------------------------------------------------------
diff --cc conf/cassandra-env.ps1
index 806eabc,49d03ce..72d632b
--- a/conf/cassandra-env.ps1
+++ b/conf/cassandra-env.ps1
@@@ -1,381 -1,381 +1,381 @@@
- #
- # Licensed to the Apache Software Foundation (ASF) under one or more
- # contributor license agreements. See the NOTICE file distributed with
- # this work for additional information regarding copyright ownership.
- # The ASF licenses this file to You under the Apache License, Version 2.0
- # (the "License"); you may not use this file except in compliance with
- # the License. You may obtain a copy of the License at
- #
- # http://www.apache.org/licenses/LICENSE-2.0
- #
- # Unless required by applicable law or agreed to in writing, software
- # distributed under the License is distributed on an "AS IS" BASIS,
- # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- # See the License for the specific language governing permissions and
- # limitations under the License.
-
- # NOTE: All param tuning can be done in the SetCassandraEnvironment Function below
-
- #-----------------------------------------------------------------------------
- Function SetCassandraHome()
- {
- if (! $env:CASSANDRA_HOME)
- {
- $cwd = [System.IO.Directory]::GetCurrentDirectory()
- $cwd = Split-Path $cwd -parent
- $env:CASSANDRA_HOME = $cwd -replace "\\", "/"
- }
- }
-
- #-----------------------------------------------------------------------------
- Function SetCassandraMain()
- {
- if (! $env:CASSANDRA_MAIN)
- {
- $env:CASSANDRA_MAIN="org.apache.cassandra.service.CassandraDaemon"
- }
- }
-
- #-----------------------------------------------------------------------------
- Function BuildClassPath
- {
- $cp = """$env:CASSANDRA_HOME\conf"""
- foreach ($file in Get-ChildItem "$env:CASSANDRA_HOME\lib\*.jar")
- {
- $file = $file -replace "\\", "/"
- $cp = $cp + ";" + """$file"""
- }
-
- # Add build/classes/main so it works in development
- $cp = $cp + ";" + """$env:CASSANDRA_HOME\build\classes\main"""
- $env:CLASSPATH=$cp
- }
-
- #-----------------------------------------------------------------------------
- Function CalculateHeapSizes
- {
- # Check if swapping is enabled on the host and warn if so - reference CASSANDRA-7316
-
- $osInfo = Get-WmiObject -class "Win32_computersystem"
- $autoPage = $osInfo.AutomaticManagedPageFile
-
- if ($autoPage)
- {
- echo "*---------------------------------------------------------------------*"
- echo "*---------------------------------------------------------------------*"
- echo ""
- echo " WARNING! Automatic page file configuration detected."
- echo " It is recommended that you disable swap when running Cassandra"
- echo " for performance and stability reasons."
- echo ""
- echo "*---------------------------------------------------------------------*"
- echo "*---------------------------------------------------------------------*"
- }
- else
- {
- $pageFileInfo = Get-WmiObject -class "Win32_PageFileSetting" -EnableAllPrivileges
- $pageFileCount = $PageFileInfo.Count
- if ($pageFileInfo)
- {
- $files = @()
- $sizes = @()
- $hasSizes = $FALSE
-
- # PageFileCount isn't populated and obj comes back as single if there's only 1
- if ([string]::IsNullOrEmpty($PageFileCount))
- {
- $PageFileCount = 1
- $files += $PageFileInfo.Name
- if ($PageFileInfo.MaximumSize -ne 0)
- {
- $hasSizes = $TRUE
- $sizes += $PageFileInfo.MaximumSize
- }
- }
- else
- {
- for ($i = 0; $i -le $PageFileCount; $i++)
- {
- $files += $PageFileInfo[$i].Name
- if ($PageFileInfo[$i].MaximumSize -ne 0)
- {
- $hasSizes = $TRUE
- $sizes += $PageFileInfo[$i].MaximumSize
- }
- }
- }
-
- echo "*---------------------------------------------------------------------*"
- echo "*---------------------------------------------------------------------*"
- echo ""
- echo " WARNING! $PageFileCount swap file(s) detected"
- for ($i = 0; $i -lt $PageFileCount; $i++)
- {
- $toPrint = " Name: " + $files[$i]
- if ($hasSizes)
- {
- $toPrint = $toPrint + " Size: " + $sizes[$i]
- $toPrint = $toPrint -replace [Environment]::NewLine, ""
- }
- echo $toPrint
- }
- echo " It is recommended that you disable swap when running Cassandra"
- echo " for performance and stability reasons."
- echo ""
- echo "*---------------------------------------------------------------------*"
- echo "*---------------------------------------------------------------------*"
- }
- }
-
- # Validate that we need to run this function and that our config is good
- if ($env:MAX_HEAP_SIZE -and $env:HEAP_NEWSIZE)
- {
- return
- }
-
- if ((($env:MAX_HEAP_SIZE -and !$env:HEAP_NEWSIZE) -or (!$env:MAX_HEAP_SIZE -and $env:HEAP_NEWSIZE)) -and ($using_cms -eq $true))
- {
- echo "Please set or unset MAX_HEAP_SIZE and HEAP_NEWSIZE in pairs. Aborting startup."
- exit 1
- }
-
- $memObject = Get-WMIObject -class win32_physicalmemory
- if ($memObject -eq $null)
- {
- echo "WARNING! Could not determine system memory. Defaulting to 2G heap, 512M newgen. Manually override in conf\jvm.options for different heap values."
- $env:MAX_HEAP_SIZE = "2048M"
- $env:HEAP_NEWSIZE = "512M"
- return
- }
-
- $memory = ($memObject | Measure-Object Capacity -Sum).sum
- $memoryMB = [Math]::Truncate($memory / (1024*1024))
-
- $cpu = gwmi Win32_ComputerSystem | Select-Object NumberOfLogicalProcessors
- $systemCores = $cpu.NumberOfLogicalProcessors
-
- # set max heap size based on the following
- # max(min(1/2 ram, 1024MB), min(1/4 ram, 8GB))
- # calculate 1/2 ram and cap to 1024MB
- # calculate 1/4 ram and cap to 8192MB
- # pick the max
- $halfMem = [Math]::Truncate($memoryMB / 2)
- $quarterMem = [Math]::Truncate($halfMem / 2)
-
- if ($halfMem -gt 1024)
- {
- $halfMem = 1024
- }
- if ($quarterMem -gt 8192)
- {
- $quarterMem = 8192
- }
-
- $maxHeapMB = ""
- if ($halfMem -gt $quarterMem)
- {
- $maxHeapMB = $halfMem
- }
- else
- {
- $maxHeapMB = $quarterMem
- }
- $env:MAX_HEAP_SIZE = [System.Convert]::ToString($maxHeapMB) + "M"
-
- # Young gen: min(max_sensible_per_modern_cpu_core * num_cores, 1/4
- $maxYGPerCore = 100
- $maxYGTotal = $maxYGPerCore * $systemCores
- $desiredYG = [Math]::Truncate($maxHeapMB / 4)
-
- if ($desiredYG -gt $maxYGTotal)
- {
- $env:HEAP_NEWSIZE = [System.Convert]::ToString($maxYGTotal) + "M"
- }
- else
- {
- $env:HEAP_NEWSIZE = [System.Convert]::ToString($desiredYG) + "M"
- }
- }
-
- #-----------------------------------------------------------------------------
- Function ParseJVMInfo
- {
- # grab info about the JVM
- $pinfo = New-Object System.Diagnostics.ProcessStartInfo
- $pinfo.FileName = "$env:JAVA_BIN"
- $pinfo.RedirectStandardError = $true
- $pinfo.RedirectStandardOutput = $true
- $pinfo.UseShellExecute = $false
- $pinfo.Arguments = "-d64 -version"
- $p = New-Object System.Diagnostics.Process
- $p.StartInfo = $pinfo
- $p.Start() | Out-Null
- $p.WaitForExit()
- $stderr = $p.StandardError.ReadToEnd()
-
- $env:JVM_ARCH = "64-bit"
-
- if ($stderr.Contains("Error"))
- {
- # 32-bit JVM. re-run w/out -d64
- echo "Failed 64-bit check. Re-running to get version from 32-bit"
- $pinfo.Arguments = "-version"
- $p = New-Object System.Diagnostics.Process
- $p.StartInfo = $pinfo
- $p.Start() | Out-Null
- $p.WaitForExit()
- $stderr = $p.StandardError.ReadToEnd()
- $env:JVM_ARCH = "32-bit"
- }
-
- $sa = $stderr.Split("""")
- $env:JVM_VERSION = $sa[1]
-
- if ($stderr.Contains("OpenJDK"))
- {
- $env:JVM_VENDOR = "OpenJDK"
- }
- elseif ($stderr.Contains("Java(TM)"))
- {
- $env:JVM_VENDOR = "Oracle"
- }
- else
- {
- $JVM_VENDOR = "other"
- }
-
- $pa = $sa[1].Split("_")
- $subVersion = $pa[1]
- # Deal with -b (build) versions
- if ($subVersion -contains '-')
- {
- $patchAndBuild = $subVersion.Split("-")
- $subVersion = $patchAndBuild[0]
- }
- $env:JVM_PATCH_VERSION = $subVersion
- }
-
- #-----------------------------------------------------------------------------
- Function SetCassandraEnvironment
- {
- if (Test-Path Env:\JAVA_HOME)
- {
- $env:JAVA_BIN = "$env:JAVA_HOME\bin\java.exe"
- }
- elseif (Get-Command "java.exe")
- {
- $env:JAVA_BIN = "java.exe"
- }
- else
- {
- echo "ERROR! No JAVA_HOME set and could not find java.exe in the path."
- exit
- }
- SetCassandraHome
- $env:CASSANDRA_CONF = "$env:CASSANDRA_HOME\conf"
- $env:CASSANDRA_PARAMS="-Dcassandra -Dlogback.configurationFile=logback.xml"
-
- $logdir = "$env:CASSANDRA_HOME\logs"
- $storagedir = "$env:CASSANDRA_HOME\data"
- $env:CASSANDRA_PARAMS = $env:CASSANDRA_PARAMS + " -Dcassandra.logdir=""$logdir"" -Dcassandra.storagedir=""$storagedir"""
-
- SetCassandraMain
- BuildClassPath
-
- # Override these to set the amount of memory to allocate to the JVM at
- # start-up. For production use you may wish to adjust this for your
- # environment. MAX_HEAP_SIZE is the total amount of memory dedicated
- # to the Java heap. HEAP_NEWSIZE refers to the size of the young
- # generation. Both MAX_HEAP_SIZE and HEAP_NEWSIZE should be either set
- # or not (if you set one, set the other).
- #
- # The main trade-off for the young generation is that the larger it
- # is, the longer GC pause times will be. The shorter it is, the more
- # expensive GC will be (usually).
- #
- # The example HEAP_NEWSIZE assumes a modern 8-core+ machine for decent
- # times. If in doubt, and if you do not particularly want to tweak, go
- # 100 MB per physical CPU core.
-
- #GC log path has to be defined here since it needs to find CASSANDRA_HOME
- $env:JVM_OPTS="$env:JVM_OPTS -Xloggc:""$env:CASSANDRA_HOME/logs/gc.log"""
-
- # Read user-defined JVM options from jvm.options file
- $content = Get-Content "$env:CASSANDRA_CONF\jvm.options"
- for ($i = 0; $i -lt $content.Count; $i++)
- {
- $line = $content[$i]
- if ($line.StartsWith("-"))
- {
- $env:JVM_OPTS = "$env:JVM_OPTS $line"
- }
- }
-
- $defined_xmn = $env:JVM_OPTS -like '*Xmn*'
- $defined_xmx = $env:JVM_OPTS -like '*Xmx*'
- $defined_xms = $env:JVM_OPTS -like '*Xms*'
- $using_cms = $env:JVM_OPTS -like '*UseConcMarkSweepGC*'
-
- #$env:MAX_HEAP_SIZE="4096M"
- #$env:HEAP_NEWSIZE="800M"
- CalculateHeapSizes
-
- ParseJVMInfo
-
- # We only set -Xms and -Xmx if they were not defined on jvm.options file
- # If defined, both Xmx and Xms should be defined together.
- if (($defined_xmx -eq $false) -and ($defined_xms -eq $false))
- {
- $env:JVM_OPTS="$env:JVM_OPTS -Xms$env:MAX_HEAP_SIZE"
- $env:JVM_OPTS="$env:JVM_OPTS -Xmx$env:MAX_HEAP_SIZE"
- }
- elseif (($defined_xmx -eq $false) -or ($defined_xms -eq $false))
- {
- echo "Please set or unset -Xmx and -Xms flags in pairs on jvm.options file."
- exit
- }
-
- # We only set -Xmn flag if it was not defined in jvm.options file
- # and if the CMS GC is being used
- # If defined, both Xmn and Xmx should be defined together.
- if (($defined_xmn -eq $true) -and ($defined_xmx -eq $false))
- {
- echo "Please set or unset -Xmx and -Xmn flags in pairs on jvm.options file."
- exit
- }
- elseif (($defined_xmn -eq $false) -and ($using_cms -eq $true))
- {
- $env:JVM_OPTS="$env:JVM_OPTS -Xmn$env:HEAP_NEWSIZE"
- }
-
- if (($env:JVM_ARCH -eq "64-Bit") -and ($using_cms -eq $true))
- {
- $env:JVM_OPTS="$env:JVM_OPTS -XX:+UseCondCardMark"
- }
-
- # Add sigar env - see Cassandra-7838
- $env:JVM_OPTS = "$env:JVM_OPTS -Djava.library.path=""$env:CASSANDRA_HOME\lib\sigar-bin"""
-
- # Confirm we're on high performance power plan, warn if not
- # Change to $true to suppress this warning
- $suppressPowerWarning = $false
- if (!$suppressPowerWarning)
- {
- $currentProfile = powercfg /GETACTIVESCHEME
- if (!$currentProfile.Contains("High performance"))
- {
- echo "*---------------------------------------------------------------------*"
- echo "*---------------------------------------------------------------------*"
- echo ""
- echo " WARNING! Detected a power profile other than High Performance."
- echo " Performance of this node will suffer."
- echo " Modify conf\cassandra.env.ps1 to suppress this warning."
- echo ""
- echo "*---------------------------------------------------------------------*"
- echo "*---------------------------------------------------------------------*"
- }
- }
-
+ #
+ # Licensed to the Apache Software Foundation (ASF) under one or more
+ # contributor license agreements. See the NOTICE file distributed with
+ # this work for additional information regarding copyright ownership.
+ # The ASF licenses this file to You under the Apache License, Version 2.0
+ # (the "License"); you may not use this file except in compliance with
+ # the License. You may obtain a copy of the License at
+ #
+ # http://www.apache.org/licenses/LICENSE-2.0
+ #
+ # Unless required by applicable law or agreed to in writing, software
+ # distributed under the License is distributed on an "AS IS" BASIS,
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ # See the License for the specific language governing permissions and
+ # limitations under the License.
+
+ # NOTE: All param tuning can be done in the SetCassandraEnvironment Function below
+
+ #-----------------------------------------------------------------------------
+ Function SetCassandraHome()
+ {
+ if (! $env:CASSANDRA_HOME)
+ {
+ $cwd = [System.IO.Directory]::GetCurrentDirectory()
+ $cwd = Split-Path $cwd -parent
+ $env:CASSANDRA_HOME = $cwd -replace "\\", "/"
+ }
+ }
+
+ #-----------------------------------------------------------------------------
+ Function SetCassandraMain()
+ {
+ if (! $env:CASSANDRA_MAIN)
+ {
+ $env:CASSANDRA_MAIN="org.apache.cassandra.service.CassandraDaemon"
+ }
+ }
+
+ #-----------------------------------------------------------------------------
+ Function BuildClassPath
+ {
+ $cp = """$env:CASSANDRA_HOME\conf"""
+ foreach ($file in Get-ChildItem "$env:CASSANDRA_HOME\lib\*.jar")
+ {
+ $file = $file -replace "\\", "/"
+ $cp = $cp + ";" + """$file"""
+ }
+
+ # Add build/classes/main so it works in development
- $cp = $cp + ";" + """$env:CASSANDRA_HOME\build\classes\main"";""$env:CASSANDRA_HOME\build\classes\thrift"""
++ $cp = $cp + ";" + """$env:CASSANDRA_HOME\build\classes\main"""
+ $env:CLASSPATH=$cp
+ }
+
+ #-----------------------------------------------------------------------------
+ Function CalculateHeapSizes
+ {
+ # Check if swapping is enabled on the host and warn if so - reference CASSANDRA-7316
+
+ $osInfo = Get-WmiObject -class "Win32_computersystem"
+ $autoPage = $osInfo.AutomaticManagedPageFile
+
+ if ($autoPage)
+ {
+ echo "*---------------------------------------------------------------------*"
+ echo "*---------------------------------------------------------------------*"
+ echo ""
+ echo " WARNING! Automatic page file configuration detected."
+ echo " It is recommended that you disable swap when running Cassandra"
+ echo " for performance and stability reasons."
+ echo ""
+ echo "*---------------------------------------------------------------------*"
+ echo "*---------------------------------------------------------------------*"
+ }
+ else
+ {
+ $pageFileInfo = Get-WmiObject -class "Win32_PageFileSetting" -EnableAllPrivileges
+ $pageFileCount = $PageFileInfo.Count
+ if ($pageFileInfo)
+ {
+ $files = @()
+ $sizes = @()
+ $hasSizes = $FALSE
+
+ # PageFileCount isn't populated and obj comes back as single if there's only 1
+ if ([string]::IsNullOrEmpty($PageFileCount))
+ {
+ $PageFileCount = 1
+ $files += $PageFileInfo.Name
+ if ($PageFileInfo.MaximumSize -ne 0)
+ {
+ $hasSizes = $TRUE
+ $sizes += $PageFileInfo.MaximumSize
+ }
+ }
+ else
+ {
+ for ($i = 0; $i -le $PageFileCount; $i++)
+ {
+ $files += $PageFileInfo[$i].Name
+ if ($PageFileInfo[$i].MaximumSize -ne 0)
+ {
+ $hasSizes = $TRUE
+ $sizes += $PageFileInfo[$i].MaximumSize
+ }
+ }
+ }
+
+ echo "*---------------------------------------------------------------------*"
+ echo "*---------------------------------------------------------------------*"
+ echo ""
+ echo " WARNING! $PageFileCount swap file(s) detected"
+ for ($i = 0; $i -lt $PageFileCount; $i++)
+ {
+ $toPrint = " Name: " + $files[$i]
+ if ($hasSizes)
+ {
+ $toPrint = $toPrint + " Size: " + $sizes[$i]
+ $toPrint = $toPrint -replace [Environment]::NewLine, ""
+ }
+ echo $toPrint
+ }
+ echo " It is recommended that you disable swap when running Cassandra"
+ echo " for performance and stability reasons."
+ echo ""
+ echo "*---------------------------------------------------------------------*"
+ echo "*---------------------------------------------------------------------*"
+ }
+ }
+
+ # Validate that we need to run this function and that our config is good
+ if ($env:MAX_HEAP_SIZE -and $env:HEAP_NEWSIZE)
+ {
+ return
+ }
+
+ if ((($env:MAX_HEAP_SIZE -and !$env:HEAP_NEWSIZE) -or (!$env:MAX_HEAP_SIZE -and $env:HEAP_NEWSIZE)) -and ($using_cms -eq $true))
+ {
+ echo "Please set or unset MAX_HEAP_SIZE and HEAP_NEWSIZE in pairs. Aborting startup."
+ exit 1
+ }
+
+ $memObject = Get-WMIObject -class win32_physicalmemory
+ if ($memObject -eq $null)
+ {
+ echo "WARNING! Could not determine system memory. Defaulting to 2G heap, 512M newgen. Manually override in conf\jvm.options for different heap values."
+ $env:MAX_HEAP_SIZE = "2048M"
+ $env:HEAP_NEWSIZE = "512M"
+ return
+ }
+
+ $memory = ($memObject | Measure-Object Capacity -Sum).sum
+ $memoryMB = [Math]::Truncate($memory / (1024*1024))
+
+ $cpu = gwmi Win32_ComputerSystem | Select-Object NumberOfLogicalProcessors
+ $systemCores = $cpu.NumberOfLogicalProcessors
+
+ # set max heap size based on the following
+ # max(min(1/2 ram, 1024MB), min(1/4 ram, 8GB))
+ # calculate 1/2 ram and cap to 1024MB
+ # calculate 1/4 ram and cap to 8192MB
+ # pick the max
+ $halfMem = [Math]::Truncate($memoryMB / 2)
+ $quarterMem = [Math]::Truncate($halfMem / 2)
+
+ if ($halfMem -gt 1024)
+ {
+ $halfMem = 1024
+ }
+ if ($quarterMem -gt 8192)
+ {
+ $quarterMem = 8192
+ }
+
+ $maxHeapMB = ""
+ if ($halfMem -gt $quarterMem)
+ {
+ $maxHeapMB = $halfMem
+ }
+ else
+ {
+ $maxHeapMB = $quarterMem
+ }
+ $env:MAX_HEAP_SIZE = [System.Convert]::ToString($maxHeapMB) + "M"
+
+ # Young gen: min(max_sensible_per_modern_cpu_core * num_cores, 1/4
+ $maxYGPerCore = 100
+ $maxYGTotal = $maxYGPerCore * $systemCores
+ $desiredYG = [Math]::Truncate($maxHeapMB / 4)
+
+ if ($desiredYG -gt $maxYGTotal)
+ {
+ $env:HEAP_NEWSIZE = [System.Convert]::ToString($maxYGTotal) + "M"
+ }
+ else
+ {
+ $env:HEAP_NEWSIZE = [System.Convert]::ToString($desiredYG) + "M"
+ }
+ }
+
+ #-----------------------------------------------------------------------------
+ Function ParseJVMInfo
+ {
+ # grab info about the JVM
+ $pinfo = New-Object System.Diagnostics.ProcessStartInfo
+ $pinfo.FileName = "$env:JAVA_BIN"
+ $pinfo.RedirectStandardError = $true
+ $pinfo.RedirectStandardOutput = $true
+ $pinfo.UseShellExecute = $false
+ $pinfo.Arguments = "-d64 -version"
+ $p = New-Object System.Diagnostics.Process
+ $p.StartInfo = $pinfo
+ $p.Start() | Out-Null
+ $p.WaitForExit()
+ $stderr = $p.StandardError.ReadToEnd()
+
+ $env:JVM_ARCH = "64-bit"
+
+ if ($stderr.Contains("Error"))
+ {
+ # 32-bit JVM. re-run w/out -d64
+ echo "Failed 64-bit check. Re-running to get version from 32-bit"
+ $pinfo.Arguments = "-version"
+ $p = New-Object System.Diagnostics.Process
+ $p.StartInfo = $pinfo
+ $p.Start() | Out-Null
+ $p.WaitForExit()
+ $stderr = $p.StandardError.ReadToEnd()
+ $env:JVM_ARCH = "32-bit"
+ }
+
+ $sa = $stderr.Split("""")
+ $env:JVM_VERSION = $sa[1]
+
+ if ($stderr.Contains("OpenJDK"))
+ {
+ $env:JVM_VENDOR = "OpenJDK"
+ }
+ elseif ($stderr.Contains("Java(TM)"))
+ {
+ $env:JVM_VENDOR = "Oracle"
+ }
+ else
+ {
+ $JVM_VENDOR = "other"
+ }
+
+ $pa = $sa[1].Split("_")
+ $subVersion = $pa[1]
+ # Deal with -b (build) versions
+ if ($subVersion -contains '-')
+ {
+ $patchAndBuild = $subVersion.Split("-")
+ $subVersion = $patchAndBuild[0]
+ }
+ $env:JVM_PATCH_VERSION = $subVersion
+ }
+
+ #-----------------------------------------------------------------------------
+ Function SetCassandraEnvironment
+ {
+ if (Test-Path Env:\JAVA_HOME)
+ {
+ $env:JAVA_BIN = "$env:JAVA_HOME\bin\java.exe"
+ }
+ elseif (Get-Command "java.exe")
+ {
+ $env:JAVA_BIN = "java.exe"
+ }
+ else
+ {
+ echo "ERROR! No JAVA_HOME set and could not find java.exe in the path."
+ exit
+ }
+ SetCassandraHome
+ $env:CASSANDRA_CONF = "$env:CASSANDRA_HOME\conf"
+ $env:CASSANDRA_PARAMS="-Dcassandra -Dlogback.configurationFile=logback.xml"
+
+ $logdir = "$env:CASSANDRA_HOME\logs"
+ $storagedir = "$env:CASSANDRA_HOME\data"
+ $env:CASSANDRA_PARAMS = $env:CASSANDRA_PARAMS + " -Dcassandra.logdir=""$logdir"" -Dcassandra.storagedir=""$storagedir"""
+
+ SetCassandraMain
+ BuildClassPath
+
+ # Override these to set the amount of memory to allocate to the JVM at
+ # start-up. For production use you may wish to adjust this for your
+ # environment. MAX_HEAP_SIZE is the total amount of memory dedicated
+ # to the Java heap. HEAP_NEWSIZE refers to the size of the young
+ # generation. Both MAX_HEAP_SIZE and HEAP_NEWSIZE should be either set
+ # or not (if you set one, set the other).
+ #
+ # The main trade-off for the young generation is that the larger it
+ # is, the longer GC pause times will be. The shorter it is, the more
+ # expensive GC will be (usually).
+ #
+ # The example HEAP_NEWSIZE assumes a modern 8-core+ machine for decent
+ # times. If in doubt, and if you do not particularly want to tweak, go
+ # 100 MB per physical CPU core.
+
+ #GC log path has to be defined here since it needs to find CASSANDRA_HOME
+ $env:JVM_OPTS="$env:JVM_OPTS -Xloggc:""$env:CASSANDRA_HOME/logs/gc.log"""
+
+ # Read user-defined JVM options from jvm.options file
+ $content = Get-Content "$env:CASSANDRA_CONF\jvm.options"
+ for ($i = 0; $i -lt $content.Count; $i++)
+ {
+ $line = $content[$i]
+ if ($line.StartsWith("-"))
+ {
+ $env:JVM_OPTS = "$env:JVM_OPTS $line"
+ }
+ }
+
+ $defined_xmn = $env:JVM_OPTS -like '*Xmn*'
+ $defined_xmx = $env:JVM_OPTS -like '*Xmx*'
+ $defined_xms = $env:JVM_OPTS -like '*Xms*'
+ $using_cms = $env:JVM_OPTS -like '*UseConcMarkSweepGC*'
+
+ #$env:MAX_HEAP_SIZE="4096M"
+ #$env:HEAP_NEWSIZE="800M"
+ CalculateHeapSizes
+
+ ParseJVMInfo
+
+ # We only set -Xms and -Xmx if they were not defined on jvm.options file
+ # If defined, both Xmx and Xms should be defined together.
+ if (($defined_xmx -eq $false) -and ($defined_xms -eq $false))
+ {
+ $env:JVM_OPTS="$env:JVM_OPTS -Xms$env:MAX_HEAP_SIZE"
+ $env:JVM_OPTS="$env:JVM_OPTS -Xmx$env:MAX_HEAP_SIZE"
+ }
+ elseif (($defined_xmx -eq $false) -or ($defined_xms -eq $false))
+ {
+ echo "Please set or unset -Xmx and -Xms flags in pairs on jvm.options file."
+ exit
+ }
+
+ # We only set -Xmn flag if it was not defined in jvm.options file
+ # and if the CMS GC is being used
+ # If defined, both Xmn and Xmx should be defined together.
+ if (($defined_xmn -eq $true) -and ($defined_xmx -eq $false))
+ {
+ echo "Please set or unset -Xmx and -Xmn flags in pairs on jvm.options file."
+ exit
+ }
+ elseif (($defined_xmn -eq $false) -and ($using_cms -eq $true))
+ {
+ $env:JVM_OPTS="$env:JVM_OPTS -Xmn$env:HEAP_NEWSIZE"
+ }
+
+ if (($env:JVM_ARCH -eq "64-Bit") -and ($using_cms -eq $true))
+ {
+ $env:JVM_OPTS="$env:JVM_OPTS -XX:+UseCondCardMark"
+ }
+
+ # Add sigar env - see Cassandra-7838
+ $env:JVM_OPTS = "$env:JVM_OPTS -Djava.library.path=""$env:CASSANDRA_HOME\lib\sigar-bin"""
+
+ # Confirm we're on high performance power plan, warn if not
+ # Change to $true to suppress this warning
+ $suppressPowerWarning = $false
+ if (!$suppressPowerWarning)
+ {
+ $currentProfile = powercfg /GETACTIVESCHEME
+ if (!$currentProfile.Contains("High performance"))
+ {
+ echo "*---------------------------------------------------------------------*"
+ echo "*---------------------------------------------------------------------*"
+ echo ""
+ echo " WARNING! Detected a power profile other than High Performance."
+ echo " Performance of this node will suffer."
+ echo " Modify conf\cassandra.env.ps1 to suppress this warning."
+ echo ""
+ echo "*---------------------------------------------------------------------*"
+ echo "*---------------------------------------------------------------------*"
+ }
+ }
+
# provides hints to the JIT compiler
$env:JVM_OPTS = "$env:JVM_OPTS -XX:CompileCommandFile=$env:CASSANDRA_CONF\hotspot_compiler"
http://git-wip-us.apache.org/repos/asf/cassandra/blob/17e602b7/conf/cassandra-env.sh
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/17e602b7/src/java/org/apache/cassandra/service/StartupChecks.java
----------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org