You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by jb...@apache.org on 2013/12/03 04:39:41 UTC
[1/3] git commit: Reduce gossip memory use by interning
VersionedValue strings patch by jbellis;
tested by Quentin Conner for CASSANDRA-6410
Updated Branches:
refs/heads/cassandra-2.0 ef768a555 -> 672496430
refs/heads/trunk ab9fe1ae8 -> 57516e082
Reduce gossip memory use by interning VersionedValue strings
patch by jbellis; tested by Quentin Conner for CASSANDRA-6410
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/67249643
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/67249643
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/67249643
Branch: refs/heads/cassandra-2.0
Commit: 6724964308f70df72cc2ebd1827b8d318e5df7f0
Parents: ef768a5
Author: Jonathan Ellis <jb...@apache.org>
Authored: Mon Dec 2 20:56:14 2013 -0600
Committer: Jonathan Ellis <jb...@apache.org>
Committed: Mon Dec 2 21:36:22 2013 -0600
----------------------------------------------------------------------
CHANGES.txt | 1 +
conf/cassandra-env.sh | 3 +++
src/java/org/apache/cassandra/gms/VersionedValue.java | 6 +++++-
3 files changed, 9 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/67249643/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index 8a95fa8..11f4c09 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.4
+ * Reduce gossip memory use by interning VersionedValue strings (CASSANDRA-6410)
* Allow specifying datacenters to participate in a repair (CASSANDRA-6218)
* Fix divide-by-zero in PCI (CASSANDRA-6403)
* Fix setting last compacted key in the wrong level for LCS (CASSANDRA-6284)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/67249643/conf/cassandra-env.sh
----------------------------------------------------------------------
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index e229297..95acc4e 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -195,6 +195,9 @@ startswith() { [ "${1#$2}" != "$1" ]; }
# Per-thread stack size.
JVM_OPTS="$JVM_OPTS -Xss256k"
+# Larger interned string table, for gossip's benefit (CASSANDRA-6410)
+JVM_OPTS="$JVM_OPTS -XX:StringTableSize=1000003"
+
# GC tuning options
JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC"
JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC"
http://git-wip-us.apache.org/repos/asf/cassandra/blob/67249643/src/java/org/apache/cassandra/gms/VersionedValue.java
----------------------------------------------------------------------
diff --git a/src/java/org/apache/cassandra/gms/VersionedValue.java b/src/java/org/apache/cassandra/gms/VersionedValue.java
index ec0f39f..a7ee047 100644
--- a/src/java/org/apache/cassandra/gms/VersionedValue.java
+++ b/src/java/org/apache/cassandra/gms/VersionedValue.java
@@ -80,7 +80,11 @@ public class VersionedValue implements Comparable<VersionedValue>
private VersionedValue(String value, int version)
{
assert value != null;
- this.value = value;
+ // blindly interning everything is somewhat suboptimal -- lots of VersionedValues are unique --
+ // but harmless, and interning the non-unique ones saves significant memory. (Unfortunately,
+ // we don't really have enough information here in VersionedValue to tell the probably-unique
+ // values apart.) See CASSANDRA-6410.
+ this.value = value.intern();
this.version = version;
}
[2/3] git commit: Reduce gossip memory use by interning
VersionedValue strings patch by jbellis;
tested by Quentin Conner for CASSANDRA-6410
Posted by jb...@apache.org.
Reduce gossip memory use by interning VersionedValue strings
patch by jbellis; tested by Quentin Conner for CASSANDRA-6410
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/67249643
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/67249643
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/67249643
Branch: refs/heads/trunk
Commit: 6724964308f70df72cc2ebd1827b8d318e5df7f0
Parents: ef768a5
Author: Jonathan Ellis <jb...@apache.org>
Authored: Mon Dec 2 20:56:14 2013 -0600
Committer: Jonathan Ellis <jb...@apache.org>
Committed: Mon Dec 2 21:36:22 2013 -0600
----------------------------------------------------------------------
CHANGES.txt | 1 +
conf/cassandra-env.sh | 3 +++
src/java/org/apache/cassandra/gms/VersionedValue.java | 6 +++++-
3 files changed, 9 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/67249643/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index 8a95fa8..11f4c09 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.4
+ * Reduce gossip memory use by interning VersionedValue strings (CASSANDRA-6410)
* Allow specifying datacenters to participate in a repair (CASSANDRA-6218)
* Fix divide-by-zero in PCI (CASSANDRA-6403)
* Fix setting last compacted key in the wrong level for LCS (CASSANDRA-6284)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/67249643/conf/cassandra-env.sh
----------------------------------------------------------------------
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index e229297..95acc4e 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -195,6 +195,9 @@ startswith() { [ "${1#$2}" != "$1" ]; }
# Per-thread stack size.
JVM_OPTS="$JVM_OPTS -Xss256k"
+# Larger interned string table, for gossip's benefit (CASSANDRA-6410)
+JVM_OPTS="$JVM_OPTS -XX:StringTableSize=1000003"
+
# GC tuning options
JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC"
JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC"
http://git-wip-us.apache.org/repos/asf/cassandra/blob/67249643/src/java/org/apache/cassandra/gms/VersionedValue.java
----------------------------------------------------------------------
diff --git a/src/java/org/apache/cassandra/gms/VersionedValue.java b/src/java/org/apache/cassandra/gms/VersionedValue.java
index ec0f39f..a7ee047 100644
--- a/src/java/org/apache/cassandra/gms/VersionedValue.java
+++ b/src/java/org/apache/cassandra/gms/VersionedValue.java
@@ -80,7 +80,11 @@ public class VersionedValue implements Comparable<VersionedValue>
private VersionedValue(String value, int version)
{
assert value != null;
- this.value = value;
+ // blindly interning everything is somewhat suboptimal -- lots of VersionedValues are unique --
+ // but harmless, and interning the non-unique ones saves significant memory. (Unfortunately,
+ // we don't really have enough information here in VersionedValue to tell the probably-unique
+ // values apart.) See CASSANDRA-6410.
+ this.value = value.intern();
this.version = version;
}
[3/3] git commit: Merge branch 'cassandra-2.0' into trunk
Posted by jb...@apache.org.
Merge branch 'cassandra-2.0' into trunk
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/57516e08
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/57516e08
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/57516e08
Branch: refs/heads/trunk
Commit: 57516e082b71a761513767107e76e5eb19c49253
Parents: ab9fe1a 6724964
Author: Jonathan Ellis <jb...@apache.org>
Authored: Mon Dec 2 21:39:20 2013 -0600
Committer: Jonathan Ellis <jb...@apache.org>
Committed: Mon Dec 2 21:39:20 2013 -0600
----------------------------------------------------------------------
CHANGES.txt | 1 +
conf/cassandra-env.sh | 3 +++
src/java/org/apache/cassandra/gms/VersionedValue.java | 6 +++++-
3 files changed, 9 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/57516e08/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index bb3a98a,11f4c09..3bc50ac
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,22 -1,5 +1,23 @@@
+2.1
+ * Multithreaded commitlog (CASSANDRA-3578)
+ * allocate fixed index summary memory pool and resample cold index summaries
+ to use less memory (CASSANDRA-5519)
+ * Removed multithreaded compaction (CASSANDRA-6142)
+ * Parallelize fetching rows for low-cardinality indexes (CASSANDRA-1337)
+ * change logging from log4j to logback (CASSANDRA-5883)
+ * switch to LZ4 compression for internode communication (CASSANDRA-5887)
+ * Stop using Thrift-generated Index* classes internally (CASSANDRA-5971)
+ * Remove 1.2 network compatibility code (CASSANDRA-5960)
+ * Remove leveled json manifest migration code (CASSANDRA-5996)
+ * Remove CFDefinition (CASSANDRA-6253)
+ * Use AtomicIntegerFieldUpdater in RefCountedMemory (CASSANDRA-6278)
+ * User-defined types for CQL3 (CASSANDRA-5590)
+ * Use of o.a.c.metrics in nodetool (CASSANDRA-5871, 6406)
+ * Batch read from OTC's queue and cleanup (CASSANDRA-1632)
+
+
2.0.4
+ * Reduce gossip memory use by interning VersionedValue strings (CASSANDRA-6410)
* Allow specifying datacenters to participate in a repair (CASSANDRA-6218)
* Fix divide-by-zero in PCI (CASSANDRA-6403)
* Fix setting last compacted key in the wrong level for LCS (CASSANDRA-6284)