You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (Created) (JIRA)" <ji...@apache.org> on 2011/10/20 15:06:10 UTC
[jira] [Created] (CASSANDRA-3389) Evaluate CSLM alternatives for
improved cache or GC performance
Evaluate CSLM alternatives for improved cache or GC performance
---------------------------------------------------------------
Key: CASSANDRA-3389
URL: https://issues.apache.org/jira/browse/CASSANDRA-3389
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Jonathan Ellis
Assignee: Brandon Williams
Priority: Minor
Fix For: 1.1
Ben Manes commented on http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance that it's worth evaluating https://github.com/mspiegel/lockfreeskiptree and https://github.com/nbronson/snaptree as CSLM replacements.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3389) Evaluate CSLM alternatives for
improved cache or GC performance
Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183589#comment-13183589 ]
Brandon Williams commented on CASSANDRA-3389:
---------------------------------------------
No significant difference with G1.
> Evaluate CSLM alternatives for improved cache or GC performance
> ---------------------------------------------------------------
>
> Key: CASSANDRA-3389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3389
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Priority: Minor
> Fix For: 1.1
>
> Attachments: 0001-Replace-CSLM-with-ConcurrentSkipTreeMap.patch, 0001-Switch-CSLM-to-SnapTree.patch
>
>
> Ben Manes commented on http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance that it's worth evaluating https://github.com/mspiegel/lockfreeskiptree and https://github.com/nbronson/snaptree as CSLM replacements.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3389) Evaluate CSLM alternatives for
improved cache or GC performance
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140406#comment-13140406 ]
Jonathan Ellis commented on CASSANDRA-3389:
-------------------------------------------
We also want to allow multiple clients to append to a wide row, so the write lock design probably doesn't work for us either.
> Evaluate CSLM alternatives for improved cache or GC performance
> ---------------------------------------------------------------
>
> Key: CASSANDRA-3389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3389
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Priority: Minor
> Fix For: 1.1
>
> Attachments: 0001-Replace-CSLM-with-ConcurrentSkipTreeMap.patch, 0001-Switch-CSLM-to-SnapTree.patch
>
>
> Ben Manes commented on http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance that it's worth evaluating https://github.com/mspiegel/lockfreeskiptree and https://github.com/nbronson/snaptree as CSLM replacements.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3389) Evaluate CSLM alternatives for
improved cache or GC performance
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170462#comment-13170462 ]
Jonathan Ellis commented on CASSANDRA-3389:
-------------------------------------------
Can you test these under G1 garbage collector? CSLM is the main reason G1 works poorly for us. (Especially the uses in Memtable.)
> Evaluate CSLM alternatives for improved cache or GC performance
> ---------------------------------------------------------------
>
> Key: CASSANDRA-3389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3389
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Priority: Minor
> Fix For: 1.1
>
> Attachments: 0001-Replace-CSLM-with-ConcurrentSkipTreeMap.patch, 0001-Switch-CSLM-to-SnapTree.patch
>
>
> Ben Manes commented on http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance that it's worth evaluating https://github.com/mspiegel/lockfreeskiptree and https://github.com/nbronson/snaptree as CSLM replacements.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3389) Evaluate CSLM alternatives for
improved cache or GC performance
Posted by "Jason Rutherglen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133420#comment-13133420 ]
Jason Rutherglen commented on CASSANDRA-3389:
---------------------------------------------
For GC improvements, a strict array based approach could work well. I was going to test this in LUCENE-2312 for the terms dict. The idea is to merge sort arrays into larger structures. It's admittedly crude however in some tests it's very fast. Arrays can be easily pooled, further reducing garbage collection.
> Evaluate CSLM alternatives for improved cache or GC performance
> ---------------------------------------------------------------
>
> Key: CASSANDRA-3389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3389
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Priority: Minor
> Fix For: 1.1
>
>
> Ben Manes commented on http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance that it's worth evaluating https://github.com/mspiegel/lockfreeskiptree and https://github.com/nbronson/snaptree as CSLM replacements.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3389) Evaluate CSLM alternatives for
improved cache or GC performance
Posted by "Ben Manes (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183608#comment-13183608 ]
Ben Manes commented on CASSANDRA-3389:
--------------------------------------
It might be worth using isolated benchmarks:
- Google Caliper for single-threaded (GetPutBenchmark)
- Cliff Click's for multi-threaded (PerfHashBenchmark)
You can hack from the examples below (see README):
http://code.google.com/p/concurrentlinkedhashmap/source/browse/trunk#trunk%2Fsrc%2Ftest%2Fjava%2Fcom%2Fgooglecode%2Fconcurrentlinkedhashmap%2Fbenchmark
> Evaluate CSLM alternatives for improved cache or GC performance
> ---------------------------------------------------------------
>
> Key: CASSANDRA-3389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3389
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Priority: Minor
> Fix For: 1.1
>
> Attachments: 0001-Replace-CSLM-with-ConcurrentSkipTreeMap.patch, 0001-Switch-CSLM-to-SnapTree.patch
>
>
> Ben Manes commented on http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance that it's worth evaluating https://github.com/mspiegel/lockfreeskiptree and https://github.com/nbronson/snaptree as CSLM replacements.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3389) Evaluate CSLM alternatives for
improved cache or GC performance
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183706#comment-13183706 ]
Jonathan Ellis commented on CASSANDRA-3389:
-------------------------------------------
I think we've invested as much time as makes sense for us at this point. (Whether CSLM isn't as much of a bottleneck as we thought, or CSTM/ST don't help as much w/ G1 as we thought, is kind of moot.)
> Evaluate CSLM alternatives for improved cache or GC performance
> ---------------------------------------------------------------
>
> Key: CASSANDRA-3389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3389
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Priority: Minor
> Attachments: 0001-Replace-CSLM-with-ConcurrentSkipTreeMap.patch, 0001-Switch-CSLM-to-SnapTree.patch
>
>
> Ben Manes commented on http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance that it's worth evaluating https://github.com/mspiegel/lockfreeskiptree and https://github.com/nbronson/snaptree as CSLM replacements.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3389) Evaluate CSLM alternatives for
improved cache or GC performance
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3389:
----------------------------------------
Attachment: 0001-Switch-CSLM-to-SnapTree.patch
0001-Replace-CSLM-with-ConcurrentSkipTreeMap.patch
'naive' patches (in that all I did was swap out CSLM) for lockfreeskiptree and snaptree. I didn't see any low hanging fruit improvement.
> Evaluate CSLM alternatives for improved cache or GC performance
> ---------------------------------------------------------------
>
> Key: CASSANDRA-3389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3389
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Priority: Minor
> Fix For: 1.1
>
> Attachments: 0001-Replace-CSLM-with-ConcurrentSkipTreeMap.patch, 0001-Switch-CSLM-to-SnapTree.patch
>
>
> Ben Manes commented on http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance that it's worth evaluating https://github.com/mspiegel/lockfreeskiptree and https://github.com/nbronson/snaptree as CSLM replacements.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3389) Evaluate CSLM alternatives for
improved cache or GC performance
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3389:
--------------------------------------
Comment: was deleted
(was: Too bad. Thanks for checking it out.)
> Evaluate CSLM alternatives for improved cache or GC performance
> ---------------------------------------------------------------
>
> Key: CASSANDRA-3389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3389
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Priority: Minor
> Attachments: 0001-Replace-CSLM-with-ConcurrentSkipTreeMap.patch, 0001-Switch-CSLM-to-SnapTree.patch
>
>
> Ben Manes commented on http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance that it's worth evaluating https://github.com/mspiegel/lockfreeskiptree and https://github.com/nbronson/snaptree as CSLM replacements.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3389) Evaluate CSLM alternatives for
improved cache or GC performance
Posted by "Jason Rutherglen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140339#comment-13140339 ]
Jason Rutherglen commented on CASSANDRA-3389:
---------------------------------------------
It is probably worth looking at Accumolo's implementation of a NativeMap [1] that implements sorted key value pairs in C++ to avoid 'stop the world' GC problems that CSLM can cause. NM uses a read write lock, bulk reading KV pairs in the read lock to avoid contention. I think that part may not work for Cassandra which is more often iterative with it's reads than a purely MapReduce motivated BigTable design, however it could be improved on.
1. https://github.com/apache/accumulo/blob/trunk/src/server/src/main/java/org/apache/accumulo/server/tabletserver/NativeMap.java
> Evaluate CSLM alternatives for improved cache or GC performance
> ---------------------------------------------------------------
>
> Key: CASSANDRA-3389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3389
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Priority: Minor
> Fix For: 1.1
>
> Attachments: 0001-Replace-CSLM-with-ConcurrentSkipTreeMap.patch, 0001-Switch-CSLM-to-SnapTree.patch
>
>
> Ben Manes commented on http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance that it's worth evaluating https://github.com/mspiegel/lockfreeskiptree and https://github.com/nbronson/snaptree as CSLM replacements.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3389) Evaluate CSLM alternatives for
improved cache or GC performance
Posted by "Jonathan Ellis (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-3389.
---------------------------------------
Resolution: Fixed
Fix Version/s: (was: 1.1)
Too bad. Thanks for checking it out.
> Evaluate CSLM alternatives for improved cache or GC performance
> ---------------------------------------------------------------
>
> Key: CASSANDRA-3389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3389
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Priority: Minor
> Attachments: 0001-Replace-CSLM-with-ConcurrentSkipTreeMap.patch, 0001-Switch-CSLM-to-SnapTree.patch
>
>
> Ben Manes commented on http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance that it's worth evaluating https://github.com/mspiegel/lockfreeskiptree and https://github.com/nbronson/snaptree as CSLM replacements.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3389) Evaluate CSLM alternatives for
improved cache or GC performance
Posted by "Jason Rutherglen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133423#comment-13133423 ]
Jason Rutherglen commented on CASSANDRA-3389:
---------------------------------------------
Additionally, the idea would make use of Lucene's BytesRefHash [1,2] structure for primary key lookups (which is faster than CSLM's O(log N).
1. http://lucene.apache.org/java/3_4_0/api/all/org/apache/lucene/util/BytesRefHash.html
2. http://svn.apache.org/repos/asf/lucene/dev/branches/branch_3x/lucene/src/java/org/apache/lucene/util/BytesRefHash.java
> Evaluate CSLM alternatives for improved cache or GC performance
> ---------------------------------------------------------------
>
> Key: CASSANDRA-3389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3389
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Priority: Minor
> Fix For: 1.1
>
>
> Ben Manes commented on http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance that it's worth evaluating https://github.com/mspiegel/lockfreeskiptree and https://github.com/nbronson/snaptree as CSLM replacements.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira