You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2009/09/05 00:08:57 UTC
[jira] Created: (CASSANDRA-423) add new LRU cache
add new LRU cache
-----------------
Key: CASSANDRA-423
URL: https://issues.apache.org/jira/browse/CASSANDRA-423
Project: Cassandra
Issue Type: New Feature
Reporter: Jonathan Ellis
Fix For: 0.5
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-423:
-------------------------------------
Attachment: (was: 423-v0.patch)
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Sammy Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753797#action_12753797 ]
Sammy Yu commented on CASSANDRA-423:
------------------------------------
Both jars are needed. CLRUCache depends on the PriorityQueue class in lucene.
I think the isAlive is used to indicate if the lucene index that this cache is associated is alive, this happens when the a new index is being swapped in for a new index. During that time the CLRUCache for the old index is marked as no longer being alive so that the gets don't update the last accessed time (used to repopulate the old cached keys in the new CLRUCache). If we were to use it we could just leave it as alive.
We could remove the atomic lock for the get miss since it's only used for statistical purposes. The other atomic get is needed to keep track of when the key was last used, this doesn't look like it could be easily removed, but then again it's only a single lock.
Yeah I just arbitrary picked size-1 for the low watermark because it more closely matches what we have right now. We could probably expose this in the config. I would assume that we still want to keep the existing fraction number of keys cached and then have apply another percentage of top the fraction number of keys cached for the low watermark which can help limit memory usage.
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-v0.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755132#action_12755132 ]
Jonathan Ellis commented on CASSANDRA-423:
------------------------------------------
fwiw I checked hbase's LruHashMap and it just synchronizes completely get, put, contains, etc. So, probably not worth trying.
IMO the best option is probably the original clru's fixed version.
I'll dust off my stress.py-with-readers code so we can do some benchmarking.
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-v0.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-423:
-------------------------------------
Attachment: (was: 423-clhm-rebased.patch)
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm-rebased-2.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-423:
-------------------------------------
Attachment: (was: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch)
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Sammy Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sammy Yu updated CASSANDRA-423:
-------------------------------
Attachment: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch
Separated out new SOLR/Lucene JAR file and patch
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-v0.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765125#action_12765125 ]
Chris Goffinet commented on CASSANDRA-423:
------------------------------------------
Tested the latest rebased patch (v3). In production with nodes that are not using LRU cache we see on average 3.4ms on reads. Node with LRU cache enabled we see on average 1.71ms.
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm-rebased-3.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768721#action_12768721 ]
Jonathan Ellis commented on CASSANDRA-423:
------------------------------------------
added missing jar
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm-rebased-3.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Michael Greene (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768716#action_12768716 ]
Michael Greene commented on CASSANDRA-423:
------------------------------------------
Unable to build trunk after commit:
/home/cassandra/sources/cassandra-clhm/src/java/org/apache/cassandra/io/SSTableReader.java:39: package com.reardencommerce.kernel.collections.shared.evictable does not exist
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm-rebased-3.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753645#action_12753645 ]
Jonathan Ellis commented on CASSANDRA-423:
------------------------------------------
we need both solr _and_ lucene jars for ConcurrentLRUCache?
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-v0.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12769206#action_12769206 ]
Hudson commented on CASSANDRA-423:
----------------------------------
Integrated in Cassandra #236 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/236/])
CLHM, fixed version
patch by jbellis; reviewed by goffinet for
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm-rebased-3.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-423:
-------------------------------------
Attachment: 423-v0.patch
this patch adds a key cache skeleton similar to the one reverted from 0.4. Doesn't actually build since we're not actually using CLRU anymore, but takes care of some boilerplate for whatever its replacement is.
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 423-v0.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755673#action_12755673 ]
Jonathan Ellis commented on CASSANDRA-423:
------------------------------------------
i added a "read" method to stress.py for CASSANDRA-442. could you use that to benchmark the two cache versions here? i would expect to see greater throughput with that, than with the solr one, since there is less lock contention.
See http://code.google.com/p/concurrentlinkedhashmap/issues/detail?id=9 and http://code.google.com/p/concurrentlinkedhashmap/wiki/ProductionVersion for information on the CLHM bug fix.
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm.patch, 423-v0.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-423:
-------------------------------------
Attachment: 423-clhm-rebased.patch
rebased clhm-based patch
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm-rebased.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-423:
-------------------------------------
Attachment: 423-clhm.patch
Here is a patch for the CLHM code, rebased to trunk against the bugfixed "production" jar produced by the CLHM project.
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm.patch, 423-v0.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12757805#action_12757805 ]
Chris Goffinet commented on CASSANDRA-423:
------------------------------------------
I am working on trying to get real test numbers from our cluster on this. I am waiting on Digg's Ops to allocate some more machines for us, so we can try this patch out.
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm.patch, 423-v0.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12757810#action_12757810 ]
Jonathan Ellis commented on CASSANDRA-423:
------------------------------------------
which, sammy's or mine? or both?
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm.patch, 423-v0.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Sammy Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sammy Yu updated CASSANDRA-423:
-------------------------------
Attachment: (was: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch)
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-v0.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-423:
-------------------------------------
Attachment: clhm-production.jar
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm.patch, 423-v0.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Sammy Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sammy Yu updated CASSANDRA-423:
-------------------------------
Attachment: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch
Attached first version using SOLR's LRUConcurrentCache.
There are some opened questions. One regarding how to configure the cache, it has a lowWatermark and acceptableWaterMark, I've made some questionable code to allow this to use KeysCachedFraction. If size is less than 0, force key size to 1 and lowWaterMark to 0.
There's also a second hack which calls destroy on construction of keycache this is because by default the finalize() method will log an error message if the destroy has been set.
See code at http://svn.apache.org/viewvc/lucene/solr/trunk/src/common/org/apache/solr/common/util/ConcurrentLRUCache.java?view=markup
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-v0.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Sammy Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sammy Yu updated CASSANDRA-423:
-------------------------------
Attachment: lucene-core-2.9.0-rc2.jar
apache-solr-solrj-2009-09-04_08-06-33.jar
SOLR and Lucene jar files
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-v0.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Sammy Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sammy Yu updated CASSANDRA-423:
-------------------------------
Attachment: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch
Attached is initial version of the patch using SOLR's LRUConcurrentCache.
There are some opened questions. One regarding how to configure the cache, it has a lowWatermark and acceptableWaterMark, I've made some questionable code to allow this to use KeysCachedFraction. If size is less than 0, force key size to 1 and lowWaterMark to 0.
There's also a second hack which calls destroy on construction of keycache this is because by default the finalize() method will log an error message if the destroy has been set.
See code at http://svn.apache.org/viewvc/lucene/solr/trunk/src/common/org/apache/solr/common/util/ConcurrentLRUCache.java?view=markup
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-v0.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780253#action_12780253 ]
Jonathan Ellis commented on CASSANDRA-423:
------------------------------------------
We're not using a soft-reference based cache, but for the record, here is some very good information about the gory details of when soft references are collected: http://jeremymanson.blogspot.com/2009/07/how-hotspot-decides-to-clear_07.html
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm-rebased-3.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768525#action_12768525 ]
Chris Goffinet commented on CASSANDRA-423:
------------------------------------------
+1
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm-rebased-3.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-423:
-------------------------------------
Attachment: 423-clhm-rebased-2.patch
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm-rebased-2.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-423:
-------------------------------------
Attachment: (was: 423-clhm-rebased-2.patch)
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm-rebased-3.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12757806#action_12757806 ]
Chris Goffinet commented on CASSANDRA-423:
------------------------------------------
Please rebase. Does not apply to trunk.
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm.patch, 423-v0.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12757812#action_12757812 ]
Chris Goffinet commented on CASSANDRA-423:
------------------------------------------
Both. All of these patches are very confusing. Could we get a new patchset rebased working with trunk that just includes the CLHM prod.
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm.patch, 423-v0.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-423:
-------------------------------------
Attachment: (was: 423-clhm.patch)
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Sammy Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sammy Yu updated CASSANDRA-423:
-------------------------------
Comment: was deleted
(was: Attached first version using SOLR's LRUConcurrentCache.
There are some opened questions. One regarding how to configure the cache, it has a lowWatermark and acceptableWaterMark, I've made some questionable code to allow this to use KeysCachedFraction. If size is less than 0, force key size to 1 and lowWaterMark to 0.
There's also a second hack which calls destroy on construction of keycache this is because by default the finalize() method will log an error message if the destroy has been set.
See code at http://svn.apache.org/viewvc/lucene/solr/trunk/src/common/org/apache/solr/common/util/ConcurrentLRUCache.java?view=markup
)
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-v0.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12752679#action_12752679 ]
Jonathan Ellis commented on CASSANDRA-423:
------------------------------------------
Can you break the jar files out of the .patch and attach separately?
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-v0.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-423:
-------------------------------------
Attachment: 423-clhm-rebased-3.patch
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-clhm-rebased-3.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, clhm-production.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-423) add new LRU cache
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753652#action_12753652 ]
Jonathan Ellis commented on CASSANDRA-423:
------------------------------------------
new ConcurrentLRUCache(size, size-1);
this is not the way to use CLRUCache judging from the source. (lower should not be upper-1 except for really really small sizes.)
wth is up with CLRUC.setAlive?
CLRUC.get goes through several atomicints. you're not going to get very good concurrency that way.
not very impressed with CLRUC, although on the plus side it is pretty simple.
> add new LRU cache
> -----------------
>
> Key: CASSANDRA-423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-423
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 0001-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 0002-CASSANDRA-423-Committed-first-version-using-Solr-s-C.patch, 423-v0.patch, apache-solr-solrj-2009-09-04_08-06-33.jar, lucene-core-2.9.0-rc2.jar
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.