You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Shawn Heisey (Commented) (JIRA)" <ji...@apache.org> on 2011/12/09 19:26:40 UTC
[jira] [Commented] (SOLR-2906) Implement LFU Cache
[ https://issues.apache.org/jira/browse/SOLR-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166399#comment-13166399 ]
Shawn Heisey commented on SOLR-2906:
------------------------------------
I finally got a chance to do some testing in production. I have two distributed index chains, both running 3.5.0 with this patch and the one from SOLR-1972 applied. The chains are updated independently, there is no replication. It's not a truly definitive test, because my queries are load balanced between the two chains and I do have some hardware discrepancies. I cannot create a valid test environment to compare the two caches as they should be compared, with identical queries going to two completely identical servers.
On average, commits made on chain A where filterCache is set to LFU and the servers have 48GB of RAM happen faster than those on chain B, with FastLRU and 64GB of RAM. One of the two servers on chain A has slightly faster processors than its counterpart on chain B -- 2.83 GHz vs. 2.66 GHz. The other two servers on both chains have 2.5 GHz processors.
I suspect that most of the potential gains that the LFU algorithm might be able to provide are swallowed by the very inefficient implementation.
If anyone has some thoughts for me to pursue, I will be happy to do so, but I am out of my own ideas. I hope the patch will be committed. It could use a lot of optimization and there's probably cosmetic cleanup to do.
> Implement LFU Cache
> -------------------
>
> Key: SOLR-2906
> URL: https://issues.apache.org/jira/browse/SOLR-2906
> Project: Solr
> Issue Type: Sub-task
> Components: search
> Affects Versions: 3.4
> Reporter: Shawn Heisey
> Priority: Minor
> Attachments: ConcurrentLFUCache.java, LFUCache.java, SOLR-2906.patch, SOLR-2906.patch, TestLFUCache.java
>
>
> Implement an LFU (Least Frequently Used) cache as the first step towards a full ARC cache
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org