You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Artem Vasiliev (JIRA)" <ji...@apache.org> on 2007/01/10 19:10:27 UTC
[jira] Commented: (LUCENE-769) [PATCH] Performance improvement for
some cases of sorted search
[ https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12463652 ]
Artem Vasiliev commented on LUCENE-769:
---------------------------------------
I've tried the test with DOCS_NUM == 10,000,000. DocCaching sort took about 1s with standart amount of memory (-Xmx80m) while FieldCache'd trapped with OutOfMemoryError even with 1G (-Xmx1000m). Resulting index size was 640M, its creation took at least 7hrs.
> [PATCH] Performance improvement for some cases of sorted search
> ---------------------------------------------------------------
>
> Key: LUCENE-769
> URL: https://issues.apache.org/jira/browse/LUCENE-769
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.0.0
> Reporter: Artem Vasiliev
> Attachments: DocCachingSorting.patch
>
>
> It's a small addition to Lucene that significantly lowers memory consumption and improves performance for sorted searches with frequent index updates and relatively big indexes (>1mln docs) scenario. This solution supports only single-field sorting currently (which seem to be quite popular use case). Multiple fields support can be added without much trouble.
> The solution is this: documents from the sorting set (instead of given field's values from the whole index - current FieldCache approach) are cached in a WeakHashMap so the cached items are candidates for GC. Their fields values are then fetched from the cache and compared while sorting.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org