You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Mark Miller (JIRA)" <ji...@apache.org> on 2008/12/09 14:42:44 UTC
[jira] Issue Comment Edited: (LUCENE-1483) Change IndexSearcher to
use MultiSearcher semantics for sorted searches
[ https://issues.apache.org/jira/browse/LUCENE-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654798#action_12654798 ]
markrmiller@gmail.com edited comment on LUCENE-1483 at 12/9/08 5:41 AM:
--------------------------------------------------------------
Quick micro bench - did it twice and both times came out 17% slower. Hopefully get a lot of that back with the new MultiSearcher sort stuff and maybe some optimizations.
{panel:title=OLD}
{noformat}
[java] ------------> Report Sum By (any) Name (12 about 42 out of 43)
[java] Operation round 20000 mrg runCnt recsPerRun rec/s elapsedSec avgUsedMem avgTotalMem
[java] Rounds 0 10000 50 1 2020012 11,532.6 175.16 49,688,960 200,736,768
[java] Run_4 - - - - - - - 0 10000 50 - - 1 - - 2020012 - 11,532.6 - - 175.16 - 49,688,960 - 200,736,768
[java] Populate - - - 4 500000 21,446.3 93.26 81,397,296 156,942,336
[java] CreateIndex - - - - - - - - - - - - 4 - - - - 0 - - - 0.0 - - 0.23 - 16,492,194 - 112,984,064
[java] MAddDocs_500000 - - - 4 500000 28,656.2 69.79 83,686,552 153,223,168
[java] Optimize - - - - - - - - - - - - - 4 - - - - 0 - - - 0.0 - - 23.22 - 101,362,928 - 156,942,336
[java] CloseIndex - - - 4 0 0.0 0.00 81,397,296 156,942,336
[java] TestSortSpeed - - - - - - - - - - - - 4 - - - 5003 - - 246.0 - - 81.35 - 98,312,320 - 157,941,760
[java] OpenReader - - - 4 1 266.7 0.01 81,397,296 156,942,336
[java] LoadFieldCacheAndSearch - - - - - - - - 4 - - - - 1 - - - 6.2 - - 0.64 - 90,550,496 - 156,942,336
[java] SearchWithSort_5000 - - - 4 5000 247.9 80.69 101,017,720 157,941,760
[java] CloseReader - - - - - - - - - - - - 4 - - - - 1 - - 4,000.0 - - 0.00 - 95,036,504 - 157,941,760
[java] ####################
[java] ### D O N E !!! ###
[java] ####################
{noformat}
{panel}
{panel:title=NEW}
{noformat}
[java] ------------> Report Sum By (any) Name (12 about 42 out of 43)
[java] Operation round 20000 mrg runCnt recsPerRun rec/s elapsedSec avgUsedMem avgTotalMem
[java] Rounds 0 10000 50 1 2020012 10,445.5 193.38 125,468,912 208,535,552
[java] Run_4 - - - - - - - 0 10000 50 - - 1 - - 2020012 - 10,445.5 - - 193.38 - 125,468,912 - 208,535,552
[java] Populate - - - 4 500000 20,650.1 96.85 84,097,072 162,316,288
[java] CreateIndex - - - - - - - - - - - - 4 - - - - 0 - - - 0.0 - - 0.12 - 16,564,602 - 116,604,928
[java] MAddDocs_500000 - - - 4 500000 28,772.4 69.51 87,705,952 159,956,992
[java] Optimize - - - - - - - - - - - - - 4 - - - - 0 - - - 0.0 - - 27.20 - 99,096,816 - 162,316,288
[java] CloseIndex - - - 4 0 0.0 0.00 84,097,072 162,316,288
[java] TestSortSpeed - - - - - - - - - - - - 4 - - - 5003 - - 208.5 - - 95.99 - 98,749,480 - 164,020,224
[java] OpenReader - - - 4 1 222.2 0.02 84,097,072 162,316,288
[java] LoadFieldCacheAndSearch - - - - - - - - 4 - - - - 1 - - - 5.0 - - 0.81 - 90,882,496 - 163,725,312
[java] SearchWithSort_5000 - - - 4 5000 210.2 95.17 95,207,336 164,020,224
[java] CloseReader - - - - - - - - - - - - 4 - - - - 1 - - 4,000.0 - - 0.00 - 93,868,880 - 163,905,536
[java] ####################
[java] ### D O N E !!! ###
[java] ####################
{noformat}
{panel}
was (Author: markrmiller@gmail.com):
Quick micro bench - did it twice and both times came out 17% slower. Hopefully get a lot of that back with the new MultiSearcher sort stuff and maybe some optimizations.
{code}
OLD
[java] ------------> Report Sum By (any) Name (12 about 42 out of 43)
[java] Operation round 20000 mrg runCnt recsPerRun rec/s elapsedSec avgUsedMem avgTotalMem
[java] Rounds 0 10000 50 1 2020012 11,532.6 175.16 49,688,960 200,736,768
[java] Run_4 - - - - - - - 0 10000 50 - - 1 - - 2020012 - 11,532.6 - - 175.16 - 49,688,960 - 200,736,768
[java] Populate - - - 4 500000 21,446.3 93.26 81,397,296 156,942,336
[java] CreateIndex - - - - - - - - - - - - 4 - - - - 0 - - - 0.0 - - 0.23 - 16,492,194 - 112,984,064
[java] MAddDocs_500000 - - - 4 500000 28,656.2 69.79 83,686,552 153,223,168
[java] Optimize - - - - - - - - - - - - - 4 - - - - 0 - - - 0.0 - - 23.22 - 101,362,928 - 156,942,336
[java] CloseIndex - - - 4 0 0.0 0.00 81,397,296 156,942,336
[java] TestSortSpeed - - - - - - - - - - - - 4 - - - 5003 - - 246.0 - - 81.35 - 98,312,320 - 157,941,760
[java] OpenReader - - - 4 1 266.7 0.01 81,397,296 156,942,336
[java] LoadFieldCacheAndSearch - - - - - - - - 4 - - - - 1 - - - 6.2 - - 0.64 - 90,550,496 - 156,942,336
[java] SearchWithSort_5000 - - - 4 5000 247.9 80.69 101,017,720 157,941,760
[java] CloseReader - - - - - - - - - - - - 4 - - - - 1 - - 4,000.0 - - 0.00 - 95,036,504 - 157,941,760
[java] ####################
[java] ### D O N E !!! ###
[java] ####################
NEW
[java] ------------> Report Sum By (any) Name (12 about 42 out of 43)
[java] Operation round 20000 mrg runCnt recsPerRun rec/s elapsedSec avgUsedMem avgTotalMem
[java] Rounds 0 10000 50 1 2020012 10,445.5 193.38 125,468,912 208,535,552
[java] Run_4 - - - - - - - 0 10000 50 - - 1 - - 2020012 - 10,445.5 - - 193.38 - 125,468,912 - 208,535,552
[java] Populate - - - 4 500000 20,650.1 96.85 84,097,072 162,316,288
[java] CreateIndex - - - - - - - - - - - - 4 - - - - 0 - - - 0.0 - - 0.12 - 16,564,602 - 116,604,928
[java] MAddDocs_500000 - - - 4 500000 28,772.4 69.51 87,705,952 159,956,992
[java] Optimize - - - - - - - - - - - - - 4 - - - - 0 - - - 0.0 - - 27.20 - 99,096,816 - 162,316,288
[java] CloseIndex - - - 4 0 0.0 0.00 84,097,072 162,316,288
[java] TestSortSpeed - - - - - - - - - - - - 4 - - - 5003 - - 208.5 - - 95.99 - 98,749,480 - 164,020,224
[java] OpenReader - - - 4 1 222.2 0.02 84,097,072 162,316,288
[java] LoadFieldCacheAndSearch - - - - - - - - 4 - - - - 1 - - - 5.0 - - 0.81 - 90,882,496 - 163,725,312
[java] SearchWithSort_5000 - - - 4 5000 210.2 95.17 95,207,336 164,020,224
[java] CloseReader - - - - - - - - - - - - 4 - - - - 1 - - 4,000.0 - - 0.00 - 93,868,880 - 163,905,536
[java] ####################
[java] ### D O N E !!! ###
[java] ####################
{/code}
> Change IndexSearcher to use MultiSearcher semantics for sorted searches
> -----------------------------------------------------------------------
>
> Key: LUCENE-1483
> URL: https://issues.apache.org/jira/browse/LUCENE-1483
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.9
> Reporter: Mark Miller
> Priority: Minor
> Attachments: LUCENE-1483.patch
>
>
> Here is a quick test patch. FieldCache for sorting is done at the individual IndexReader level and reloading the fieldcache on reopen can be much faster as only changed segments need to be reloaded.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org