What about the index writing efficiency of large index ?

[Eric Th <bl...@gmail.com>]

Hi All,
Does anyone do a benchmark to verify the index writing efficiency of lucene?
When the index size is larger than 10G, will it be much slower than smaller
ones ?

Actually i did some works about this issue,
and i found that, if build small index firstly then merge them all, the time
taken will significantly decline.

Hi Yonik,
How does Solr deal with this issue, does it just leave this problem to
lucene?


Thanks,

Eric

Re: What about the index writing efficiency of large index ?

[Yonik Seeley <yo...@apache.org>]

On Thu, Mar 6, 2008 at 3:57 AM, Eric Th <bl...@gmail.com> wrote:
> Hi All,
>  Does anyone do a benchmark to verify the index writing efficiency of lucene?
>  When the index size is larger than 10G, will it be much slower than smaller
>  ones ?
>
>  Actually i did some works about this issue,
>  and i found that, if build small index firstly then merge them all, the time
>  taken will significantly decline.

You can increase the merge factor to get fewer merges,
or you can set maxMergeDocs or setMaxMergeMB to prevent merging of any
segments above a certain size and them call optimize at the end.

>  Hi Yonik,
>  How does Solr deal with this issue, does it just leave this problem to
>  lucene?

Solr leaves it to Lucene.

-Yonik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org