How important is single segment optimization to Search time performance?

[Gili Nachum <gi...@gmail.com>]

Hi there Lucene samurai!

*I was wondering how important is single segment merging for search time
performance compared to a more modest merging goal like merging down to
just 4 segment.
*

Currently my system merges every evening, it takes ~1-2 hours, and
invalidates the File-system cache.
What would happen if I'll reduce the merging target to just 4 segments?
e.g., When searching a term, does Lucene needs to visit each segment (disk
seek operation) to access the relevant posting list? If not then what's the
advantage of merging down segments (besides open files limit).

*Was wondering if anyone has tested # of segments against search time
performance?*

I should add I have ~10 indexes, at a total size of 50GB, and I use
mutli-index searcher to search over them (Lucene 3.0.3 - yeah it's old I
know). The index is updated every 15min.

Gili Nachum.

Re: How important is single segment optimization to Search time performance?

[Uwe Schindler <uw...@thetaphi.de>]

Hi,

I would suggest to read: http://www.searchworkings.org/blog/-/blogs/380798

In general, if the index changes often, don't force merges. IndexWriter automatically merges to a suitable number of segments. 

Uwe



Gili Nachum <gi...@gmail.com> schrieb:
>Hi there Lucene samurai!
>
>*I was wondering how important is single segment merging for search
>time
>performance compared to a more modest merging goal like merging down to
>just 4 segment.
>*
>
>Currently my system merges every evening, it takes ~1-2 hours, and
>invalidates the File-system cache.
>What would happen if I'll reduce the merging target to just 4 segments?
>e.g., When searching a term, does Lucene needs to visit each segment
>(disk
>seek operation) to access the relevant posting list? If not then what's
>the
>advantage of merging down segments (besides open files limit).
>
>*Was wondering if anyone has tested # of segments against search time
>performance?*
>
>I should add I have ~10 indexes, at a total size of 50GB, and I use
>mutli-index searcher to search over them (Lucene 3.0.3 - yeah it's old
>I
>know). The index is updated every 15min.
>
>Gili Nachum.

--
Uwe Schindler
H.-H.-Meier-Allee 63, 28213 Bremen
http://www.thetaphi.de