Lucene Optimization

[Siraj Haider <si...@jobdiva.com>]

We currently use Lucene 2.9 and to keep the indexes running faster we optimize the indexes during night. In our application the volume of new documents coming in is very high so most of our indexes have to merge segments during the day too, when the document count reaches certain number. This causes tremendous slowness on the machine the online searches suffer too to the extent that we need to restart tomcat on those machines very frequently. Right now we are in the process to upgrade to version 6.1.0. I have read that there is no need to optimize in 6.1.0 but what about the segment merges, will those happen as before in 6.1.0 too? I would like to know the best way to tackle this issue, so please help me out.

------
Regards
-Siraj


________________________________

This electronic mail message and any attachments may contain information which is privileged, sensitive and/or otherwise exempt from disclosure under applicable law. The information is intended only for the use of the individual or entity named as the addressee above. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution (electronic or otherwise) or forwarding of, or the taking of any action in reliance on, the contents of this transmission is strictly prohibited. If you have received this electronic transmission in error, please notify us by telephone, facsimile, or e-mail as noted above to arrange for the return of any electronic mail or attachments. Thank You.

Re: Lucene Optimization

[Parit Bansal <Pa...@sib.swiss>]

On 07/13/2016 12:43 AM, Siraj Haider wrote:
> We currently use Lucene 2.9 and to keep the indexes running faster we optimize the indexes during night. In our application the volume of new documents coming in is very high so most of our indexes have to merge segments during the day too, when the document count reaches certain number. This causes tremendous slowness on the machine the online searches suffer too to the extent that we need to restart tomcat on those machines very frequently. Right now we are in the process to upgrade to version 6.1.0. I have read that there is no need to optimize in 6.1.0 but what about the segment merges, will those happen as before in 6.1.0 too? I would like to know the best way to tackle this issue, so please help me out.
>
> ------
> Regards
> -Siraj
>
>
> ________________________________
>
> This electronic mail message and any attachments may contain information which is privileged, sensitive and/or otherwise exempt from disclosure under applicable law. The information is intended only for the use of the individual or entity named as the addressee above. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution (electronic or otherwise) or forwarding of, or the taking of any action in reliance on, the contents of this transmission is strictly prohibited. If you have received this electronic transmission in error, please notify us by telephone, facsimile, or e-mail as noted above to arrange for the return of any electronic mail or attachments. Thank You.
>
Hi Siraj,

In 2.9 lucene, underlying index was considered one big index and 
therefore there was a need to optimize (read 'merge segments') index. 
But (I think) since 3.5, the orientation is towards per segment 
searching. Therefore as such 'no need' to merge segments. But you can 
still force it in cases like based on number of documents in a segment.

- Best

Parit



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org