Re: How to leverage the LogMergePolicy "calibrateSizeByDeletes" patch in Solr ?

[Grant Ingersoll <gs...@apache.org>]

On Sep 21, 2009, at 9:35 PM, John Wang wrote:

> Jason:
>
>     You are missing the point.
>
>     The idea is to avoid merging of large segments. The point of  
> this MergePolicy is to balance segment merges across the index. The  
> aim is not to have 1 large segment, it is to have n segments with  
> balanced sizes.
>
>     When the large segment is out of the IO cache, replacing it is  
> very costly. What we have done is to split the cost over time by  
> having more frequent but faster merges.
>


Yeah, I have seen this in action several times as well.  See also some  
discussion at:  http://www.lucidimagination.com/search/document/bd53b0431f7eada5/concurrentmergescheduler_and_mergepolicy_question 
.