Re: How to leverage the LogMergePolicy "calibrateSizeByDeletes" patch in Solr ?

[Jibo John <ji...@mac.com>]

On Sep 17, 2009, at 1:30 PM, Shalin Shekhar Mangar wrote:

> On Fri, Sep 18, 2009 at 1:06 AM, Jibo John <ji...@mac.com> wrote:
>
>> Hello,
>>
>> Came across a lucene patch (
>> http://issues.apache.org/jira/browse/LUCENE-1634) that would  
>> consider the
>> number of deleted documents as the criteria when deciding which  
>> segments to
>> merge.
>>
>> Since we expect to have very frequent deletes, we hope this would  
>> help
>> reclaim the space consumed by the deleted documents in a much more  
>> efficient
>> way.
>>
>> Currently, we can specify a mergepolicy in solrconfig.xml like this:
>>
>>
>>
>> <!--<mergePolicy>org.apache.lucene.index.LogByteSizeMergePolicy</ 
>> mergePolicy>-->
>>
>>
>> However, by default, calibrateSizeByDeletes = false in  
>> LogMergePolicy.
>>
>> I was wondering if there is a way I can modify  
>> calibrateSizeByDeletes just
>> by configuration ?
>>
>
> Alas, no. The only option that I see for you is to sub-class
> LogByteSizeMergePolicy and set calibrateSizeByDeletes to true in the
> constructor. However, please open a Jira issue and so we don't  
> forget about
> it.

Created a jira issue https://issues.apache.org/jira/browse/SOLR-1444

>
> Also, you might be interested in expungeDeletes which has been added  
> as a
> request parameter for commits. Calling commit with  
> expungeDeletes=true will
> remove all deleted documents from the index but unlike an optimize  
> it won't
> always reduce the index to a single segment.

Thanks for this information. Will explore this.



>
> -- 
> Regards,
> Shalin Shekhar Mangar.


Thanks,
-Jibo