You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jibo John <ji...@mac.com> on 2009/09/26 03:43:03 UTC

Problem changing the default MergePolicy/Scheduler

Hello,

It looks like solr is not allowing me to change the default  
MergePolicy/Scheduler classes.

Even if I change the default MergePolicy/ 
Scheduler(LogByteSizeMErgePolicy and ConcurrentMergeScheduler) defined  
in solrconfig.xml to a different one (LogDocMergePolicy and  
SerialMergeScheduler), my profiler shows the default classes are still  
being loaded.

Also, if I use the default LogByteSizeMergePolicy, I can't seem to  
override the 'calibrateSizeByDeletes' to 'true' value using solrconfig  
using the new syntax that was introduced this week (SOLR-1447).

I'm using the version checked out from trunk yesterday.

Any pointers will be helpful.

Thanks,
-Jibo

Re: Problem changing the default MergePolicy/Scheduler

Posted by Jibo John <ji...@mac.com>.

On Sep 27, 2009, at 9:42 PM, Shalin Shekhar Mangar wrote:

> On Mon, Sep 28, 2009 at 2:59 AM, Jibo John <ji...@mac.com> wrote:
>
>> Additionally, I get the same exception even if I declare the  
>> <mergePolicy>
>> in the <mainIndex>.
>>
>> <mainIndex>
>>   <mergePolicy  
>> class="org.apache.lucene.index.LogByteSizeMergePolicy">
>>     <boolean name="calibrateSizeByDeletes">true</boolean>
>>   </mergePolicy>
>> </mainIndex>
>>
>
> That should be <bool> instead of <boolean>


Yeah, that was it. Thank you very much.

Thanks,
-Jibo


>
> -- 
> Regards,
> Shalin Shekhar Mangar.

Re: Problem changing the default MergePolicy/Scheduler

Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Mon, Sep 28, 2009 at 2:59 AM, Jibo John <ji...@mac.com> wrote:

> Additionally, I get the same exception even if I declare the <mergePolicy>
> in the <mainIndex>.
>
>  <mainIndex>
>    <mergePolicy class="org.apache.lucene.index.LogByteSizeMergePolicy">
>      <boolean name="calibrateSizeByDeletes">true</boolean>
>    </mergePolicy>
>  </mainIndex>
>

That should be <bool> instead of <boolean>

-- 
Regards,
Shalin Shekhar Mangar.

Re: Problem changing the default MergePolicy/Scheduler

Posted by Jibo John <ji...@mac.com>.
Additionally, I get the same exception even if I declare the  
<mergePolicy> in the <mainIndex>.

   <mainIndex>
     <mergePolicy  
class="org.apache.lucene.index.LogByteSizeMergePolicy">
       <boolean name="calibrateSizeByDeletes">true</boolean>
     </mergePolicy>
   </mainIndex>

Thanks,
-Jibo


On Sep 27, 2009, at 2:03 PM, Jibo John wrote:

> Thanks for this. I've updated trunk/, rebuilt solr.war, however,  
> running into another issue.
>
> Sep 27, 2009 1:55:44 PM org.apache.solr.common.SolrException log
> SEVERE: java.lang.IllegalArgumentException
> 	at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
> 	at  
> sun 
> .reflect 
> .DelegatingMethodAccessorImpl 
> .invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:592)
> 	at  
> org 
> .apache.solr.util.SolrPluginUtils.invokeSetters(SolrPluginUtils.java: 
> 989)
> 	at org.apache.solr.update.SolrIndexWriter.init(SolrIndexWriter.java: 
> 87)
> 	at  
> org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java: 
> 185)
> 	at  
> org 
> .apache 
> .solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java: 
> 98)
> 	at  
> org 
> .apache 
> .solr 
> .update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:173)
> 	at  
> org 
> .apache 
> .solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java: 
> 220)
> 	at  
> org 
> .apache 
> .solr 
> .update 
> .processor 
> .RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:61)
> 	at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java: 
> 140)
> 	at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69)
>
>
> Also, the log file has a bunch of these lines:
>
> Sep 27, 2009 1:55:56 PM org.apache.solr.update.SolrIndexWriter  
> finalize
> SEVERE: SolrIndexWriter was not closed prior to finalize(),  
> indicates a bug -- POSSIBLE RESOURCE LEAK!!!
>
>
> Here is the snippet from my solrconfig.xml
>
>
>  <indexDefaults>
>   <!-- Values here affect all index writers and act as a default  
> unless overridden. -->
>    <useCompoundFile>false</useCompoundFile>
>
>    <mergeFactor>6</mergeFactor>
>    <ramBufferSizeMB>500</ramBufferSizeMB>
>    <maxFieldLength>10000</maxFieldLength>
>    <writeLockTimeout>1000</writeLockTimeout>
>    <commitLockTimeout>10000</commitLockTimeout>
>    <mergePolicy  
> class="org.apache.lucene.index.LogByteSizeMergePolicy">
>      <boolean name="calibrateSizeByDeletes">true</boolean>
>    </mergePolicy>
>    <mergeScheduler>org.apache.lucene.index.ConcurrentMergeScheduler</ 
> mergeScheduler>
>    <lockType>single</lockType>
>  </indexDefaults>
>
>
>
>
> Thanks,
> -Jibo
>
>
>
> On Sep 27, 2009, at 1:43 PM, Shalin Shekhar Mangar wrote:
>
>> On Mon, Sep 28, 2009 at 1:18 AM, Shalin Shekhar Mangar <
>> shalinmangar@gmail.com> wrote:
>>
>>> On Sat, Sep 26, 2009 at 7:13 AM, Jibo John <ji...@mac.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> It looks like solr is not allowing me to change the default
>>>> MergePolicy/Scheduler classes.
>>>>
>>>> Even if I change the default MergePolicy/ 
>>>> Scheduler(LogByteSizeMErgePolicy
>>>> and ConcurrentMergeScheduler) defined in solrconfig.xml to a  
>>>> different one
>>>> (LogDocMergePolicy and SerialMergeScheduler), my profiler shows  
>>>> the default
>>>> classes are still being loaded.
>>>>
>>>> Also, if I use the default LogByteSizeMergePolicy, I can't seem to
>>>> override the 'calibrateSizeByDeletes' to 'true' value using  
>>>> solrconfig using
>>>> the new syntax that was introduced this week (SOLR-1447).
>>>>
>>>> I'm using the version checked out from trunk yesterday.
>>>>
>>>> Any pointers will be helpful.
>>>>
>>>>
>>> Specifying mergePolicy and mergeScheduler in <indexDefaults> does  
>>> not work
>>> in trunk. If you specify in the <mainIndex> section, it will work.  
>>> I'll give
>>> a patch with a fix.
>>>
>>
>> This is fixed in trunk now. Thanks!
>>
>> -- 
>> Regards,
>> Shalin Shekhar Mangar.
>

Re: Problem changing the default MergePolicy/Scheduler

Posted by Jibo John <ji...@mac.com>.
Thanks for this. I've updated trunk/, rebuilt solr.war, however,  
running into another issue.

Sep 27, 2009 1:55:44 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.IllegalArgumentException
	at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
	at  
sun 
.reflect 
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 
25)
	at java.lang.reflect.Method.invoke(Method.java:592)
	at  
org 
.apache.solr.util.SolrPluginUtils.invokeSetters(SolrPluginUtils.java: 
989)
	at org.apache.solr.update.SolrIndexWriter.init(SolrIndexWriter.java:87)
	at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java: 
185)
	at  
org 
.apache 
.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:98)
	at  
org 
.apache 
.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java: 
173)
	at  
org 
.apache 
.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:220)
	at  
org 
.apache 
.solr 
.update 
.processor 
.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:61)
	at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:140)
	at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69)


Also, the log file has a bunch of these lines:

Sep 27, 2009 1:55:56 PM org.apache.solr.update.SolrIndexWriter finalize
SEVERE: SolrIndexWriter was not closed prior to finalize(), indicates  
a bug -- POSSIBLE RESOURCE LEAK!!!


Here is the snippet from my solrconfig.xml


   <indexDefaults>
    <!-- Values here affect all index writers and act as a default  
unless overridden. -->
     <useCompoundFile>false</useCompoundFile>

     <mergeFactor>6</mergeFactor>
     <ramBufferSizeMB>500</ramBufferSizeMB>
     <maxFieldLength>10000</maxFieldLength>
     <writeLockTimeout>1000</writeLockTimeout>
     <commitLockTimeout>10000</commitLockTimeout>
     <mergePolicy  
class="org.apache.lucene.index.LogByteSizeMergePolicy">
       <boolean name="calibrateSizeByDeletes">true</boolean>
     </mergePolicy>
     <mergeScheduler>org.apache.lucene.index.ConcurrentMergeScheduler</ 
mergeScheduler>
     <lockType>single</lockType>
   </indexDefaults>




Thanks,
-Jibo



On Sep 27, 2009, at 1:43 PM, Shalin Shekhar Mangar wrote:

> On Mon, Sep 28, 2009 at 1:18 AM, Shalin Shekhar Mangar <
> shalinmangar@gmail.com> wrote:
>
>> On Sat, Sep 26, 2009 at 7:13 AM, Jibo John <ji...@mac.com> wrote:
>>
>>> Hello,
>>>
>>> It looks like solr is not allowing me to change the default
>>> MergePolicy/Scheduler classes.
>>>
>>> Even if I change the default MergePolicy/ 
>>> Scheduler(LogByteSizeMErgePolicy
>>> and ConcurrentMergeScheduler) defined in solrconfig.xml to a  
>>> different one
>>> (LogDocMergePolicy and SerialMergeScheduler), my profiler shows  
>>> the default
>>> classes are still being loaded.
>>>
>>> Also, if I use the default LogByteSizeMergePolicy, I can't seem to
>>> override the 'calibrateSizeByDeletes' to 'true' value using  
>>> solrconfig using
>>> the new syntax that was introduced this week (SOLR-1447).
>>>
>>> I'm using the version checked out from trunk yesterday.
>>>
>>> Any pointers will be helpful.
>>>
>>>
>> Specifying mergePolicy and mergeScheduler in <indexDefaults> does  
>> not work
>> in trunk. If you specify in the <mainIndex> section, it will work.  
>> I'll give
>> a patch with a fix.
>>
>
> This is fixed in trunk now. Thanks!
>
> -- 
> Regards,
> Shalin Shekhar Mangar.

Re: Problem changing the default MergePolicy/Scheduler

Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Mon, Sep 28, 2009 at 1:18 AM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:

> On Sat, Sep 26, 2009 at 7:13 AM, Jibo John <ji...@mac.com> wrote:
>
>> Hello,
>>
>> It looks like solr is not allowing me to change the default
>> MergePolicy/Scheduler classes.
>>
>> Even if I change the default MergePolicy/Scheduler(LogByteSizeMErgePolicy
>> and ConcurrentMergeScheduler) defined in solrconfig.xml to a different one
>> (LogDocMergePolicy and SerialMergeScheduler), my profiler shows the default
>> classes are still being loaded.
>>
>> Also, if I use the default LogByteSizeMergePolicy, I can't seem to
>> override the 'calibrateSizeByDeletes' to 'true' value using solrconfig using
>> the new syntax that was introduced this week (SOLR-1447).
>>
>> I'm using the version checked out from trunk yesterday.
>>
>> Any pointers will be helpful.
>>
>>
> Specifying mergePolicy and mergeScheduler in <indexDefaults> does not work
> in trunk. If you specify in the <mainIndex> section, it will work. I'll give
> a patch with a fix.
>

This is fixed in trunk now. Thanks!

-- 
Regards,
Shalin Shekhar Mangar.

Re: Problem changing the default MergePolicy/Scheduler

Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Sat, Sep 26, 2009 at 7:13 AM, Jibo John <ji...@mac.com> wrote:

> Hello,
>
> It looks like solr is not allowing me to change the default
> MergePolicy/Scheduler classes.
>
> Even if I change the default MergePolicy/Scheduler(LogByteSizeMErgePolicy
> and ConcurrentMergeScheduler) defined in solrconfig.xml to a different one
> (LogDocMergePolicy and SerialMergeScheduler), my profiler shows the default
> classes are still being loaded.
>
> Also, if I use the default LogByteSizeMergePolicy, I can't seem to override
> the 'calibrateSizeByDeletes' to 'true' value using solrconfig using the new
> syntax that was introduced this week (SOLR-1447).
>
> I'm using the version checked out from trunk yesterday.
>
> Any pointers will be helpful.
>
>
Specifying mergePolicy and mergeScheduler in <indexDefaults> does not work
in trunk. If you specify in the <mainIndex> section, it will work. I'll give
a patch with a fix.

-- 
Regards,
Shalin Shekhar Mangar.