Re: NullPointerException in ReplicationHandler.postCommit + question about compression

[Stephen Weiss <sw...@stylesight.com>]

Hi Shalin,

Thanks for your reply.  Please see below.


On Jan 18, 2010, at 4:19 AM, Shalin Shekhar Mangar wrote:

> On Wed, Jan 13, 2010 at 12:51 AM, Stephen Weiss  
> <sw...@stylesight.com>wrote:
> ...

>>  When we replicate
>> manually (via the admin page) things seem to go well.  However, when
>> replication is triggered by a commit event on the master, the  
>> master gets a
>> NullPointerException and no replication seems to take place.
>>
>> SEVERE: java.lang.NullPointerException
>>>       at
>>> org.apache.solr.handler.ReplicationHandler 
>>> $4.postCommit(ReplicationHandler.java:922)
>>>       at...
>>
>> Does anyone know off the top of their head what this might  
>> indicate, or
>> know what further troubleshooting steps we should be taking to  
>> isolate the
>> issue?
>>
>
> That is a strange one. It looks like the latest commit point was  
> null. Do
> you have a deletion policy section in your solrconfig.xml? Are you  
> always
> able to reproduce the exception?

We are always able to reproduce the exception.

The master has committed changes many times for over a year now... so  
if that's what's being reported, it's not quite accurate.

This is our deletion policy.  I don't believe that I've edited it, it  
is probably verbatim from the example (the example of what version of  
Solr, I can't tell you for sure, but I imagine it's from 1.2 or 1.4 -  
we never updated the config when using 1.3).

>     <deletionPolicy class="solr.SolrDeletionPolicy">
>       <str name="maxCommitsToKeep">1</str>
>       <str name="maxOptimizedCommitsToKeep">0</str>
>     </deletionPolicy>

(removing comments per Noble Paul's request...  we keep these in the  
file for our own readability purposes but agreed, we have no need to e- 
mail them along)

I would have never thought to look there but it does seem suspicious  
now that you mention it.  For a proper replication configuration where  
we replicate on commit, is there a recommended setting?

>> ...
>>
> During our tests we found that enabling compression on a gigabit  
> ethernet
> actually degrades transfer rate because of the compress/de-compress
> overhead. Just comment out that line to disable compression.

Thank you for the clarification.  We will comment it out.

--
Steve

Re: NullPointerException in ReplicationHandler.postCommit + question about compression

[Shalin Shekhar Mangar <sh...@gmail.com>]

On Sat, Jan 30, 2010 at 5:08 AM, Chris Hostetter
<ho...@fucit.org>wrote:

>
> : never keep a <str name="maxOptimizedCommitsToKeep">0</str>.
> :
> : It is better to leave not mention the deletionPolicy at all. The
> : defaults are usually fine.
>
> if setting the "keep" values to 0 results in NPEs we should do one (if not
> both) of the following...
>
> 1) change the init code to warn/fail if the values are 0 (not sure if
> there is ever a legitimate use for 0 as a value)
>
> 2) change the code that's currently throwing an NPE to check it's
> assumptings and log a more meaninful error if it can't function because of
> the existing config.
>
>
Setting the keep values to 0 does not result in NPEs. Setting both
maxCommitsToKeep and maxOptimizedCommitsToKeep to 0 is invalid and should be
checked for. However, this problem is different. We use the same
configuration in production and we haven't seen an NPE like that. I'm not
able to reproduce this locally too.

-- 
Regards,
Shalin Shekhar Mangar.

Re: NullPointerException in ReplicationHandler.postCommit + question about compression

[Chris Hostetter <ho...@fucit.org>]

: never keep a <str name="maxOptimizedCommitsToKeep">0</str>.
: 
: It is better to leave not mention the deletionPolicy at all. The
: defaults are usually fine.

if setting the "keep" values to 0 results in NPEs we should do one (if not 
both) of the following...

1) change the init code to warn/fail if the values are 0 (not sure if 
there is ever a legitimate use for 0 as a value)

2) change the code that's currently throwing an NPE to check it's 
assumptings and log a more meaninful error if it can't function because of 
the existing config.


-Hoss

Re: NullPointerException in ReplicationHandler.postCommit + question about compression

[Noble Paul നോബിള്‍ नोब्ळ् <no...@corp.aol.com>]

never keep a <str name="maxOptimizedCommitsToKeep">0</str>.

It is better to leave not mention the deletionPolicy at all. The
defaults are usually fine.

On Fri, Jan 22, 2010 at 11:12 AM, Stephen Weiss <sw...@stylesight.com> wrote:
> Hi Shalin,
>
> Thanks for your reply.  Please see below.
>
>
> On Jan 18, 2010, at 4:19 AM, Shalin Shekhar Mangar wrote:
>
>> On Wed, Jan 13, 2010 at 12:51 AM, Stephen Weiss
>> <sw...@stylesight.com>wrote:
>> ...
>
>>>  When we replicate
>>> manually (via the admin page) things seem to go well.  However, when
>>> replication is triggered by a commit event on the master, the master gets
>>> a
>>> NullPointerException and no replication seems to take place.
>>>
>>> SEVERE: java.lang.NullPointerException
>>>>
>>>>      at
>>>>
>>>> org.apache.solr.handler.ReplicationHandler$4.postCommit(ReplicationHandler.java:922)
>>>>      at...
>>>
>>> Does anyone know off the top of their head what this might indicate, or
>>> know what further troubleshooting steps we should be taking to isolate
>>> the
>>> issue?
>>>
>>
>> That is a strange one. It looks like the latest commit point was null. Do
>> you have a deletion policy section in your solrconfig.xml? Are you always
>> able to reproduce the exception?
>
> We are always able to reproduce the exception.
>
> The master has committed changes many times for over a year now... so if
> that's what's being reported, it's not quite accurate.
>
> This is our deletion policy.  I don't believe that I've edited it, it is
> probably verbatim from the example (the example of what version of Solr, I
> can't tell you for sure, but I imagine it's from 1.2 or 1.4 - we never
> updated the config when using 1.3).
>
>>    <deletionPolicy class="solr.SolrDeletionPolicy">
>>      <str name="maxCommitsToKeep">1</str>
>>      <str name="maxOptimizedCommitsToKeep">0</str>
>>    </deletionPolicy>
>
> (removing comments per Noble Paul's request...  we keep these in the file
> for our own readability purposes but agreed, we have no need to e-mail them
> along)
>
> I would have never thought to look there but it does seem suspicious now
> that you mention it.  For a proper replication configuration where we
> replicate on commit, is there a recommended setting?
>
>>> ...
>>>
>> During our tests we found that enabling compression on a gigabit ethernet
>> actually degrades transfer rate because of the compress/de-compress
>> overhead. Just comment out that line to disable compression.
>
> Thank you for the clarification.  We will comment it out.
>
> --
> Steve



-- 
-----------------------------------------------------
Noble Paul | Systems Architect| AOL | http://aol.com