You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by Marc Sturlese <ma...@gmail.com> on 2009/03/13 17:39:56 UTC

Re: Tomcat holding deleted snapshots until it's restarted - SOLVED!!!

Hey Yonik,
I tested the last nightly build and still happens... but I have solved it! I
tell you my solution, it seems to be working well but just want to be sure
that it doesn't have any bad effects as for me this is one of the most
complicated parts of the Solr source (the fact of dealing with multiple
indexsearchers in a syncronized way).
I noticed that in the SolrCore.java, there's a part in the function
getSearcher where there is a comment saying:

// we are all done with the old searcher we used
// for warming...

And after that the code is:
if (currSearcherHolderF!=null) currSearcherHolderF.decref();

The problem here is that this old SolrIndexSearcher is never closed and
never removed from _searchers
What I have done:

if (currSearcherHolderF!=null){

        currSearcherHolderF.get().close(); //close SolrIndexSearcher proper
        currSearcherHolderF.decref();
        _searchers.remove(); //remove the
}

Doing that... if I do a "lsof | grep tomcat" will see that tomcat is not
holding deleted files anymore (as indexsearcher was proper close) and the
_searchers var will not accumulate infinite references...
It sorts the problem in the stats screen aswell... after 5 full-imports it
just shows one IndexSearcher
What do you think? 
-- 
View this message in context: http://www.nabble.com/Tomcat-holding-deleted-snapshots-until-it%27s-restarted---SOLVED%21%21%21-tp22451252p22500372.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Tomcat holding deleted snapshots until it's restarted - SOLVED!!!

Posted by Yonik Seeley <yo...@lucidimagination.com>.

On Fri, Mar 13, 2009 at 1:00 PM, Marc Sturlese <ma...@gmail.com> wrote:
>
> Ok, I will open a bug issue now.
>
>> Forcing it to close at the point you did is unsafe since other threads
>> may still be using that searcher.
>
> Can you give me an example where other threads would be using that searcher?

Any searches that started before the new searcher was registered will
still be using the old searcher.
  Thread A starts executing a search request with Searcher1
  Thread B issues a "commit"
      - close the writer
      - open Searcher2
      - register Searcher2 (and decrement Searcher1 ref count)
  Thread B finishes
  Thread A finishes (decrement Searcher1 ref count)

-Yonik
http://www.lucidimagination.com

Re: Tomcat holding deleted snapshots until it's restarted - SOLVED!!!

Posted by Marc Sturlese <ma...@gmail.com>.

Ok, I will open a bug issue now.

> Forcing it to close at the point you did is unsafe since other threads
> may still be using that searcher. 

Can you give me an example where other threads would be using that searcher?
(As I said I find this part of the source dufficult to understand and
imagined was missing something...)




Yonik Seeley-2 wrote:
> 
> decref() decrements the reference count and closes the searcher when
> it reaches 0 (no more users).
> Forcing it to close at the point you did is unsafe since other threads
> may still be using that searcher.
> The real issue lies somewhere else - either a stuck thread, or some
> code that is not decrementing the reference when it's done.  It's most
> likely the latter.
> 
> We need to get to the root cause.  Can you open a JIRA bug for this?
> 
> -Yonik
> http://www.lucidimagination.com
> 
> On Fri, Mar 13, 2009 at 12:39 PM, Marc Sturlese <ma...@gmail.com>
> wrote:
>>
>> Hey Yonik,
>> I tested the last nightly build and still happens... but I have solved
>> it! I
>> tell you my solution, it seems to be working well but just want to be
>> sure
>> that it doesn't have any bad effects as for me this is one of the most
>> complicated parts of the Solr source (the fact of dealing with multiple
>> indexsearchers in a syncronized way).
>> I noticed that in the SolrCore.java, there's a part in the function
>> getSearcher where there is a comment saying:
>>
>> // we are all done with the old searcher we used
>> // for warming...
>>
>> And after that the code is:
>> if (currSearcherHolderF!=null) currSearcherHolderF.decref();
>>
>> The problem here is that this old SolrIndexSearcher is never closed and
>> never removed from _searchers
>> What I have done:
>>
>> if (currSearcherHolderF!=null){
>>
>>        currSearcherHolderF.get().close(); //close SolrIndexSearcher
>> proper
>>        currSearcherHolderF.decref();
>>        _searchers.remove(); //remove the
>> }
>>
>> Doing that... if I do a "lsof | grep tomcat" will see that tomcat is not
>> holding deleted files anymore (as indexsearcher was proper close) and the
>> _searchers var will not accumulate infinite references...
>> It sorts the problem in the stats screen aswell... after 5 full-imports
>> it
>> just shows one IndexSearcher
>> What do you think?
> 
> 

-- 
View this message in context: http://www.nabble.com/Tomcat-holding-deleted-snapshots-until-it%27s-restarted---SOLVED%21%21%21-tp22451252p22500762.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Tomcat holding deleted snapshots until it's restarted - SOLVED!!!

Posted by Yonik Seeley <yo...@lucidimagination.com>.

decref() decrements the reference count and closes the searcher when
it reaches 0 (no more users).
Forcing it to close at the point you did is unsafe since other threads
may still be using that searcher.
The real issue lies somewhere else - either a stuck thread, or some
code that is not decrementing the reference when it's done.  It's most
likely the latter.

We need to get to the root cause.  Can you open a JIRA bug for this?

-Yonik
http://www.lucidimagination.com

On Fri, Mar 13, 2009 at 12:39 PM, Marc Sturlese <ma...@gmail.com> wrote:
>
> Hey Yonik,
> I tested the last nightly build and still happens... but I have solved it! I
> tell you my solution, it seems to be working well but just want to be sure
> that it doesn't have any bad effects as for me this is one of the most
> complicated parts of the Solr source (the fact of dealing with multiple
> indexsearchers in a syncronized way).
> I noticed that in the SolrCore.java, there's a part in the function
> getSearcher where there is a comment saying:
>
> // we are all done with the old searcher we used
> // for warming...
>
> And after that the code is:
> if (currSearcherHolderF!=null) currSearcherHolderF.decref();
>
> The problem here is that this old SolrIndexSearcher is never closed and
> never removed from _searchers
> What I have done:
>
> if (currSearcherHolderF!=null){
>
>        currSearcherHolderF.get().close(); //close SolrIndexSearcher proper
>        currSearcherHolderF.decref();
>        _searchers.remove(); //remove the
> }
>
> Doing that... if I do a "lsof | grep tomcat" will see that tomcat is not
> holding deleted files anymore (as indexsearcher was proper close) and the
> _searchers var will not accumulate infinite references...
> It sorts the problem in the stats screen aswell... after 5 full-imports it
> just shows one IndexSearcher
> What do you think?