You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Jesse Glick (Created) (JIRA)" <ji...@apache.org> on 2011/11/07 18:52:51 UTC

[jira] [Created] (LUCENE-3567) NPE from SegmentTermDocs. from SegmentReader.termDocs

NPE from SegmentTermDocs.<init> from SegmentReader.termDocs
-----------------------------------------------------------

                 Key: LUCENE-3567
                 URL: https://issues.apache.org/jira/browse/LUCENE-3567
             Project: Lucene - Java
          Issue Type: Bug
          Components: core/index
    Affects Versions: 3.0.3
         Environment: various OS and JRE combos: http://statistics.netbeans.org/exceptions/detail.do?id=180686
            Reporter: Jesse Glick


Occasionally NetBeans IDE users receive an NPE from Lucene 3.0.3 inside Maven Indexer (currently 4.1.2) code:

{code}
java.lang.NullPointerException
	at org.apache.lucene.index.SegmentTermDocs.<init>(SegmentTermDocs.java:52)
	at org.apache.lucene.index.SegmentReader.termDocs(SegmentReader.java:963)
	at org.apache.lucene.index.IndexReader.termDocs(IndexReader.java:813)
	at org.apache.lucene.index.SegmentReader.termDocs(SegmentReader.java:956)
	at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:74)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:210)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:170)
	at org.apache.lucene.search.Searcher.search(Searcher.java:98)
	at org.apache.lucene.search.Searcher.search(Searcher.java:108)
	at org.apache.maven.index.DefaultIndexerEngine.getOldDocument(DefaultIndexerEngine.java:158)
        ....
{code}

Working backwards, {{parent.core.getTermsReader()}} is null, which means {{SegmentReaders.CoreReaders.decRef}} was called, which means {{SegmentReader.doClose}} was called, which means {{IndexReader.doClose}} was called, which I suppose means something called {{IndexReader.decRef}} prematurely. But plenty of things can call {{IndexReader.decRef}} and it is not clear how to track down the root cause.

Note that {{SegmentReader.termDocs}} first calls {{ensureOpen()}}, which is presumably supposed to ensure that {{decRef}} had not been called too many times; perhaps this assertion did not work?

Downstream bug, for reference: https://netbeans.org/bugzilla/show_bug.cgi?id=201057

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Commented] (LUCENE-3567) NPE from SegmentTermDocs. from SegmentReader.termDocs

Posted by "Michael McCandless (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145665#comment-13145665 ] 

Michael McCandless commented on LUCENE-3567:
--------------------------------------------

Hmm try to double check all places that close/decRef the reader; most likely the IR is over-decRef'd, or...

Are there threads involved?  If so, it could be one thread closes the IR while another thread is still using it (which could explain why SR.termDocs call to ensureOpen didn't catch that the reader was closed, ie the close happened in the tiny window after ensureOpen and before STD.init called .getTermsReader()).

Lucene 3.5.0 will have the SearcherManager class to handle correctly sharing a searcher across threads with reopens... probably back-porting to work on 3.0.x wouldn't be so bad?  I described it here: http://blog.mikemccandless.com/2011/09/lucenes-searchermanager-simplifies.html
                
> NPE from SegmentTermDocs.<init> from SegmentReader.termDocs
> -----------------------------------------------------------
>
>                 Key: LUCENE-3567
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3567
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 3.0.3
>         Environment: various OS and JRE combos: http://statistics.netbeans.org/exceptions/detail.do?id=180686
>            Reporter: Jesse Glick
>
> Occasionally NetBeans IDE users receive an NPE from Lucene 3.0.3 inside Maven Indexer (currently 4.1.2) code:
> {code}
> java.lang.NullPointerException
> 	at org.apache.lucene.index.SegmentTermDocs.<init>(SegmentTermDocs.java:52)
> 	at org.apache.lucene.index.SegmentReader.termDocs(SegmentReader.java:963)
> 	at org.apache.lucene.index.IndexReader.termDocs(IndexReader.java:813)
> 	at org.apache.lucene.index.SegmentReader.termDocs(SegmentReader.java:956)
> 	at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:74)
> 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:210)
> 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:170)
> 	at org.apache.lucene.search.Searcher.search(Searcher.java:98)
> 	at org.apache.lucene.search.Searcher.search(Searcher.java:108)
> 	at org.apache.maven.index.DefaultIndexerEngine.getOldDocument(DefaultIndexerEngine.java:158)
>         ....
> {code}
> Working backwards, {{parent.core.getTermsReader()}} is null, which means {{SegmentReaders.CoreReaders.decRef}} was called, which means {{SegmentReader.doClose}} was called, which means {{IndexReader.doClose}} was called, which I suppose means something called {{IndexReader.decRef}} prematurely. But plenty of things can call {{IndexReader.decRef}} and it is not clear how to track down the root cause.
> Note that {{SegmentReader.termDocs}} first calls {{ensureOpen()}}, which is presumably supposed to ensure that {{decRef}} had not been called too many times; perhaps this assertion did not work?
> Downstream bug, for reference: https://netbeans.org/bugzilla/show_bug.cgi?id=201057

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] [Commented] (LUCENE-3567) NPE from SegmentTermDocs. from SegmentReader.termDocs

Posted by "Jesse Glick (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150624#comment-13150624 ] 

Jesse Glick commented on LUCENE-3567:
-------------------------------------

Various application threads could call Maven Indexer methods but they should in all cases be using an exclusive lock to prevent race conditions. I do not believe Maven Indexer itself would be accessing Lucene from multiple threads. In other words, the application is not _intentionally_ using multithreaded index access. It is possible that it is doing so accidentally. Unfortunately the low-level {{NullPointerException}} produced deep in the Lucene stack does not indicate whether this or something else is the problem, nor does the stack trace point toward a clear diagnosis since the searcher might have been rendered invalid earlier. It would be nice to have more thorough assertions in Lucene code, for example verifying in entry points like {{Searcher.search}} that all related objects are in a valid state, or even tracking which {{Thread}} is actively mutating state so that definitely inappropriate calls fail early and reproducibly. Assuming Java {{assert}} is used, this would add no runtime overhead for production code.
                
> NPE from SegmentTermDocs.<init> from SegmentReader.termDocs
> -----------------------------------------------------------
>
>                 Key: LUCENE-3567
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3567
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 3.0.3
>         Environment: various OS and JRE combos: http://statistics.netbeans.org/exceptions/detail.do?id=180686
>            Reporter: Jesse Glick
>
> Occasionally NetBeans IDE users receive an NPE from Lucene 3.0.3 inside Maven Indexer (currently 4.1.2) code:
> {code}
> java.lang.NullPointerException
> 	at org.apache.lucene.index.SegmentTermDocs.<init>(SegmentTermDocs.java:52)
> 	at org.apache.lucene.index.SegmentReader.termDocs(SegmentReader.java:963)
> 	at org.apache.lucene.index.IndexReader.termDocs(IndexReader.java:813)
> 	at org.apache.lucene.index.SegmentReader.termDocs(SegmentReader.java:956)
> 	at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:74)
> 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:210)
> 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:170)
> 	at org.apache.lucene.search.Searcher.search(Searcher.java:98)
> 	at org.apache.lucene.search.Searcher.search(Searcher.java:108)
> 	at org.apache.maven.index.DefaultIndexerEngine.getOldDocument(DefaultIndexerEngine.java:158)
>         ....
> {code}
> Working backwards, {{parent.core.getTermsReader()}} is null, which means {{SegmentReaders.CoreReaders.decRef}} was called, which means {{SegmentReader.doClose}} was called, which means {{IndexReader.doClose}} was called, which I suppose means something called {{IndexReader.decRef}} prematurely. But plenty of things can call {{IndexReader.decRef}} and it is not clear how to track down the root cause.
> Note that {{SegmentReader.termDocs}} first calls {{ensureOpen()}}, which is presumably supposed to ensure that {{decRef}} had not been called too many times; perhaps this assertion did not work?
> Downstream bug, for reference: https://netbeans.org/bugzilla/show_bug.cgi?id=201057

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org