You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Peter Keegan <pe...@gmail.com> on 2009/10/15 19:17:44 UTC
NPE in NearSpansUnordered
I'm using Lucene 2.9 and sometimes get a NPE in NearSpansUnordered:
<java.lang.NullPointerException>java.lang.NullPointerException
at
org.apache.lucene.search.spans.NearSpansUnordered.start(NearSpansUnordered.java:219)
at
org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.processPayloads(PayloadNearQuery.java:201)
at
org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.getPayloads(PayloadNearQuery.java:180)
at
org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.getPayloads(PayloadNearQuery.java:183)
at
org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.setFreqCurrentDoc(PayloadNearQuery.java:214)
at org.apache.lucene.search.spans.SpanScorer.nextDoc(SpanScorer.java:64)
at org.apache.lucene.search.Scorer.score(Scorer.java:74)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:247)
at org.apache.lucene.search.Searcher.search(Searcher.java:152)
The CellQueue pq is empty when this occurs. Are there any conditions in
which the queue might be expected to be empty?
Peter
Re: NPE in NearSpansUnordered
Posted by Peter Keegan <pe...@gmail.com>.
I can reproduce this with a unit test - will post to JIRA shortly.
Peter
On Fri, Oct 16, 2009 at 8:06 AM, Peter Keegan <pe...@gmail.com>wrote:
> next() is called in PayloadNearQuery->setFreqCurrentDoc:
> super.setFreqCurrentDoc();
> But, I think it should be called before 'getPayloads'. That doesn't fix the
> NPE, though.
> The empty PQ occurs in the outermost span in the query, and seems to fail
> on the last document it scores.
> Prior to 2.9, I'd been using a patch on 2.3.2. and never saw the NPE.
>
> Peter
>
>
> On Fri, Oct 16, 2009 at 6:53 AM, Grant Ingersoll <gs...@apache.org>wrote:
>
>> And, you don't get this on 2.4.1?
>>
>> Are you sure you've called next()? Is it by chance on the first document
>> it tries to score that it fails?
>>
>> -Grant
>>
>>
>>
>>
>> On Oct 15, 2009, at 1:28 PM, Peter Keegan wrote:
>>
>> The query is:
>>> +payloadNear([spanNear([contents:insurance, contents:agent], 10000,
>>> false),
>>> spanNear([contents:winston, contents:salem], 10000, false)], 10, false)
>>> It's using the default payload function scorer (average value)
>>>
>>> It doesn't happen on all queries of this type, only a handful. This is
>>> production-like index of > 1million docs. I haven't been able to
>>> reproduce
>>> with a simple index, unfortunately.
>>>
>>> Peter
>>>
>>> On Thu, Oct 15, 2009 at 1:21 PM, Yonik Seeley <
>>> yonik@lucidimagination.com>wrote:
>>>
>>> Are you using any custom query types? Anything to help us reproduce
>>>> (like the acutal query this happened on) would be greatly appreciated.
>>>>
>>>> -Yonik
>>>> http://www.lucidimagination.com
>>>>
>>>>
>>>> On Thu, Oct 15, 2009 at 1:17 PM, Peter Keegan <pe...@gmail.com>
>>>> wrote:
>>>>
>>>>> I'm using Lucene 2.9 and sometimes get a NPE in NearSpansUnordered:
>>>>>
>>>>> <java.lang.NullPointerException>java.lang.NullPointerException
>>>>> at
>>>>>
>>>>> org.apache.lucene.search.spans.NearSpansUnordered.start(NearSpansUnordered.java:219)
>>>>
>>>>> at
>>>>>
>>>>> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.processPayloads(PayloadNearQuery.java:201)
>>>>
>>>>> at
>>>>>
>>>>> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.getPayloads(PayloadNearQuery.java:180)
>>>>
>>>>> at
>>>>>
>>>>> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.getPayloads(PayloadNearQuery.java:183)
>>>>
>>>>> at
>>>>>
>>>>> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.setFreqCurrentDoc(PayloadNearQuery.java:214)
>>>>
>>>>> at
>>>>> org.apache.lucene.search.spans.SpanScorer.nextDoc(SpanScorer.java:64)
>>>>> at org.apache.lucene.search.Scorer.score(Scorer.java:74)
>>>>> at
>>>>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:247)
>>>>> at org.apache.lucene.search.Searcher.search(Searcher.java:152)
>>>>>
>>>>> The CellQueue pq is empty when this occurs. Are there any conditions in
>>>>> which the queue might be expected to be empty?
>>>>>
>>>>> Peter
>>>>>
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>>
>> --------------------------
>> Grant Ingersoll
>> http://www.lucidimagination.com/
>>
>> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using
>> Solr/Lucene:
>> http://www.lucidimagination.com/search
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
Re: NPE in NearSpansUnordered
Posted by Peter Keegan <pe...@gmail.com>.
next() is called in PayloadNearQuery->setFreqCurrentDoc:
super.setFreqCurrentDoc();
But, I think it should be called before 'getPayloads'. That doesn't fix the
NPE, though.
The empty PQ occurs in the outermost span in the query, and seems to fail on
the last document it scores.
Prior to 2.9, I'd been using a patch on 2.3.2. and never saw the NPE.
Peter
On Fri, Oct 16, 2009 at 6:53 AM, Grant Ingersoll <gs...@apache.org>wrote:
> And, you don't get this on 2.4.1?
>
> Are you sure you've called next()? Is it by chance on the first document
> it tries to score that it fails?
>
> -Grant
>
>
>
>
> On Oct 15, 2009, at 1:28 PM, Peter Keegan wrote:
>
> The query is:
>> +payloadNear([spanNear([contents:insurance, contents:agent], 10000,
>> false),
>> spanNear([contents:winston, contents:salem], 10000, false)], 10, false)
>> It's using the default payload function scorer (average value)
>>
>> It doesn't happen on all queries of this type, only a handful. This is
>> production-like index of > 1million docs. I haven't been able to reproduce
>> with a simple index, unfortunately.
>>
>> Peter
>>
>> On Thu, Oct 15, 2009 at 1:21 PM, Yonik Seeley <yonik@lucidimagination.com
>> >wrote:
>>
>> Are you using any custom query types? Anything to help us reproduce
>>> (like the acutal query this happened on) would be greatly appreciated.
>>>
>>> -Yonik
>>> http://www.lucidimagination.com
>>>
>>>
>>> On Thu, Oct 15, 2009 at 1:17 PM, Peter Keegan <pe...@gmail.com>
>>> wrote:
>>>
>>>> I'm using Lucene 2.9 and sometimes get a NPE in NearSpansUnordered:
>>>>
>>>> <java.lang.NullPointerException>java.lang.NullPointerException
>>>> at
>>>>
>>>> org.apache.lucene.search.spans.NearSpansUnordered.start(NearSpansUnordered.java:219)
>>>
>>>> at
>>>>
>>>> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.processPayloads(PayloadNearQuery.java:201)
>>>
>>>> at
>>>>
>>>> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.getPayloads(PayloadNearQuery.java:180)
>>>
>>>> at
>>>>
>>>> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.getPayloads(PayloadNearQuery.java:183)
>>>
>>>> at
>>>>
>>>> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.setFreqCurrentDoc(PayloadNearQuery.java:214)
>>>
>>>> at org.apache.lucene.search.spans.SpanScorer.nextDoc(SpanScorer.java:64)
>>>> at org.apache.lucene.search.Scorer.score(Scorer.java:74)
>>>> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:247)
>>>> at org.apache.lucene.search.Searcher.search(Searcher.java:152)
>>>>
>>>> The CellQueue pq is empty when this occurs. Are there any conditions in
>>>> which the queue might be expected to be empty?
>>>>
>>>> Peter
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using
> Solr/Lucene:
> http://www.lucidimagination.com/search
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: NPE in NearSpansUnordered
Posted by Grant Ingersoll <gs...@apache.org>.
And, you don't get this on 2.4.1?
Are you sure you've called next()? Is it by chance on the first
document it tries to score that it fails?
-Grant
On Oct 15, 2009, at 1:28 PM, Peter Keegan wrote:
> The query is:
> +payloadNear([spanNear([contents:insurance, contents:agent], 10000,
> false),
> spanNear([contents:winston, contents:salem], 10000, false)], 10,
> false)
> It's using the default payload function scorer (average value)
>
> It doesn't happen on all queries of this type, only a handful. This is
> production-like index of > 1million docs. I haven't been able to
> reproduce
> with a simple index, unfortunately.
>
> Peter
>
> On Thu, Oct 15, 2009 at 1:21 PM, Yonik Seeley <yonik@lucidimagination.com
> >wrote:
>
>> Are you using any custom query types? Anything to help us reproduce
>> (like the acutal query this happened on) would be greatly
>> appreciated.
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>>
>> On Thu, Oct 15, 2009 at 1:17 PM, Peter Keegan
>> <pe...@gmail.com>
>> wrote:
>>> I'm using Lucene 2.9 and sometimes get a NPE in NearSpansUnordered:
>>>
>>> <java.lang.NullPointerException>java.lang.NullPointerException
>>> at
>>>
>> org.apache.lucene.search.spans.NearSpansUnordered.start
>> (NearSpansUnordered.java:219)
>>> at
>>>
>> org.apache.lucene.search.payloads.PayloadNearQuery
>> $PayloadNearSpanScorer.processPayloads(PayloadNearQuery.java:201)
>>> at
>>>
>> org.apache.lucene.search.payloads.PayloadNearQuery
>> $PayloadNearSpanScorer.getPayloads(PayloadNearQuery.java:180)
>>> at
>>>
>> org.apache.lucene.search.payloads.PayloadNearQuery
>> $PayloadNearSpanScorer.getPayloads(PayloadNearQuery.java:183)
>>> at
>>>
>> org.apache.lucene.search.payloads.PayloadNearQuery
>> $PayloadNearSpanScorer.setFreqCurrentDoc(PayloadNearQuery.java:214)
>>> at org.apache.lucene.search.spans.SpanScorer.nextDoc
>>> (SpanScorer.java:64)
>>> at org.apache.lucene.search.Scorer.score(Scorer.java:74)
>>> at org.apache.lucene.search.IndexSearcher.search
>>> (IndexSearcher.java:247)
>>> at org.apache.lucene.search.Searcher.search(Searcher.java:152)
>>>
>>> The CellQueue pq is empty when this occurs. Are there any
>>> conditions in
>>> which the queue might be expected to be empty?
>>>
>>> Peter
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: NPE in NearSpansUnordered
Posted by Peter Keegan <pe...@gmail.com>.
The query is:
+payloadNear([spanNear([contents:insurance, contents:agent], 10000, false),
spanNear([contents:winston, contents:salem], 10000, false)], 10, false)
It's using the default payload function scorer (average value)
It doesn't happen on all queries of this type, only a handful. This is
production-like index of > 1million docs. I haven't been able to reproduce
with a simple index, unfortunately.
Peter
On Thu, Oct 15, 2009 at 1:21 PM, Yonik Seeley <yo...@lucidimagination.com>wrote:
> Are you using any custom query types? Anything to help us reproduce
> (like the acutal query this happened on) would be greatly appreciated.
>
> -Yonik
> http://www.lucidimagination.com
>
>
> On Thu, Oct 15, 2009 at 1:17 PM, Peter Keegan <pe...@gmail.com>
> wrote:
> > I'm using Lucene 2.9 and sometimes get a NPE in NearSpansUnordered:
> >
> > <java.lang.NullPointerException>java.lang.NullPointerException
> > at
> >
> org.apache.lucene.search.spans.NearSpansUnordered.start(NearSpansUnordered.java:219)
> > at
> >
> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.processPayloads(PayloadNearQuery.java:201)
> > at
> >
> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.getPayloads(PayloadNearQuery.java:180)
> > at
> >
> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.getPayloads(PayloadNearQuery.java:183)
> > at
> >
> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.setFreqCurrentDoc(PayloadNearQuery.java:214)
> > at org.apache.lucene.search.spans.SpanScorer.nextDoc(SpanScorer.java:64)
> > at org.apache.lucene.search.Scorer.score(Scorer.java:74)
> > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:247)
> > at org.apache.lucene.search.Searcher.search(Searcher.java:152)
> >
> > The CellQueue pq is empty when this occurs. Are there any conditions in
> > which the queue might be expected to be empty?
> >
> > Peter
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: NPE in NearSpansUnordered
Posted by Yonik Seeley <yo...@lucidimagination.com>.
Are you using any custom query types? Anything to help us reproduce
(like the acutal query this happened on) would be greatly appreciated.
-Yonik
http://www.lucidimagination.com
On Thu, Oct 15, 2009 at 1:17 PM, Peter Keegan <pe...@gmail.com> wrote:
> I'm using Lucene 2.9 and sometimes get a NPE in NearSpansUnordered:
>
> <java.lang.NullPointerException>java.lang.NullPointerException
> at
> org.apache.lucene.search.spans.NearSpansUnordered.start(NearSpansUnordered.java:219)
> at
> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.processPayloads(PayloadNearQuery.java:201)
> at
> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.getPayloads(PayloadNearQuery.java:180)
> at
> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.getPayloads(PayloadNearQuery.java:183)
> at
> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.setFreqCurrentDoc(PayloadNearQuery.java:214)
> at org.apache.lucene.search.spans.SpanScorer.nextDoc(SpanScorer.java:64)
> at org.apache.lucene.search.Scorer.score(Scorer.java:74)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:247)
> at org.apache.lucene.search.Searcher.search(Searcher.java:152)
>
> The CellQueue pq is empty when this occurs. Are there any conditions in
> which the queue might be expected to be empty?
>
> Peter
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org