You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Jason Rutherglen <ja...@gmail.com> on 2009/12/09 01:34:07 UTC
NearSpansUnordered payloads not returning all the time
Howdy,
I am wondering if anyone has seen
NearSpansUnordered.getPayload() not return payloads that are
verifiably accessible via IR.termPositions? It's a bit confusing
because most of the time they're returned properly.
I suspect the payload logic gets tripped up in
NearSpansUnordered. I'll put together a test case, however the
difficulty is that we're only seeing the issue with largish 800
MB indexes, which could make the test case a little crazy.
Jason
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: NearSpansUnordered payloads not returning all the time
Posted by Michael McCandless <lu...@mikemccandless.com>.
Yes, you found it! Is that what you're hitting?
I don't know of a workaround though... this is just how SpanQuery
currently works...
Mike
On Wed, Dec 9, 2009 at 4:56 PM, Jason Rutherglen
<ja...@gmail.com> wrote:
> Mike,
>
> Is this the thread?
>
> http://www.lucidimagination.com/search/document/1e87d488a904b89f/spannearquery_s_spans_payloads#8103efdc9705a763
>
> Maybe we need a recommended workaround for this?
>
> Jason
>
> On Wed, Dec 9, 2009 at 1:17 PM, Michael McCandless
> <lu...@mikemccandless.com> wrote:
>> That sounds familiar... try to track down the last thread maybe?
>>
>> I think it was this: if the payload was already retrieved for a prior
>> span then the current span won't be able to retrieve it, so even
>> though you know a payload falls within the span you're looking at, you
>> won't get it back, if it already fell on a prior span.
>>
>> Mike
>>
>> On Wed, Dec 9, 2009 at 11:25 AM, Jason Rutherglen
>> <ja...@gmail.com> wrote:
>>> Right we're getting the spans, however it's just the payloads that are
>>> missing, randomly...
>>>
>>> On Wed, Dec 9, 2009 at 2:23 AM, Michael McCandless
>>> <lu...@mikemccandless.com> wrote:
>>>> There was a thread a while back about how span queries don't enumerate
>>>> every possible span, but I can't remember if that included sometimes
>>>> missing payloads...
>>>>
>>>> Mike
>>>>
>>>> On Tue, Dec 8, 2009 at 7:34 PM, Jason Rutherglen
>>>> <ja...@gmail.com> wrote:
>>>>> Howdy,
>>>>>
>>>>> I am wondering if anyone has seen
>>>>> NearSpansUnordered.getPayload() not return payloads that are
>>>>> verifiably accessible via IR.termPositions? It's a bit confusing
>>>>> because most of the time they're returned properly.
>>>>>
>>>>> I suspect the payload logic gets tripped up in
>>>>> NearSpansUnordered. I'll put together a test case, however the
>>>>> difficulty is that we're only seeing the issue with largish 800
>>>>> MB indexes, which could make the test case a little crazy.
>>>>>
>>>>> Jason
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: NearSpansUnordered payloads not returning all the time
Posted by Jason Rutherglen <ja...@gmail.com>.
Mike,
Is this the thread?
http://www.lucidimagination.com/search/document/1e87d488a904b89f/spannearquery_s_spans_payloads#8103efdc9705a763
Maybe we need a recommended workaround for this?
Jason
On Wed, Dec 9, 2009 at 1:17 PM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> That sounds familiar... try to track down the last thread maybe?
>
> I think it was this: if the payload was already retrieved for a prior
> span then the current span won't be able to retrieve it, so even
> though you know a payload falls within the span you're looking at, you
> won't get it back, if it already fell on a prior span.
>
> Mike
>
> On Wed, Dec 9, 2009 at 11:25 AM, Jason Rutherglen
> <ja...@gmail.com> wrote:
>> Right we're getting the spans, however it's just the payloads that are
>> missing, randomly...
>>
>> On Wed, Dec 9, 2009 at 2:23 AM, Michael McCandless
>> <lu...@mikemccandless.com> wrote:
>>> There was a thread a while back about how span queries don't enumerate
>>> every possible span, but I can't remember if that included sometimes
>>> missing payloads...
>>>
>>> Mike
>>>
>>> On Tue, Dec 8, 2009 at 7:34 PM, Jason Rutherglen
>>> <ja...@gmail.com> wrote:
>>>> Howdy,
>>>>
>>>> I am wondering if anyone has seen
>>>> NearSpansUnordered.getPayload() not return payloads that are
>>>> verifiably accessible via IR.termPositions? It's a bit confusing
>>>> because most of the time they're returned properly.
>>>>
>>>> I suspect the payload logic gets tripped up in
>>>> NearSpansUnordered. I'll put together a test case, however the
>>>> difficulty is that we're only seeing the issue with largish 800
>>>> MB indexes, which could make the test case a little crazy.
>>>>
>>>> Jason
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: NearSpansUnordered payloads not returning all the time
Posted by Michael McCandless <lu...@mikemccandless.com>.
That sounds familiar... try to track down the last thread maybe?
I think it was this: if the payload was already retrieved for a prior
span then the current span won't be able to retrieve it, so even
though you know a payload falls within the span you're looking at, you
won't get it back, if it already fell on a prior span.
Mike
On Wed, Dec 9, 2009 at 11:25 AM, Jason Rutherglen
<ja...@gmail.com> wrote:
> Right we're getting the spans, however it's just the payloads that are
> missing, randomly...
>
> On Wed, Dec 9, 2009 at 2:23 AM, Michael McCandless
> <lu...@mikemccandless.com> wrote:
>> There was a thread a while back about how span queries don't enumerate
>> every possible span, but I can't remember if that included sometimes
>> missing payloads...
>>
>> Mike
>>
>> On Tue, Dec 8, 2009 at 7:34 PM, Jason Rutherglen
>> <ja...@gmail.com> wrote:
>>> Howdy,
>>>
>>> I am wondering if anyone has seen
>>> NearSpansUnordered.getPayload() not return payloads that are
>>> verifiably accessible via IR.termPositions? It's a bit confusing
>>> because most of the time they're returned properly.
>>>
>>> I suspect the payload logic gets tripped up in
>>> NearSpansUnordered. I'll put together a test case, however the
>>> difficulty is that we're only seeing the issue with largish 800
>>> MB indexes, which could make the test case a little crazy.
>>>
>>> Jason
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: NearSpansUnordered payloads not returning all the time
Posted by Jason Rutherglen <ja...@gmail.com>.
Right we're getting the spans, however it's just the payloads that are
missing, randomly...
On Wed, Dec 9, 2009 at 2:23 AM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> There was a thread a while back about how span queries don't enumerate
> every possible span, but I can't remember if that included sometimes
> missing payloads...
>
> Mike
>
> On Tue, Dec 8, 2009 at 7:34 PM, Jason Rutherglen
> <ja...@gmail.com> wrote:
>> Howdy,
>>
>> I am wondering if anyone has seen
>> NearSpansUnordered.getPayload() not return payloads that are
>> verifiably accessible via IR.termPositions? It's a bit confusing
>> because most of the time they're returned properly.
>>
>> I suspect the payload logic gets tripped up in
>> NearSpansUnordered. I'll put together a test case, however the
>> difficulty is that we're only seeing the issue with largish 800
>> MB indexes, which could make the test case a little crazy.
>>
>> Jason
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: NearSpansUnordered payloads not returning all the time
Posted by Michael McCandless <lu...@mikemccandless.com>.
There was a thread a while back about how span queries don't enumerate
every possible span, but I can't remember if that included sometimes
missing payloads...
Mike
On Tue, Dec 8, 2009 at 7:34 PM, Jason Rutherglen
<ja...@gmail.com> wrote:
> Howdy,
>
> I am wondering if anyone has seen
> NearSpansUnordered.getPayload() not return payloads that are
> verifiably accessible via IR.termPositions? It's a bit confusing
> because most of the time they're returned properly.
>
> I suspect the payload logic gets tripped up in
> NearSpansUnordered. I'll put together a test case, however the
> difficulty is that we're only seeing the issue with largish 800
> MB indexes, which could make the test case a little crazy.
>
> Jason
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org