You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Mark Miller <ma...@gmail.com> on 2009/04/24 00:48:16 UTC
Lucene 1483 and Auto resolution
Just got off the train and ny to ct has a brilliant bar car, so lest I
forget:
1483 moved auto resolution from fshq to indexsearcher - which is a back
compat break if you were using a fshq without indexsearcher (Solr does
it - anyone could). Annoying. If I remember right, I did it to resolve
auto on the multireader rather than each individual segment reader. So
the change is needed and not allowed. Perhaps it could just re-resolve
like before though - if indexsearcher has already resolved, fine,
otherwise it will be done again at the fshq level. Ill issue it up later.
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene 1483 and Auto resolution
Posted by Mark Miller <ma...@gmail.com>.
Will do.
Michael McCandless wrote:
> OK that looks right.
>
> Though, maybe you should add javadocs to FieldValueHitQueue saying it
> does *not* do AUTO resolution? (Since we've deprecated FSHQ in favor
> of FVHQ, I think we should call out this difference?). And state the
> workaround (calling FVHQ.detectFieldType yourself)?
>
> Mike
>
> On Fri, Apr 24, 2009 at 8:25 AM, Mark Miller <ma...@gmail.com> wrote:
>
>> I'm just putting the auto resolution back in fshq and there is no double
>> check - foolish me. As I first said, we will either resolve first in back
>> compat mode and it will be skipped in fshq, or it will be resolved in fshq
>> for anyone using it externally (and not resolving first on there own). I'll
>> commit the change shortly.
>>
>> Mark Miller wrote:
>>
>>> Eh - we have to spin through all the fields to check for legacy first
>>> anyway. Just doing it twice in legacy mode is probably best? I think there
>>> is no way to avoid spinning through the fields twice in any case (you have
>>> to spin through the sort fields to know you dont want to spin through the
>>> sort fields), so I guess I just go with that.
>>>
>>> - Mark
>>>
>>> Mark Miller wrote:
>>>
>>>> Ah, right - thats basically what I was suggesting, though I was thinking
>>>> that we might need to resolve twice, but of course your right and we don't
>>>> have to. Lucene does still use fshq for back compat (for custom field source
>>>> I think?), but I can just do the auto resolution in IndexSearcher only in
>>>> non legacy mode and restore auto resolution to fshq. That will avoid the
>>>> harmless but wasteful double resolve in legacy mode. Had no code to look at
>>>> when I sent that out.
>>>>
>>>> - Mark
>>>>
>>>> Michael McCandless wrote:
>>>>
>>>>> Urgh, right. Can't we simply restore the AUTO resolution into it?
>>>>> Existing (direct) usage of it must be passing in the top-level
>>>>> IndexReader. (Lucene doesn't use FSHQ internally).
>>>>>
>>>>> Mike
>>>>>
>>>>> On Thu, Apr 23, 2009 at 6:48 PM, Mark Miller <ma...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>>> Just got off the train and ny to ct has a brilliant bar car, so lest I
>>>>>> forget:
>>>>>>
>>>>>> 1483 moved auto resolution from fshq to indexsearcher - which is a back
>>>>>> compat break if you were using a fshq without indexsearcher (Solr does
>>>>>> it -
>>>>>> anyone could). Annoying. If I remember right, I did it to resolve auto
>>>>>> on
>>>>>> the multireader rather than each individual segment reader. So the
>>>>>> change is
>>>>>> needed and not allowed. Perhaps it could just re-resolve like before
>>>>>> though
>>>>>> - if indexsearcher has already resolved, fine, otherwise it will be
>>>>>> done
>>>>>> again at the fshq level. Ill issue it up later.
>>>>>>
>>>>>> --
>>>>>> - Mark
>>>>>>
>>>>>> http://www.lucidimagination.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>>
>>>>>
>>>>>
>>>>
>>>
>> --
>> - Mark
>>
>> http://www.lucidimagination.com
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene 1483 and Auto resolution
Posted by Michael McCandless <lu...@mikemccandless.com>.
OK that looks right.
Though, maybe you should add javadocs to FieldValueHitQueue saying it
does *not* do AUTO resolution? (Since we've deprecated FSHQ in favor
of FVHQ, I think we should call out this difference?). And state the
workaround (calling FVHQ.detectFieldType yourself)?
Mike
On Fri, Apr 24, 2009 at 8:25 AM, Mark Miller <ma...@gmail.com> wrote:
> I'm just putting the auto resolution back in fshq and there is no double
> check - foolish me. As I first said, we will either resolve first in back
> compat mode and it will be skipped in fshq, or it will be resolved in fshq
> for anyone using it externally (and not resolving first on there own). I'll
> commit the change shortly.
>
> Mark Miller wrote:
>>
>> Eh - we have to spin through all the fields to check for legacy first
>> anyway. Just doing it twice in legacy mode is probably best? I think there
>> is no way to avoid spinning through the fields twice in any case (you have
>> to spin through the sort fields to know you dont want to spin through the
>> sort fields), so I guess I just go with that.
>>
>> - Mark
>>
>> Mark Miller wrote:
>>>
>>> Ah, right - thats basically what I was suggesting, though I was thinking
>>> that we might need to resolve twice, but of course your right and we don't
>>> have to. Lucene does still use fshq for back compat (for custom field source
>>> I think?), but I can just do the auto resolution in IndexSearcher only in
>>> non legacy mode and restore auto resolution to fshq. That will avoid the
>>> harmless but wasteful double resolve in legacy mode. Had no code to look at
>>> when I sent that out.
>>>
>>> - Mark
>>>
>>> Michael McCandless wrote:
>>>>
>>>> Urgh, right. Can't we simply restore the AUTO resolution into it?
>>>> Existing (direct) usage of it must be passing in the top-level
>>>> IndexReader. (Lucene doesn't use FSHQ internally).
>>>>
>>>> Mike
>>>>
>>>> On Thu, Apr 23, 2009 at 6:48 PM, Mark Miller <ma...@gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> Just got off the train and ny to ct has a brilliant bar car, so lest I
>>>>> forget:
>>>>>
>>>>> 1483 moved auto resolution from fshq to indexsearcher - which is a back
>>>>> compat break if you were using a fshq without indexsearcher (Solr does
>>>>> it -
>>>>> anyone could). Annoying. If I remember right, I did it to resolve auto
>>>>> on
>>>>> the multireader rather than each individual segment reader. So the
>>>>> change is
>>>>> needed and not allowed. Perhaps it could just re-resolve like before
>>>>> though
>>>>> - if indexsearcher has already resolved, fine, otherwise it will be
>>>>> done
>>>>> again at the fshq level. Ill issue it up later.
>>>>>
>>>>> --
>>>>> - Mark
>>>>>
>>>>> http://www.lucidimagination.com
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>>
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>
>>>>
>>>
>>>
>>
>>
>
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene 1483 and Auto resolution
Posted by Mark Miller <ma...@gmail.com>.
I'm just putting the auto resolution back in fshq and there is no double
check - foolish me. As I first said, we will either resolve first in
back compat mode and it will be skipped in fshq, or it will be resolved
in fshq for anyone using it externally (and not resolving first on there
own). I'll commit the change shortly.
Mark Miller wrote:
> Eh - we have to spin through all the fields to check for legacy first
> anyway. Just doing it twice in legacy mode is probably best? I think
> there is no way to avoid spinning through the fields twice in any case
> (you have to spin through the sort fields to know you dont want to
> spin through the sort fields), so I guess I just go with that.
>
> - Mark
>
> Mark Miller wrote:
>> Ah, right - thats basically what I was suggesting, though I was
>> thinking that we might need to resolve twice, but of course your
>> right and we don't have to. Lucene does still use fshq for back
>> compat (for custom field source I think?), but I can just do the auto
>> resolution in IndexSearcher only in non legacy mode and restore auto
>> resolution to fshq. That will avoid the harmless but wasteful double
>> resolve in legacy mode. Had no code to look at when I sent that out.
>>
>> - Mark
>>
>> Michael McCandless wrote:
>>> Urgh, right. Can't we simply restore the AUTO resolution into it?
>>> Existing (direct) usage of it must be passing in the top-level
>>> IndexReader. (Lucene doesn't use FSHQ internally).
>>>
>>> Mike
>>>
>>> On Thu, Apr 23, 2009 at 6:48 PM, Mark Miller <ma...@gmail.com>
>>> wrote:
>>>
>>>> Just got off the train and ny to ct has a brilliant bar car, so lest I
>>>> forget:
>>>>
>>>> 1483 moved auto resolution from fshq to indexsearcher - which is a
>>>> back
>>>> compat break if you were using a fshq without indexsearcher (Solr
>>>> does it -
>>>> anyone could). Annoying. If I remember right, I did it to resolve
>>>> auto on
>>>> the multireader rather than each individual segment reader. So the
>>>> change is
>>>> needed and not allowed. Perhaps it could just re-resolve like
>>>> before though
>>>> - if indexsearcher has already resolved, fine, otherwise it will be
>>>> done
>>>> again at the fshq level. Ill issue it up later.
>>>>
>>>> --
>>>> - Mark
>>>>
>>>> http://www.lucidimagination.com
>>>>
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>>
>>
>>
>
>
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene 1483 and Auto resolution
Posted by Mark Miller <ma...@gmail.com>.
Eh - we have to spin through all the fields to check for legacy first
anyway. Just doing it twice in legacy mode is probably best? I think
there is no way to avoid spinning through the fields twice in any case
(you have to spin through the sort fields to know you dont want to spin
through the sort fields), so I guess I just go with that.
- Mark
Mark Miller wrote:
> Ah, right - thats basically what I was suggesting, though I was
> thinking that we might need to resolve twice, but of course your right
> and we don't have to. Lucene does still use fshq for back compat (for
> custom field source I think?), but I can just do the auto resolution
> in IndexSearcher only in non legacy mode and restore auto resolution
> to fshq. That will avoid the harmless but wasteful double resolve in
> legacy mode. Had no code to look at when I sent that out.
>
> - Mark
>
> Michael McCandless wrote:
>> Urgh, right. Can't we simply restore the AUTO resolution into it?
>> Existing (direct) usage of it must be passing in the top-level
>> IndexReader. (Lucene doesn't use FSHQ internally).
>>
>> Mike
>>
>> On Thu, Apr 23, 2009 at 6:48 PM, Mark Miller <ma...@gmail.com>
>> wrote:
>>
>>> Just got off the train and ny to ct has a brilliant bar car, so lest I
>>> forget:
>>>
>>> 1483 moved auto resolution from fshq to indexsearcher - which is a back
>>> compat break if you were using a fshq without indexsearcher (Solr
>>> does it -
>>> anyone could). Annoying. If I remember right, I did it to resolve
>>> auto on
>>> the multireader rather than each individual segment reader. So the
>>> change is
>>> needed and not allowed. Perhaps it could just re-resolve like before
>>> though
>>> - if indexsearcher has already resolved, fine, otherwise it will be
>>> done
>>> again at the fshq level. Ill issue it up later.
>>>
>>> --
>>> - Mark
>>>
>>> http://www.lucidimagination.com
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>
>
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene 1483 and Auto resolution
Posted by Mark Miller <ma...@gmail.com>.
Ah, right - thats basically what I was suggesting, though I was thinking
that we might need to resolve twice, but of course your right and we
don't have to. Lucene does still use fshq for back compat (for custom
field source I think?), but I can just do the auto resolution in
IndexSearcher only in non legacy mode and restore auto resolution to
fshq. That will avoid the harmless but wasteful double resolve in legacy
mode. Had no code to look at when I sent that out.
- Mark
Michael McCandless wrote:
> Urgh, right. Can't we simply restore the AUTO resolution into it?
> Existing (direct) usage of it must be passing in the top-level
> IndexReader. (Lucene doesn't use FSHQ internally).
>
> Mike
>
> On Thu, Apr 23, 2009 at 6:48 PM, Mark Miller <ma...@gmail.com> wrote:
>
>> Just got off the train and ny to ct has a brilliant bar car, so lest I
>> forget:
>>
>> 1483 moved auto resolution from fshq to indexsearcher - which is a back
>> compat break if you were using a fshq without indexsearcher (Solr does it -
>> anyone could). Annoying. If I remember right, I did it to resolve auto on
>> the multireader rather than each individual segment reader. So the change is
>> needed and not allowed. Perhaps it could just re-resolve like before though
>> - if indexsearcher has already resolved, fine, otherwise it will be done
>> again at the fshq level. Ill issue it up later.
>>
>> --
>> - Mark
>>
>> http://www.lucidimagination.com
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene 1483 and Auto resolution
Posted by Michael McCandless <lu...@mikemccandless.com>.
Urgh, right. Can't we simply restore the AUTO resolution into it?
Existing (direct) usage of it must be passing in the top-level
IndexReader. (Lucene doesn't use FSHQ internally).
Mike
On Thu, Apr 23, 2009 at 6:48 PM, Mark Miller <ma...@gmail.com> wrote:
> Just got off the train and ny to ct has a brilliant bar car, so lest I
> forget:
>
> 1483 moved auto resolution from fshq to indexsearcher - which is a back
> compat break if you were using a fshq without indexsearcher (Solr does it -
> anyone could). Annoying. If I remember right, I did it to resolve auto on
> the multireader rather than each individual segment reader. So the change is
> needed and not allowed. Perhaps it could just re-resolve like before though
> - if indexsearcher has already resolved, fine, otherwise it will be done
> again at the fshq level. Ill issue it up later.
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org