You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jason Rutherglen <ja...@gmail.com> on 2009/10/19 20:55:00 UTC
Filter query optimization
If a filter query matches nothing, then no additional query should be
performed and no results returned? I don't think we have this today?
Re: Filter query optimization
Posted by Jason Rutherglen <ja...@gmail.com>.
Ok, thanks, new Lucene 2.9 features.
On Mon, Oct 19, 2009 at 2:33 PM, Yonik Seeley
<yo...@lucidimagination.com> wrote:
> On Mon, Oct 19, 2009 at 4:45 PM, Jason Rutherglen
> <ja...@gmail.com> wrote:
>> Yonik,
>>
>>> this is a fast operation anyway
>>
>> Can you elaborate on why this is a fast operation?
>
> The scorers will never really be used.
> The query will be weighted and scorers will be created, but the filter
> will be checked first and return NO_MORE_DOCS.
>
> -Yonik
> http://www.lucidimagination.com
>
>> Basically there's a distributed query with a filter, where on a
>> number of the servers, the filter query isn't matching anything,
>> however I'm seeing load on those servers (where nothing
>> matches), so I'm assuming the filter is generated (and cached)
>> which is fine, then the user query is being performed on a
>> filter where no documents match. I could misinterpreting the
>> data, however, I want to find out about this use case regardless
>> as it likely will crop up again for us.
>>
>> -J
>>
>> On Mon, Oct 19, 2009 at 12:07 PM, Yonik Seeley
>> <yo...@lucidimagination.com> wrote:
>>> On Mon, Oct 19, 2009 at 2:55 PM, Jason Rutherglen
>>> <ja...@gmail.com> wrote:
>>>> If a filter query matches nothing, then no additional query should be
>>>> performed and no results returned? I don't think we have this today?
>>>
>>> No, but this is a fast operation anyway (In Solr 1.4 at least).
>>>
>>> Another thing to watch out for is to not try this with filters that
>>> you don't know the size of (or else you may force a popcount on a
>>> BitDocSet that would not otherwise have been needed).
>>>
>>> It could also potentially complicate warming queries - need to be
>>> careful that the combination of filters you are warming with matches
>>> something, or it would cause the fieldCache entries to not be
>>> populated.
>>>
>>> -Yonik
>>> http://www.lucidimagination.com
>>>
>>
>
Re: Filter query optimization
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Mon, Oct 19, 2009 at 4:45 PM, Jason Rutherglen
<ja...@gmail.com> wrote:
> Yonik,
>
>> this is a fast operation anyway
>
> Can you elaborate on why this is a fast operation?
The scorers will never really be used.
The query will be weighted and scorers will be created, but the filter
will be checked first and return NO_MORE_DOCS.
-Yonik
http://www.lucidimagination.com
> Basically there's a distributed query with a filter, where on a
> number of the servers, the filter query isn't matching anything,
> however I'm seeing load on those servers (where nothing
> matches), so I'm assuming the filter is generated (and cached)
> which is fine, then the user query is being performed on a
> filter where no documents match. I could misinterpreting the
> data, however, I want to find out about this use case regardless
> as it likely will crop up again for us.
>
> -J
>
> On Mon, Oct 19, 2009 at 12:07 PM, Yonik Seeley
> <yo...@lucidimagination.com> wrote:
>> On Mon, Oct 19, 2009 at 2:55 PM, Jason Rutherglen
>> <ja...@gmail.com> wrote:
>>> If a filter query matches nothing, then no additional query should be
>>> performed and no results returned? I don't think we have this today?
>>
>> No, but this is a fast operation anyway (In Solr 1.4 at least).
>>
>> Another thing to watch out for is to not try this with filters that
>> you don't know the size of (or else you may force a popcount on a
>> BitDocSet that would not otherwise have been needed).
>>
>> It could also potentially complicate warming queries - need to be
>> careful that the combination of filters you are warming with matches
>> something, or it would cause the fieldCache entries to not be
>> populated.
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>
Re: Filter query optimization
Posted by Jason Rutherglen <ja...@gmail.com>.
Yonik,
> this is a fast operation anyway
Can you elaborate on why this is a fast operation?
Basically there's a distributed query with a filter, where on a
number of the servers, the filter query isn't matching anything,
however I'm seeing load on those servers (where nothing
matches), so I'm assuming the filter is generated (and cached)
which is fine, then the user query is being performed on a
filter where no documents match. I could misinterpreting the
data, however, I want to find out about this use case regardless
as it likely will crop up again for us.
-J
On Mon, Oct 19, 2009 at 12:07 PM, Yonik Seeley
<yo...@lucidimagination.com> wrote:
> On Mon, Oct 19, 2009 at 2:55 PM, Jason Rutherglen
> <ja...@gmail.com> wrote:
>> If a filter query matches nothing, then no additional query should be
>> performed and no results returned? I don't think we have this today?
>
> No, but this is a fast operation anyway (In Solr 1.4 at least).
>
> Another thing to watch out for is to not try this with filters that
> you don't know the size of (or else you may force a popcount on a
> BitDocSet that would not otherwise have been needed).
>
> It could also potentially complicate warming queries - need to be
> careful that the combination of filters you are warming with matches
> something, or it would cause the fieldCache entries to not be
> populated.
>
> -Yonik
> http://www.lucidimagination.com
>
Re: Filter query optimization
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Mon, Oct 19, 2009 at 2:55 PM, Jason Rutherglen
<ja...@gmail.com> wrote:
> If a filter query matches nothing, then no additional query should be
> performed and no results returned? I don't think we have this today?
No, but this is a fast operation anyway (In Solr 1.4 at least).
Another thing to watch out for is to not try this with filters that
you don't know the size of (or else you may force a popcount on a
BitDocSet that would not otherwise have been needed).
It could also potentially complicate warming queries - need to be
careful that the combination of filters you are warming with matches
something, or it would cause the fieldCache entries to not be
populated.
-Yonik
http://www.lucidimagination.com