You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jonathan Ariel <io...@gmail.com> on 2009/09/10 22:35:11 UTC
Query runs faster without filter queries?
Hi all!
I'm trying to measure the query response time when using just a query and
when using some filter queries. From what I read and understand adding
filter query should boost the query response time. I used luke to understand
over which fields I should use filter query (those that have few unique
terms, in my case 2 fields of 30 and 400 unique fields). I'm using solr 1.3.
In order to test the query performance I disabled queryCache and
documentCache, so I just have filterCache enabled.I did that because I
wanted to be sure that there is no caching when I measure my queries. I left
filterCache because it makes sense since filter query uses that.
When I first execute my query without filter cache it runs in 400ms, next
execution of the same query around 20ms.
When I first execute my query with filter cache it runs in 500ms, next
execution of the same query around 50ms.
Why the query with filter query runs slower than the query without filter
query? Shouldn't it be the other way around?
My index is around 12M documents. My filterCache max size is set to 40000 (I
think more than enough). The fields that I use as filter queries are integer
and in my query I search over a tokenized text field.
What do you think?
Thanks a lot,
Jonathan
Re: Query runs faster without filter queries?
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Fri, Sep 11, 2009 at 8:50 AM, Jonathan Ariel <io...@gmail.com> wrote:
> Oh... another question regard this. If I disabled the query cache and
> document cache and I execute a query without filter cache (no facets, no
> filter query, etc.). Why the first time I execute the query it takes around
> 400ms and the second time 10ms? It seems like it should always take the same
> amount of time, right?
Same pointer I just gave in a nother thread:
http://search.lucidimagination.com/search/document/b5eee1fc75cc454c/caching_in_lucene
-Yonik
http://www.lucidimagination.com
Re: Query runs faster without filter queries?
Posted by Jonathan Ariel <io...@gmail.com>.
Oh... another question regard this. If I disabled the query cache and
document cache and I execute a query without filter cache (no facets, no
filter query, etc.). Why the first time I execute the query it takes around
400ms and the second time 10ms? It seems like it should always take the same
amount of time, right?
On Thu, Sep 10, 2009 at 9:44 PM, Jonathan Ariel <io...@gmail.com> wrote:
> Thanks! I don't think I can use an unreleased version of solr even is it's
> stable enough (crazy infrastructure guys) but I might be able to apply the 2
> patches mentioned in the link you sent. I will try it in my local copy of
> solr and see if it improves and let you know.
> Thanks!
>
>
> On Thu, Sep 10, 2009 at 5:43 PM, Uri Boness <ub...@gmail.com> wrote:
>
>> If I recall correctly, in solr 1.3 there was an issue where filters didn't
>> really behaved as they should have. Basically, if you had a query and
>> filters defined, the query would have executed normally and only after that
>> the filter would be applied. AFAIK this is fixed in 1.4 where now the
>> documents which are defined by the filters are skipped during the query
>> execution.
>>
>> Uri
>>
>>
>> Jonathan Ariel wrote:
>>
>>> Hi all!
>>> I'm trying to measure the query response time when using just a query and
>>> when using some filter queries. From what I read and understand adding
>>> filter query should boost the query response time. I used luke to
>>> understand
>>> over which fields I should use filter query (those that have few unique
>>> terms, in my case 2 fields of 30 and 400 unique fields). I'm using solr
>>> 1.3.
>>> In order to test the query performance I disabled queryCache and
>>> documentCache, so I just have filterCache enabled.I did that because I
>>> wanted to be sure that there is no caching when I measure my queries. I
>>> left
>>> filterCache because it makes sense since filter query uses that.
>>>
>>> When I first execute my query without filter cache it runs in 400ms, next
>>> execution of the same query around 20ms.
>>> When I first execute my query with filter cache it runs in 500ms, next
>>> execution of the same query around 50ms.
>>>
>>> Why the query with filter query runs slower than the query without filter
>>> query? Shouldn't it be the other way around?
>>>
>>> My index is around 12M documents. My filterCache max size is set to 40000
>>> (I
>>> think more than enough). The fields that I use as filter queries are
>>> integer
>>> and in my query I search over a tokenized text field.
>>>
>>> What do you think?
>>>
>>> Thanks a lot,
>>>
>>> Jonathan
>>>
>>>
>>>
>>
>
Re: Query runs faster without filter queries?
Posted by Jonathan Ariel <io...@gmail.com>.
Thanks! I don't think I can use an unreleased version of solr even is it's
stable enough (crazy infrastructure guys) but I might be able to apply the 2
patches mentioned in the link you sent. I will try it in my local copy of
solr and see if it improves and let you know.
Thanks!
On Thu, Sep 10, 2009 at 5:43 PM, Uri Boness <ub...@gmail.com> wrote:
> If I recall correctly, in solr 1.3 there was an issue where filters didn't
> really behaved as they should have. Basically, if you had a query and
> filters defined, the query would have executed normally and only after that
> the filter would be applied. AFAIK this is fixed in 1.4 where now the
> documents which are defined by the filters are skipped during the query
> execution.
>
> Uri
>
>
> Jonathan Ariel wrote:
>
>> Hi all!
>> I'm trying to measure the query response time when using just a query and
>> when using some filter queries. From what I read and understand adding
>> filter query should boost the query response time. I used luke to
>> understand
>> over which fields I should use filter query (those that have few unique
>> terms, in my case 2 fields of 30 and 400 unique fields). I'm using solr
>> 1.3.
>> In order to test the query performance I disabled queryCache and
>> documentCache, so I just have filterCache enabled.I did that because I
>> wanted to be sure that there is no caching when I measure my queries. I
>> left
>> filterCache because it makes sense since filter query uses that.
>>
>> When I first execute my query without filter cache it runs in 400ms, next
>> execution of the same query around 20ms.
>> When I first execute my query with filter cache it runs in 500ms, next
>> execution of the same query around 50ms.
>>
>> Why the query with filter query runs slower than the query without filter
>> query? Shouldn't it be the other way around?
>>
>> My index is around 12M documents. My filterCache max size is set to 40000
>> (I
>> think more than enough). The fields that I use as filter queries are
>> integer
>> and in my query I search over a tokenized text field.
>>
>> What do you think?
>>
>> Thanks a lot,
>>
>> Jonathan
>>
>>
>>
>
Re: Query runs faster without filter queries?
Posted by Uri Boness <ub...@gmail.com>.
If I recall correctly, in solr 1.3 there was an issue where filters
didn't really behaved as they should have. Basically, if you had a query
and filters defined, the query would have executed normally and only
after that the filter would be applied. AFAIK this is fixed in 1.4 where
now the documents which are defined by the filters are skipped during
the query execution.
Uri
Jonathan Ariel wrote:
> Hi all!
> I'm trying to measure the query response time when using just a query and
> when using some filter queries. From what I read and understand adding
> filter query should boost the query response time. I used luke to understand
> over which fields I should use filter query (those that have few unique
> terms, in my case 2 fields of 30 and 400 unique fields). I'm using solr 1.3.
> In order to test the query performance I disabled queryCache and
> documentCache, so I just have filterCache enabled.I did that because I
> wanted to be sure that there is no caching when I measure my queries. I left
> filterCache because it makes sense since filter query uses that.
>
> When I first execute my query without filter cache it runs in 400ms, next
> execution of the same query around 20ms.
> When I first execute my query with filter cache it runs in 500ms, next
> execution of the same query around 50ms.
>
> Why the query with filter query runs slower than the query without filter
> query? Shouldn't it be the other way around?
>
> My index is around 12M documents. My filterCache max size is set to 40000 (I
> think more than enough). The fields that I use as filter queries are integer
> and in my query I search over a tokenized text field.
>
> What do you think?
>
> Thanks a lot,
>
> Jonathan
>
>
Re: Query runs faster without filter queries?
Posted by Yonik Seeley <yo...@lucidimagination.com>.
Try 1.4
http://www.lucidimagination.com/blog/2009/05/27/filtered-query-performance-increases-for-solr-14/
-Yonik
http://www.lucidimagination.com
On Thu, Sep 10, 2009 at 4:35 PM, Jonathan Ariel <io...@gmail.com> wrote:
> Hi all!
> I'm trying to measure the query response time when using just a query and
> when using some filter queries. From what I read and understand adding
> filter query should boost the query response time. I used luke to understand
> over which fields I should use filter query (those that have few unique
> terms, in my case 2 fields of 30 and 400 unique fields). I'm using solr 1.3.
> In order to test the query performance I disabled queryCache and
> documentCache, so I just have filterCache enabled.I did that because I
> wanted to be sure that there is no caching when I measure my queries. I left
> filterCache because it makes sense since filter query uses that.
>
> When I first execute my query without filter cache it runs in 400ms, next
> execution of the same query around 20ms.
> When I first execute my query with filter cache it runs in 500ms, next
> execution of the same query around 50ms.
>
> Why the query with filter query runs slower than the query without filter
> query? Shouldn't it be the other way around?
>
> My index is around 12M documents. My filterCache max size is set to 40000 (I
> think more than enough). The fields that I use as filter queries are integer
> and in my query I search over a tokenized text field.
>
> What do you think?
>
> Thanks a lot,
>
> Jonathan
>