You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Eric Grobler <im...@googlemail.com> on 2013/10/24 23:24:39 UTC
Post filter cache question
Hi
If I run this query it is very fast (<10 ms) because it uses a "TopList"
filter:
q=*:*
fl=adr_geopoint,adr_city,filterflags
*fq=(filterflags:TopList) *
and the number of relevant documents are 3000 out of 7 million.
If I run the same query but add a spatial filter with cost:
q=*:*
fl=adr_geopoint,adr_city,filterflags
*fq=(filterflags:TopList) *
pt=49.594,8.468
sfield=adr_geopoint
fq={!bbox d=30}
fq={!frange l=15 u=30 *cache=false *cost=200}geodist()
It takes over 3 seconds even though it should only scan around 3000
documents from the first cached filter?
Could it be a problem with my cache settings in solrconfig.xml (solr 3.1)
or is my query wrong?
Thanks & regards
Ericz
Re: Post filter cache question
Posted by Erick Erickson <er...@gmail.com>.
Make it at least 4.5.1, which was just released!
Erick
On Thu, Oct 24, 2013 at 8:45 PM, Eric Grobler <im...@googlemail.com>wrote:
> Hi Chris
>
> Thank you for your response.
> I will try to migrate to Solr 4.4 first!
>
> Best regards
>
>
>
> On Thu, Oct 24, 2013 at 10:44 PM, Chris Hostetter
> <ho...@fucit.org>wrote:
>
> >
> > : Could it be a problem with my cache settings in solrconfig.xml (solr
> 3.1)
> > : or is my query wrong?
> >
> > 3.1? ouch ... PostFilter wasn't even added until 3.4...
> > https://wiki.apache.org/solr/CommonQueryParameters#Caching_of_filters
> >
> > ...so your spatial filter is definitely being applied to the entire index
> > and then getting cached.
> >
> > . . .
> >
> > Below is what i wrote before i saw that 3.4 comment at the end of your
> > email...
> >
> > : If I run the same query but add a spatial filter with cost:
> > : q=*:*
> > : fl=adr_geopoint,adr_city,filterflags
> > : *fq=(filterflags:TopList) *
> > : pt=49.594,8.468
> > : sfield=adr_geopoint
> > : fq={!bbox d=30}
> > : fq={!frange l=15 u=30 *cache=false *cost=200}geodist()
> > :
> > : It takes over 3 seconds even though it should only scan around 3000
> > : documents from the first cached filter?
> >
> > You've also added a "bbox" filter, which will be computed against the
> > entire index and cached.
> >
> > I'm not sure whta FieldType you are using, and i don't know a lot of the
> > detials about hte spatial queries -- but things you should look into...
> >
> > 1) does the bbox gain you anything if you are already doing the geodist
> > filter as a post filter? (my hunch would be that the only point of a
> bbox
> > fq is if you are *scoring* documents by distance and you want to ignore
> > things beyond a set distance)
> >
> > 2) does {!bbox} support PostFilter on your FieldType? does
> > adding "cache=false cost=150" to the bbox filter improve things?
> >
> >
> >
> > -Hoss
> >
>
Re: Post filter cache question
Posted by Eric Grobler <im...@googlemail.com>.
Hi Chris
Thank you for your response.
I will try to migrate to Solr 4.4 first!
Best regards
On Thu, Oct 24, 2013 at 10:44 PM, Chris Hostetter
<ho...@fucit.org>wrote:
>
> : Could it be a problem with my cache settings in solrconfig.xml (solr 3.1)
> : or is my query wrong?
>
> 3.1? ouch ... PostFilter wasn't even added until 3.4...
> https://wiki.apache.org/solr/CommonQueryParameters#Caching_of_filters
>
> ...so your spatial filter is definitely being applied to the entire index
> and then getting cached.
>
> . . .
>
> Below is what i wrote before i saw that 3.4 comment at the end of your
> email...
>
> : If I run the same query but add a spatial filter with cost:
> : q=*:*
> : fl=adr_geopoint,adr_city,filterflags
> : *fq=(filterflags:TopList) *
> : pt=49.594,8.468
> : sfield=adr_geopoint
> : fq={!bbox d=30}
> : fq={!frange l=15 u=30 *cache=false *cost=200}geodist()
> :
> : It takes over 3 seconds even though it should only scan around 3000
> : documents from the first cached filter?
>
> You've also added a "bbox" filter, which will be computed against the
> entire index and cached.
>
> I'm not sure whta FieldType you are using, and i don't know a lot of the
> detials about hte spatial queries -- but things you should look into...
>
> 1) does the bbox gain you anything if you are already doing the geodist
> filter as a post filter? (my hunch would be that the only point of a bbox
> fq is if you are *scoring* documents by distance and you want to ignore
> things beyond a set distance)
>
> 2) does {!bbox} support PostFilter on your FieldType? does
> adding "cache=false cost=150" to the bbox filter improve things?
>
>
>
> -Hoss
>
Re: Post filter cache question
Posted by Chris Hostetter <ho...@fucit.org>.
: Could it be a problem with my cache settings in solrconfig.xml (solr 3.1)
: or is my query wrong?
3.1? ouch ... PostFilter wasn't even added until 3.4...
https://wiki.apache.org/solr/CommonQueryParameters#Caching_of_filters
...so your spatial filter is definitely being applied to the entire index
and then getting cached.
. . .
Below is what i wrote before i saw that 3.4 comment at the end of your
email...
: If I run the same query but add a spatial filter with cost:
: q=*:*
: fl=adr_geopoint,adr_city,filterflags
: *fq=(filterflags:TopList) *
: pt=49.594,8.468
: sfield=adr_geopoint
: fq={!bbox d=30}
: fq={!frange l=15 u=30 *cache=false *cost=200}geodist()
:
: It takes over 3 seconds even though it should only scan around 3000
: documents from the first cached filter?
You've also added a "bbox" filter, which will be computed against the
entire index and cached.
I'm not sure whta FieldType you are using, and i don't know a lot of the
detials about hte spatial queries -- but things you should look into...
1) does the bbox gain you anything if you are already doing the geodist
filter as a post filter? (my hunch would be that the only point of a bbox
fq is if you are *scoring* documents by distance and you want to ignore
things beyond a set distance)
2) does {!bbox} support PostFilter on your FieldType? does
adding "cache=false cost=150" to the bbox filter improve things?
-Hoss