You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jay Potharaju <js...@gmail.com> on 2016/05/27 20:32:27 UTC
Slow date filter query
Hi,
I am running filter query(range query) on date fields(high cardinality) and
the performance is really bad ...it takes about 2-5 seconds for it to come
back with response. I am rebuilding the index to have docvalues & tdates
instead of "date" field. But not sure if that will alleviate the problem
because of high cardinality.
Can I store the date as YYYYMMDD and run range queries on them instead of
date fields?
Is that a good option?
--
Thanks
Jay
Re: Slow date filter query
Posted by Jay Potharaju <js...@gmail.com>.
There are about 30 Million Docs and the index size is 75 GB. Using a full
timestamp value when querying and not using NOW. The fq queries covers
almost all the docs(20+ million) in the index.
Thanks
On Mon, May 30, 2016 at 8:17 PM, Erick Erickson <er...@gmail.com>
wrote:
> Oops, fat fingers.
>
> see:
> searchhub.org/2012/02/23/date-math-now-and-filter-queries/
>
> If you're not re-using the _same_ filter query, you'll be better
> off using fq={!cache=false}range_query
>
> Best,
> Erick
>
> On Mon, May 30, 2016 at 8:16 PM, Erick Erickson <er...@gmail.com>
> wrote:
> > That does seem long, but you haven't provided many details
> > about the fields. Are there 100 docs in your index? 100M docs? 500M docs?
> >
> > Are you using NOW in appropriately? See:
> >
> > On Fri, May 27, 2016 at 1:32 PM, Jay Potharaju <js...@gmail.com>
> wrote:
> >> Hi,
> >> I am running filter query(range query) on date fields(high cardinality)
> and
> >> the performance is really bad ...it takes about 2-5 seconds for it to
> come
> >> back with response. I am rebuilding the index to have docvalues & tdates
> >> instead of "date" field. But not sure if that will alleviate the problem
> >> because of high cardinality.
> >>
> >> Can I store the date as YYYYMMDD and run range queries on them instead
> of
> >> date fields?
> >> Is that a good option?
> >>
> >> --
> >> Thanks
> >> Jay
>
--
Thanks
Jay Potharaju
Re: Slow date filter query
Posted by Erick Erickson <er...@gmail.com>.
Oops, fat fingers.
see:
searchhub.org/2012/02/23/date-math-now-and-filter-queries/
If you're not re-using the _same_ filter query, you'll be better
off using fq={!cache=false}range_query
Best,
Erick
On Mon, May 30, 2016 at 8:16 PM, Erick Erickson <er...@gmail.com> wrote:
> That does seem long, but you haven't provided many details
> about the fields. Are there 100 docs in your index? 100M docs? 500M docs?
>
> Are you using NOW in appropriately? See:
>
> On Fri, May 27, 2016 at 1:32 PM, Jay Potharaju <js...@gmail.com> wrote:
>> Hi,
>> I am running filter query(range query) on date fields(high cardinality) and
>> the performance is really bad ...it takes about 2-5 seconds for it to come
>> back with response. I am rebuilding the index to have docvalues & tdates
>> instead of "date" field. But not sure if that will alleviate the problem
>> because of high cardinality.
>>
>> Can I store the date as YYYYMMDD and run range queries on them instead of
>> date fields?
>> Is that a good option?
>>
>> --
>> Thanks
>> Jay
Re: Slow date filter query
Posted by Erick Erickson <er...@gmail.com>.
That does seem long, but you haven't provided many details
about the fields. Are there 100 docs in your index? 100M docs? 500M docs?
Are you using NOW in appropriately? See:
On Fri, May 27, 2016 at 1:32 PM, Jay Potharaju <js...@gmail.com> wrote:
> Hi,
> I am running filter query(range query) on date fields(high cardinality) and
> the performance is really bad ...it takes about 2-5 seconds for it to come
> back with response. I am rebuilding the index to have docvalues & tdates
> instead of "date" field. But not sure if that will alleviate the problem
> because of high cardinality.
>
> Can I store the date as YYYYMMDD and run range queries on them instead of
> date fields?
> Is that a good option?
>
> --
> Thanks
> Jay