date queries too slow

[veerene <ve...@yahoo.com>]

Hello,
we are having significant performance problems with date queries on our
production server. 
we are using SOLR 1.4 (will be upgrading to latest version in the near
future) and our index size is around 4GB with 2 million documents.
for e.g: the query "tag:obama AND expirationdate:[2012-02-21T00:00:00Z TO *]
AND publicationdate:[* TO 2012-02-21T00:00:00Z]" takes 1113ms and if we
remove the dates "tag:obama" it takes only 98ms.

we tried storing the date as a long data field in the form "YYYMMDDHHMMSS"
and we are seeing improvement in the query times, but it's not significant.

I have read somewhere is we use "TrieDate" field instead "Date" it would
help.
Do you guys think the performance improvement would be significant? if not,
do you see if there is any other alternative solution to speed up the date
queries?
I appreciate your response.



--
View this message in context: http://lucene.472066.n3.nabble.com/date-queries-too-slow-tp3794345p3794345.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: date queries too slow

[veerene <ve...@yahoo.com>]

Eric,

thank you very much, it's really helpful!

--
View this message in context: http://lucene.472066.n3.nabble.com/date-queries-too-slow-tp3794345p3799766.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: date queries too slow

[Erick Erickson <er...@gmail.com>]

You might find this of interest re: filter queries and NOW (the current time)
http://www.lucidimagination.com/blog/2012/02/23/date-math-now-and-filter-queries/

Best
Erick

On Fri, Mar 2, 2012 at 3:56 PM, veerene <ve...@yahoo.com> wrote:
> thanks for responding. we will try the trie fields.
> the reason we are not using filters is these date values would change from
> query to query.
> we are dynamically populating these date values in the queries using the
> current time.
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/date-queries-too-slow-tp3794345p3794677.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: date queries too slow

[veerene <ve...@yahoo.com>]

thanks for responding. we will try the trie fields.
the reason we are not using filters is these date values would change from
query to query.
we are dynamically populating these date values in the queries using the
current time.



--
View this message in context: http://lucene.472066.n3.nabble.com/date-queries-too-slow-tp3794345p3794677.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: date queries too slow

[Ahmet Arslan <io...@yahoo.com>]

--- On Fri, 3/2/12, veerene <ve...@yahoo.com> wrote:

> From: veerene <ve...@yahoo.com>
> Subject: date queries too slow
> To: solr-user@lucene.apache.org
> Date: Friday, March 2, 2012, 8:29 PM
> Hello,
> we are having significant performance problems with date
> queries on our
> production server. 
> we are using SOLR 1.4 (will be upgrading to latest version
> in the near
> future) and our index size is around 4GB with 2 million
> documents.
> for e.g: the query "tag:obama AND
> expirationdate:[2012-02-21T00:00:00Z TO *]
> AND publicationdate:[* TO 2012-02-21T00:00:00Z]" takes
> 1113ms and if we
> remove the dates "tag:obama" it takes only 98ms.
> 
> we tried storing the date as a long data field in the form
> "YYYMMDDHHMMSS"
> and we are seeing improvement in the query times, but it's
> not significant.
> 
> I have read somewhere is we use "TrieDate" field instead
> "Date" it would
> help.

For range queries, definitely, trie based fields are the way to go. Also you can use filter query to re-use the filters.

q=tag:obama&fq=expirationdate:[2012-02-21T00:00:00Z TO *]&fq=publicationdate:[* TO 2012-02-21T00:00:00Z]

http://wiki.apache.org/solr/SolrCaching#filterCache