date range querys

[Thomas Klein <tk...@laposte.net>]

Hi there,

I'm indexing a lot of documents with lucene and I'm really happy with 
it :)

Observing my users, I understand they are querying only small range of 
time. Often, they only read results back from 1 month.

I got some years in my 1.3Go index. Querys takes some seconds getting 
and returning docs IDs.

I'm indexing a some date field like that : 
20060612223456
2006061222
20060612
200606
2006

In your opinion, will range query be fast enough or will I need to 
create, for exemple, one index per month and then open and parse them 
regarding querys ?

Regards,

Thomas.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: date range querys

[Chris Hostetter <ho...@fucit.org>]

: I suspect that if you stored your dates (use DateTools) as strings
: with a resolution of a day you'd get much faster queries, assuming
: that this is fine enough for your app.

...especialy if you use ConstanScoreRangeQUery ... if you find that isn't
fast enough, having the various granularities *may* help improve
performance, but only if they are stored in seperate fields (so the
when you query only at year granularity the termEnum doesn't have to
iterate over all the terms at finer granularity)

I've never tried this myself ... 99.99% of hte time i'm using Solr,
storing things with millisecond resolution, and trusting Solr's caching to
take care of makingthings fast enough for me (searching at day/hour
granularity 98% of the time makes that caching very effective)



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: date range querys

[Erick Erickson <er...@gmail.com>]

Search the mail archive for DateRange, Date, DateTools
ConstantScoreRangeQuery, etc and you'll find a wealth of
discussion on dates, which turn out to be
more complex than you think <G>...

But the first thing I'd think about is your resolution. Understand
that range queries may assemble a giant OR query for all
the values in your index that fall in the range. Also remember
that dates are simply strings, they aren't a distinct data type.

So, it looks from your example like you're storing your dates with
varying resolutions, and possibly assembling far larger queries
than you think.

I suspect that if you stored your dates (use DateTools) as strings
with a resolution of a day you'd get much faster queries, assuming
that this is fine enough for your app.

Best
Erick

On 3/11/07, Thomas Klein <tk...@laposte.net> wrote:
>
> Hi there,
>
> I'm indexing a lot of documents with lucene and I'm really happy with
> it :)
>
> Observing my users, I understand they are querying only small range of
> time. Often, they only read results back from 1 month.
>
> I got some years in my 1.3Go index. Querys takes some seconds getting
> and returning docs IDs.
>
> I'm indexing a some date field like that :
> 20060612223456
> 2006061222
> 20060612
> 200606
> 2006
>
> In your opinion, will range query be fast enough or will I need to
> create, for exemple, one index per month and then open and parse them
> regarding querys ?
>
> Regards,
>
> Thomas.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>