FieldType for storing date

[Jibo John <ji...@mac.com>]

Hello,

I was wondering what would be the best FieldType for storing date with a millisecond precision that would allow me to sort and run range queries against this field.
We would like to achieve the best query performance, minimal heap - fieldcache - requirements, good indexing throughput  and minimal index size in that order.

To give you some background, we have a production system that runs in a multicore set up, each core with a maximum index size of 6G each, and, the search and indexing operations occur against the same cores. We store the date with a minutely precision(format yymmddhhmm), and, we use TrieIntField with a precisionStep=1. This works well, however, as a next step, we want to store the date with a millisecond precision with minimal architectural changes.

We could probably use TrieLongField, however, as we understand, this doubles the heap requirements for fieldcache. Was wondering if there is a clever way of achieving this without adding to the heap.

Appreciate your input.

Thanks,
-Jibo

Re: FieldType for storing date

[Chris Hostetter <ho...@fucit.org>]

: I was wondering what would be the best FieldType for storing date with a 
: millisecond precision that would allow me to sort and run range queries 
: against this field. We would like to achieve the best query performance, 
: minimal heap - fieldcache - requirements, good indexing throughput and 
: minimal index size in that order.

if you don't need sortMissingLast or sortMissingFirst then TrieDateField 
should be exactly what you are looking for.

: We could probably use TrieLongField, however, as we understand, this 
: doubles the heap requirements for fieldcache. Was wondering if there is 
: a clever way of achieving this without adding to the heap.

TrieDateField uses the long[] FieldCache, I'm not sure what you mean by 
"doubles the heap requirements" ... unless you are comparing to "int" ?

In that case: using TrieIntField seems like what you want?

(but if you are comparing to DateField, the FieldCache for TrieDateField 
is going to be a lot smaller)


-Hoss

--
http://lucenerevolution.org/  ...  October 7-8, Boston
http://bit.ly/stump-hoss      ...  Stump The Chump!