FieldCache.DEFAULT.getInts vs FieldCache.DEFAULT.getStringIndex. Memory usage

[Marc Sturlese <ma...@gmail.com>]

I need to load a FieldCache for a field wich is a solr "integer" type and has
as maximum 3 digits. Let's say my index has 10M docs.
I am wandering what is more optimal and less memory consumig, to load a
FieldCache.DEFAUL.getInts or a FieldCache.DEFAULT.getStringIndex.

The second one will have a int[] for as many docs as the index have.
Additionally will have a String[] for as many unique terms. As I am dealing
with numbers, I will have to cast the values of the String[] to deal with
them.

If I load a FieldCache.DEFAULT.getInts I will have just an int[] with a
value of a doc field on each array position. I will be able to deal straight
with the ints... in this case will it be more optimal to use this?

Thanks in advance
-- 
View this message in context: http://lucene.472066.n3.nabble.com/FieldCache-DEFAULT-getInts-vs-FieldCache-DEFAULT-getStringIndex-Memory-usage-tp1348480p1348480.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: FieldCache.DEFAULT.getInts vs FieldCache.DEFAULT.getStringIndex. Memory usage

[Chris Hostetter <ho...@fucit.org>]

: I need to load a FieldCache for a field wich is a solr "integer" type and has
: as maximum 3 digits. Let's say my index has 10M docs.
: I am wandering what is more optimal and less memory consumig, to load a
: FieldCache.DEFAUL.getInts or a FieldCache.DEFAULT.getStringIndex.

by itself, getInts always uses use less memory then getStringIndex.  no 
matter what your data looks like, getStringIndex can never use less memory 
then getInts.

the question hwoever is if any other code is going to use getStringIndex 
on the same field, defeating any memory savings you have -- you said 
"integer" but you didn't say what FieldType class that was mapped to.  In 
the 1.4 example schema, "int" is mapped to a TreiField which will use 
getInts() for the field cache.  In Solr 1.3's example schema the "integer" 
type was mapped to "IntField" which also uses getInts() for hte field 
cache.

But we have no idea what your schema is using.

If it uses "SortableIntField" that's when the SOlr code under the covers 
is going to use getStringIndex(), so you might as well use it also.

(you can verify this in Solr 1.4 by looking at the stats for the 
fieldCache - it tells you exactly what is in use at any moment)

-Hoss

--
http://lucenerevolution.org/  ...  October 7-8, Boston
http://bit.ly/stump-hoss      ...  Stump The Chump!