SortedDocValuesField

[Sandeep Khanzode <sa...@yahoo.com.INVALID>]

Hi,
 
I was checking the SortedDocValuesField and its performance in Sort as opposed to a normal i.e. StringField and its performance in the same sort. So, I used the same string/bytesref value in both fields and in separate JVM processes, I launched the two sorts.

I used a RAMDirectory and created a million items. The SortedDocValuesField sort took 12/13 seconds and consumed approx 500-550 megs of RAM whereas the StringField took 10/11 seconds and consumed 350-400 megs of RAM. 
Is this normal behavior? I was expecting the SDVF to perform better since it is indexed for sorting and not stored for any other purpose.

-----------------------

Thanks n Regards,
Sandeep Ramesh Khanzode

Re: SortedDocValuesField

[Robert Muir <rc...@gmail.com>]

don't use RAMDirectory: its not very performant and really intended
for e.g. testing and so on.

also, using a ramdirectory here defeats the purpose: the idea behind
using a docvaluesfield in most cases is to keep (most of) such
datastructures out of heap memory. The datastructures and even the
compression used are optimized for mmap and nio access...



On Thu, Jun 26, 2014 at 11:59 AM, Sandeep Khanzode
<sa...@yahoo.com.invalid> wrote:
> Hi,
>
> I was checking the SortedDocValuesField and its performance in Sort as opposed to a normal i.e. StringField and its performance in the same sort. So, I used the same string/bytesref value in both fields and in separate JVM processes, I launched the two sorts.
>
> I used a RAMDirectory and created a million items. The SortedDocValuesField sort took 12/13 seconds and consumed approx 500-550 megs of RAM whereas the StringField took 10/11 seconds and consumed 350-400 megs of RAM.
> Is this normal behavior? I was expecting the SDVF to perform better since it is indexed for sorting and not stored for any other purpose.
>
> -----------------------
>
> Thanks n Regards,
> Sandeep Ramesh Khanzode

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org