Performance tips?

[Daniel Pfeifer <Da...@tradedoubler.com>]

Hi,

 

Got more questions regarding Lucene and this time it's about performance
;-)

 

We currently are using RAMDirectories to read our Indexes. This has now
become a problem since our index has grown to appx 5GB of RAM and the
machine we are running on only has 12GB of RAM and everytime we refresh
the RAMDirectories we of course keep the old Searchables so that there
is no service interruption.

 

This means we consume 10GB of RAM from time to time. One solution is of
course to stop using RAM and read anything from disk but I can imagine
that the performance will decrease significantly. Is there any
workaround you can think of? Perhaps a hybrid between FSDirectory and
RAMDirectory. For example that only frequently searched documents are
cached and the others are read from disk?

 

Well, I'd appreciate any ideas at all!
Thanks
/Daniel

Re: Performance tips?

[Chris Lamprecht <cl...@gmail.com>]

I seem to say this a lot :), but, assuming your OS has a decent
filesystem cache, try reducing your JVM heapsize, using an FSDirectory
instead of RAMDirectory, and see if your filesystem cache does ok.  If
you have 12GB, then you should have enough RAM to hold both the old
and new indexes during the switchover.

-chris

On 1/26/06, Daniel Pfeifer <Da...@tradedoubler.com> wrote:
> Hi,
>
>
>
> Got more questions regarding Lucene and this time it's about performance
> ;-)
>
>
>
> We currently are using RAMDirectories to read our Indexes. This has now
> become a problem since our index has grown to appx 5GB of RAM and the
> machine we are running on only has 12GB of RAM and everytime we refresh
> the RAMDirectories we of course keep the old Searchables so that there
> is no service interruption.
>
>
>
> This means we consume 10GB of RAM from time to time. One solution is of
> course to stop using RAM and read anything from disk but I can imagine
> that the performance will decrease significantly. Is there any
> workaround you can think of? Perhaps a hybrid between FSDirectory and
> RAMDirectory. For example that only frequently searched documents are
> cached and the others are read from disk?
>
>
>
> Well, I'd appreciate any ideas at all!
> Thanks
> /Daniel
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org