best strategy to deal with large index file

[Jeff Liang <je...@messagesolution.com>]

Hi all,

my index file is huge because of large set of data.  when I do search, I
get outofmemory exception all the time.  it's also bad for backup
because I can't do incremental backup after adding new documents.

What's the best strategy to deal with large index file?  is there a
Lucene built-in method to split the index file?

thanks,

Jeff

Re: best strategy to deal with large index file

[Stefan Groschupf <sg...@media-style.com>]

First of all check your memory setup of your tomcat, I think by  
default it uses only 64 MB RAM.
You need to change this manually, see tomcat documentation.
Second I suggest to split the index and run multiple search servers.

How to:
http://wiki.media-style.com/display/nutchDocu/setup+multiple+search 
+sever


Am 16.12.2005 um 09:41 schrieb Jeff Liang:

> Hi all,
>
> my index file is huge because of large set of data.  when I do  
> search, I
> get outofmemory exception all the time.  it's also bad for backup
> because I can't do incremental backup after adding new documents.
>
> What's the best strategy to deal with large index file?  is there a
> Lucene built-in method to split the index file?
>
> thanks,
>
> Jeff
>