DO NOT REPLY [Bug 30421] New: - Merging indexes leads to out-of-memory condition

[bu...@apache.org]

DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG 
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=30421>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND 
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=30421

Merging indexes leads to out-of-memory condition

           Summary: Merging indexes leads to out-of-memory condition
           Product: Lucene
           Version: 1.4
          Platform: Other
        OS/Version: Linux
            Status: NEW
          Severity: Major
          Priority: Other
         Component: Index
        AssignedTo: lucene-dev@jakarta.apache.org
        ReportedBy: ddreon1@yahoo.com


Version: 1.4 Final
Platform: Red Hat 9
Method: IndexWriter.addIndexes

My experience with merging indexes of any size results in one outcome: 
java.lang.OutOfMemoryError. This is true even on a 1GB machine where almost all 
free memory is allocated to the VM.

Is there a way to reliably merge indexes?

I've marked this issue as 'major' for the following reason: Without an 
efficient merge capability, the developer must deploy hundreds of 'little' 
indexes that will need to be searched in parallel. Unfortunately, this 
constraint plus java makes an application too slow to be usable in a production 
environment.

Thank you for looking into this issue.


>>>>>>>>> sample code

Directory dest = FSDirectory.getDirectory( destination, true);
IndexWriter writer = new IndexWriter( dest, new TermAnalyzer( 
StopWords.SEARCH_MAP), true);

writer.addIndexes( sources);
writer.optimize();

writer.close();

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org