Merging indexes

[Rupinder Singh Mazara <rs...@ebi.ac.uk>]

Hi all 

  I got problem with merging indexes,
  I had to split up  the indexing of my data into 20 different indexes(based on a primary key) 
  I wanted to merge them all into one master index 

  for example i have 
  /xxx/lucene/tmp/1001-1000
  /xxx/lucene/tmp/1001-2000
  /xxx/lucene/tmp/2001-3001 ....etc

  i want to merge them into the index
  /xxx/lucene/index/complete 

  i was hoping to merge the indexes as the jobs get completed,

  i tried the following code , createNew: is a boolean variable turned to true for the first merge 
 and then is turned to false for all other indexes  
 tmpRoot: is the folder in which the split jobs are entered,
 d1: is the index i wanted merged

        Directory allIndexes[] = new Directory[1];
        allIndexes[0] = FSDirectory.getDirectory(new File(tmpRoot + d1), false);
        aLog.info("Dir's opened for merge ");

        IndexWriter iWriter = new IndexWriter(FSDirectory.getDirectory(new File(op), true), myAnalyzer, createNew);
        // IndexWriter iWriter = new IndexWriter(FSDirectory.getDirectory(new File(op), true), myAnalyzer, false);
        aLog.fatal("Retrieved Indexes preparing to MERGE ");
        iWriter.addIndexes(allIndexes);
        aLog.fatal("Preparing to optimize ");
        iWriter.optimize();
        aLog.fatal("Closing indexes  ");
        iWriter.close();

  the problems is the first run happens fine ,  but  the next job runs and ends up deleting the contents of the master index directory



 


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org