performance merging indexes with addIndexesNoOptimize

[Marc Sturlese <ma...@gmail.com>]

I am doing some test about merge indexing and have a performance doubt
I am doing merge in a simple way, something like:

      FSDirectory indexes[] = new FSDirectory[indexList.size()];
      for (int i = 0; i < indexList.size(); i++) {
        indexes[i] = FSDirectory.open(new File(indexList.get(i)));
      }
      w.addIndexesNoOptimize(indexes);
      w.close();

IndexList.size() is 50 and contains paths to index. These 50 indexes contain
500.000 docs each and have about 500m size each (each index). I have
realised that 50% of the time is spent in w.addIndexesNoOptimize(indexes)
and the other 50 in w.close() (I suppose because close commits and have to
wait for all the merges to be completed).

I am wondering if is there a way to do this faster. For example, merge the
50 indexes into 25 indexes, these 25 into 12, these 12 into 6... till geting
a single big index. Could this be faster?

Does anyone have experience with this? Any advice?
Thanks in advance
-- 
View this message in context: http://lucene.472066.n3.nabble.com/performance-merging-indexes-with-addIndexesNoOptimize-tp1889368p1889368.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org