Problem during indexing

[sarfaraz masood <sa...@yahoo.com>]

i am trying to add 20 million documents to my index from another index that contains these documents(cant help this architecture..its something that i will have to follow) Now the problems i am facing are the following :

1) Too many files open error.. its at the code which is adding documents to my index.

            IndexWriter w = new IndexWriter();
IndexWriter w = new IndexWriter(index , analyzer, IndexWriter.MaxFieldLength.UNLIMITED );
            w.setMergeFactor(1000);
            w.setMaxBufferedDocs(1000);
            w.setMaxMergeDocs(600000);

            for(i =0;i<reader1.numDocs();i++)
            {
                System.out.println(i);
               addDoc(w, reader1.document(i).getField(" url ").stringValue( ),reader1.document(i).getField(" content ").stringValue().replace('.',' ').replace('-',' '));    
                    
            }
            w.optimize();
            w.close();
            reader1.close();

Due to the mergefactor parameters that i have set, around 1300 .cfs files are now opened in index, but there is only one fdt file. These files seem to be the reason for the this error. 

Are these files not closed?? do i have to do IndexWriter.commit() in the loop to close these open files??