Lucene Incremental Update to indexes

[ankum <an...@ashiana.com>]

Hi,

I did few searches around incremental updates, but found outdated posts, so
just want to ensure I use the correct solution with the latest and greatest
of Lucene API's and knowledge from experts.

---

I have a large index of documents (say 4 million, each with unique
application id's) - and I have need to frequently update this original
'main' index with 'delta' - for add, update, and delete. 

My strategy for updating the main index, for this delta index is (my test
code):

 - read the delta index (worst case 1 million documents)
 - each document in this delta index, will have an flag to identify
add/update/delete
 - for each such document call the appropriate API on the main index (opened
in OpenMode.APPEND) :
    - addDocument(doc)
    - updateDocument(new Term(ID, id), doc)
    - deleteDocuments(new Term(ID, id))
- do a writer.close() on the main index
- do DirectoryReader newReader = DirectoryReader.openIfChanged(mainReader) 
- if newReader above is != null, then I initialize the indexSearcher with
the newReader, and do mainReader.close (the old reader)


Does anyone feel above is best approach? Or it should be handled differently
(code issues, performance bottlenecks)

Thanks in advance !

    



--
View this message in context: http://lucene.472066.n3.nabble.com/Lucene-Incremental-Update-to-indexes-tp4177499.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org