[GitHub] [lucenenet] zhangbaodan commented on issue #720: Frequent Commit operations consume a lot of CPU

[GitBox <gi...@apache.org>]

zhangbaodan commented on issue #720:
URL: https://github.com/apache/lucenenet/issues/720#issuecomment-1293102257

   @nikcio @rclabo   Thank you very much for your timely reply, which really helped me a lot. By adopting your suggestions, my cpu consumption was reduced from 4% to less than 0.1%. This is really a huge improvement. Here's my solution:
   
   1. Every few seconds, retrieve file changes from the USN log and write them to IndexWriter via UpdateDocument, but do not immediately commit them to local disk.
   
   2. Create an IndexReader from the IndexWriter during each search. This ensures that the IndexReader can see the new data in the IndexWriter and the data on the disk. Here is the corresponding code :https://gist.github.com/mocobeta/4640263
   
   3. The timing of IndexWriter commits is important. Here I recommend that you commit every 10 minutes, and at the same time, you should record the location of the current USN log read. The next time the program is started, the USN will continue reading from the recorded location. This ensures that all changes read in the USN log are recorded in the index file.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org