Ask about Lucene/Core/Index DocumentsWriter

[yuncheng lu <lu...@gmail.com>]

When i check the code lucene/core DocumentsWriter.preUpdate code. I see the
flushControl is used when thread is Stalled.
When we have a lot of documents write into disk which is SSD. We monitored
that the thread is all in flush, and request continuously use addDocument
which can go into preUpdate check, it make more flush request into IO
queue. As this situation, it make all request blocked, and disk util is no
full of usage. So may be this optimization can take more condition into
consideration?


Thanks & Best Regards
Yuncheng Lu

Re: Ask about Lucene/Core/Index DocumentsWriter

[Michael McCandless <lu...@mikemccandless.com>]

Can you try increasing your IndexWriter.setRAMBufferSizeMB?  That flush
control logic will block incoming threads if the number of bytes trying to
flush to disk is too large relative to your RAM buffer.

Mike McCandless

http://blog.mikemccandless.com


On Mon, Mar 18, 2019 at 2:30 PM yuncheng lu <lu...@gmail.com> wrote:

> When i check the code lucene/core DocumentsWriter.preUpdate code. I see the
> flushControl is used when thread is Stalled.
> When we have a lot of documents write into disk which is SSD. We monitored
> that the thread is all in flush, and request continuously use addDocument
> which can go into preUpdate check, it make more flush request into IO
> queue. As this situation, it make all request blocked, and disk util is no
> full of usage. So may be this optimization can take more condition into
> consideration?
>
>
> Thanks & Best Regards
> Yuncheng Lu
>