Re: commit frequency guideline?

[Michael McCandless <lu...@mikemccandless.com>]

On Wed, Nov 30, 2016 at 9:37 AM, Rob Audenaerde
<ro...@gmail.com> wrote:
> Thanks for the quick reply!
>
>>What do you mean by "Lucene complain about too-many uncommitted docs"?
>
> --> good question, I was thoughtlessly echoing words from my colleague. I
> asked him and he said that it was about taking very long to commit and
> memory issues. So maybe this wasn't the best opening statement :)

Ahhh.  Well, fsync (which Lucene invokes on multiple files when you
call commit) is inherently a costly (in wall clock time) operation
since the OS and IO system may have to move many bytes to the
underlying durable storage.  But you should not hit memory issues.

> For the other part of the question: we need users to see the changed
> documents immediately, but I think we have this covered by using NRT Readers
> and the SearcherManager.
>
> Am I correct to conclude calling commit() is not necessary for finding
> recently changed documents?

That's correct: NRT readers give you fast visibility of changes, while
commit gives you durability.  The two are entirely orthogonal in
Lucene, as long as you don't use a single thread to call commit and
call refresh.

> I think we can then switch to a time based commit() where we just call
> commit every 5 minutes, in effect losing a maximum of 5 minutes of work
> (which we can mitigate in another way)
>  when the server somehow stops working.

Exactly.

> Thank you,

You're welcome!

Mike McCandless

http://blog.mikemccandless.com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org