You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Michael McCandless <lu...@mikemccandless.com> on 2018/12/01 16:25:29 UTC
Re: Race condition between IndexWriter.commit and IndexWriter.close
I think if you call commit and close concurrently the results are undefined
and so this is acceptable.
Mike
On Thu, Nov 29, 2018 at 5:53 AM Boris Petrov <bo...@profuzdigital.com>
wrote:
> Hi all,
>
> We're getting the following exception:
>
> java.lang.IllegalStateException: cannot close: prepareCommit was already
> called with no corresponding call to commit
> at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1025)
> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1078)
> ...
>
> We are only calling "commit" on IndexWriter. By reading the code I can
> see that if you call IndexWriter.commit in parallel to
> IndexWriter.close, it is possible to get this exception. More
> specifically, after setting "IndexWriter.pendingCommit" on line 4779
> (this is using Lucene 7.5.0) and before setting it to "null" on line
> 4793 this problem could happen.
>
> Is this by design or is it a bug?
>
> Thanks,
>
> Boris Petrov
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
> --
Mike McCandless
http://blog.mikemccandless.com
Re: Race condition between IndexWriter.commit and IndexWriter.close
Posted by Boris Petrov <bo...@profuzdigital.com>.
I just started doing that and noticed that in "close"'s documentation it
says:
NOTE: You must ensure no other threads are still making changes at the
same time that this method is invoked.
Which I guess is what I've missed and should be enough. Thanks for the
support! :)
On 12/5/18 12:52 PM, Michael McCandless wrote:
> Yeah I agree we should fix the javadocs to say that you should not try
> to close and commit concurrently from different threads.
>
> Wanna open a quick PR or issue with a patch?
>
> Thanks,
>
> Mike
>
> On Wed, Dec 5, 2018, 4:06 AM Boris Petrov <boris@profuzdigital.com
> <ma...@profuzdigital.com> wrote:
>
> So you're saying that this race-condition is OK? Nowhere in the
> documentation does it say that these two calls should be
> synchronized...
> at least that must be fixed. :)
>
> On 12/1/18 6:25 PM, Michael McCandless wrote:
> > I think if you call commit and close concurrently the results
> are undefined
> > and so this is acceptable.
> >
> > Mike
> >
> > On Thu, Nov 29, 2018 at 5:53 AM Boris Petrov
> <boris@profuzdigital.com <ma...@profuzdigital.com>>
> > wrote:
> >
> >> Hi all,
> >>
> >> We're getting the following exception:
> >>
> >> java.lang.IllegalStateException: cannot close: prepareCommit
> was already
> >> called with no corresponding call to commit
> >> at
> org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1025)
> >> at
> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1078)
> >> ...
> >>
> >> We are only calling "commit" on IndexWriter. By reading the
> code I can
> >> see that if you call IndexWriter.commit in parallel to
> >> IndexWriter.close, it is possible to get this exception. More
> >> specifically, after setting "IndexWriter.pendingCommit" on line
> 4779
> >> (this is using Lucene 7.5.0) and before setting it to "null" on
> line
> >> 4793 this problem could happen.
> >>
> >> Is this by design or is it a bug?
> >>
> >> Thanks,
> >>
> >> Boris Petrov
> >>
> >>
> >>
> >>
> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> <ma...@lucene.apache.org>
> >> For additional commands, e-mail:
> java-user-help@lucene.apache.org
> <ma...@lucene.apache.org>
> >>
> >> --
> > Mike McCandless
> >
> > http://blog.mikemccandless.com
> >
>
Re: Race condition between IndexWriter.commit and IndexWriter.close
Posted by Michael McCandless <lu...@mikemccandless.com>.
Yeah I agree we should fix the javadocs to say that you should not try to
close and commit concurrently from different threads.
Wanna open a quick PR or issue with a patch?
Thanks,
Mike
On Wed, Dec 5, 2018, 4:06 AM Boris Petrov <boris@profuzdigital.com wrote:
> So you're saying that this race-condition is OK? Nowhere in the
> documentation does it say that these two calls should be synchronized...
> at least that must be fixed. :)
>
> On 12/1/18 6:25 PM, Michael McCandless wrote:
> > I think if you call commit and close concurrently the results are
> undefined
> > and so this is acceptable.
> >
> > Mike
> >
> > On Thu, Nov 29, 2018 at 5:53 AM Boris Petrov <bo...@profuzdigital.com>
> > wrote:
> >
> >> Hi all,
> >>
> >> We're getting the following exception:
> >>
> >> java.lang.IllegalStateException: cannot close: prepareCommit was already
> >> called with no corresponding call to commit
> >> at
> org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1025)
> >> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1078)
> >> ...
> >>
> >> We are only calling "commit" on IndexWriter. By reading the code I can
> >> see that if you call IndexWriter.commit in parallel to
> >> IndexWriter.close, it is possible to get this exception. More
> >> specifically, after setting "IndexWriter.pendingCommit" on line 4779
> >> (this is using Lucene 7.5.0) and before setting it to "null" on line
> >> 4793 this problem could happen.
> >>
> >> Is this by design or is it a bug?
> >>
> >> Thanks,
> >>
> >> Boris Petrov
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >> --
> > Mike McCandless
> >
> > http://blog.mikemccandless.com
> >
>
Re: Race condition between IndexWriter.commit and IndexWriter.close
Posted by Boris Petrov <bo...@profuzdigital.com>.
So you're saying that this race-condition is OK? Nowhere in the
documentation does it say that these two calls should be synchronized...
at least that must be fixed. :)
On 12/1/18 6:25 PM, Michael McCandless wrote:
> I think if you call commit and close concurrently the results are undefined
> and so this is acceptable.
>
> Mike
>
> On Thu, Nov 29, 2018 at 5:53 AM Boris Petrov <bo...@profuzdigital.com>
> wrote:
>
>> Hi all,
>>
>> We're getting the following exception:
>>
>> java.lang.IllegalStateException: cannot close: prepareCommit was already
>> called with no corresponding call to commit
>> at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1025)
>> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1078)
>> ...
>>
>> We are only calling "commit" on IndexWriter. By reading the code I can
>> see that if you call IndexWriter.commit in parallel to
>> IndexWriter.close, it is possible to get this exception. More
>> specifically, after setting "IndexWriter.pendingCommit" on line 4779
>> (this is using Lucene 7.5.0) and before setting it to "null" on line
>> 4793 this problem could happen.
>>
>> Is this by design or is it a bug?
>>
>> Thanks,
>>
>> Boris Petrov
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>> --
> Mike McCandless
>
> http://blog.mikemccandless.com
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org