You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Chengdu Huang <ch...@patterninsight.com> on 2009/07/28 06:27:28 UTC
deadlock in indexing
Hi,
I have an application in which documents are added upon receiving a
user request and a background thread is needed to remove old
documents. I have an IndexWriter opened on a Directory that adds
documents and commits but never closes. The background thread that
removes documents uses the same instance of IndexWriter. So the code
looks like
// Thread to add document:
synchronized(writer) {
try {
Document doc = new Document();
doc.add();
...
writer.commit();
} catch (Exception e) {
writer.rollback();
}
}
Now looks like I run into some kind of deadlock here even *WITHOUT*
the background thread of removing documents. The symptom is that the
whole java process is on sleeping state and jstack shows that the
thread to add document is blocked on waiting an object. Unfortunately
I'm unable to reproduce this in unittests.
My guess is that the outer synchronized(writer) {} block is causing
the problem, but can't figure out why. Any idea?
Chengdu
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: deadlock in indexing
Posted by Chengdu Huang <ch...@patterninsight.com>.
Thanks, Mike.
Can you elaborate a bit more on why this would cause deadlock? Thanks.
Chengdu
On Tue, Jul 28, 2009 at 10:48 AM, Michael
McCandless<lu...@mikemccandless.com> wrote:
> This can in fact result in deadlock; you should sync on your own Object instead.
>
> Mike
>
> On Tue, Jul 28, 2009 at 12:27 AM, Chengdu
> Huang<ch...@patterninsight.com> wrote:
>> Hi,
>>
>> I have an application in which documents are added upon receiving a
>> user request and a background thread is needed to remove old
>> documents. I have an IndexWriter opened on a Directory that adds
>> documents and commits but never closes. The background thread that
>> removes documents uses the same instance of IndexWriter. So the code
>> looks like
>>
>> // Thread to add document:
>> synchronized(writer) {
>> try {
>> Document doc = new Document();
>> doc.add();
>> ...
>> writer.commit();
>> } catch (Exception e) {
>> writer.rollback();
>> }
>> }
>>
>> Now looks like I run into some kind of deadlock here even *WITHOUT*
>> the background thread of removing documents. The symptom is that the
>> whole java process is on sleeping state and jstack shows that the
>> thread to add document is blocked on waiting an object. Unfortunately
>> I'm unable to reproduce this in unittests.
>>
>> My guess is that the outer synchronized(writer) {} block is causing
>> the problem, but can't figure out why. Any idea?
>>
>> Chengdu
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: deadlock in indexing
Posted by Michael McCandless <lu...@mikemccandless.com>.
This can in fact result in deadlock; you should sync on your own Object instead.
Mike
On Tue, Jul 28, 2009 at 12:27 AM, Chengdu
Huang<ch...@patterninsight.com> wrote:
> Hi,
>
> I have an application in which documents are added upon receiving a
> user request and a background thread is needed to remove old
> documents. I have an IndexWriter opened on a Directory that adds
> documents and commits but never closes. The background thread that
> removes documents uses the same instance of IndexWriter. So the code
> looks like
>
> // Thread to add document:
> synchronized(writer) {
> try {
> Document doc = new Document();
> doc.add();
> ...
> writer.commit();
> } catch (Exception e) {
> writer.rollback();
> }
> }
>
> Now looks like I run into some kind of deadlock here even *WITHOUT*
> the background thread of removing documents. The symptom is that the
> whole java process is on sleeping state and jstack shows that the
> thread to add document is blocked on waiting an object. Unfortunately
> I'm unable to reproduce this in unittests.
>
> My guess is that the outer synchronized(writer) {} block is causing
> the problem, but can't figure out why. Any idea?
>
> Chengdu
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: deadlock in indexing
Posted by Simon Willnauer <si...@googlemail.com>.
On Tue, Jul 28, 2009 at 11:32 PM, Chengdu
Huang<ch...@patterninsight.com> wrote:
> Thanks Uwe!
>
> I was looking at javadoc of IndexWriter in Lucene 2.4.0 and didn't
> find anything about thread safety. The wiki page does have that
> though. Thanks.
http://wiki.apache.org/lucene-java/LuceneFAQ#head-c0e6aac25806df3a9402b398e7679987b8e78cf5
Is the IndexWriter class, and especially the method
addIndexes(Directory[]) thread safe?
Yes, IndexWriter.addIndexes(Directory[]) method is thread safe (it is
a synchronized method). IndexWriter in general is thread safe, i.e.
you should use the same IndexWriter object from all of your threads.
Actually it's impossible to use more than one IndexWriter for the same
index directory, as this will lead to an exception trying to create
the lock file.
But I agree that the first place I would look for such an information
is the javadoc. We should add a note to javadoc that the indexwriter
is in general threadsafe. Stuff like that belongs into the module
documentation in my eyes. I will raise an issue.
simon
>
> Chengdu
>
> On Tue, Jul 28, 2009 at 2:20 PM, Uwe Schindler<uw...@thetaphi.de> wrote:
>>> By "IndexWriter is threadsafe" do you mean that I can have to two
>>> threads, one calls IndexWriter.addDocument(), the other calls
>>> IndexWriter.deleteDocuments() & IndexWriter.optimize(), without any
>>> synchronization?
>>
>> Exactly.
>>
>>> For the deadlock, I also think it has something to do with merging.
>>> Below are stacktraces of some of the relevant threads:
>>
>>
>> ...
>>
>>> It seems that the last thread has locked the ConcurrentMergeScheduler,
>>> but then wait on it too. Is that the problem?
>>
>> The problem is that IndexWriter does its own synchronization. By adding
>> extra synchronized() blocks around these code parts you break the complete
>> locking logic inside IW. You can fix this by either:
>>
>> a) use another object as monitor (not the IndexWriter), if your own code
>> needs some locking.
>> b) remove extra locking at all, as IndexWriter is threadsafe (which is
>> explained n IndexReaders/Writers JavaDocs and Wiki)
>>
>>> Chengdu
>>>
>>> On Tue, Jul 28, 2009 at 12:28 AM, Simon
>>> Willnauer<si...@googlemail.com> wrote:
>>> > I can not help you to figure out your exact problem but you can use an
>>> > the same indexwriter instance without synchronization. IndexWriter is
>>> > threadsafe so you synchronized block seems obsolet.
>>> > I could imagine that there is a backgroud merge going on while you try
>>> > to access the critical section ( you synchronized block) which could
>>> > block you code for a while until the merge has finished. Can you
>>> > figure out if a merger-thread is running? The threads name should be
>>> > set to something like "Lucene Merge Thread #n"
>>> >
>>> > simon
>>> >
>>> > On Tue, Jul 28, 2009 at 6:27 AM, Chengdu
>>> > Huang<ch...@patterninsight.com> wrote:
>>> >> Hi,
>>> >>
>>> >> I have an application in which documents are added upon receiving a
>>> >> user request and a background thread is needed to remove old
>>> >> documents. I have an IndexWriter opened on a Directory that adds
>>> >> documents and commits but never closes. The background thread that
>>> >> removes documents uses the same instance of IndexWriter. So the code
>>> >> looks like
>>> >>
>>> >> // Thread to add document:
>>> >> synchronized(writer) {
>>> >> try {
>>> >> Document doc = new Document();
>>> >> doc.add();
>>> >> ...
>>> >> writer.commit();
>>> >> } catch (Exception e) {
>>> >> writer.rollback();
>>> >> }
>>> >> }
>>> >>
>>> >> Now looks like I run into some kind of deadlock here even *WITHOUT*
>>> >> the background thread of removing documents. The symptom is that the
>>> >> whole java process is on sleeping state and jstack shows that the
>>> >> thread to add document is blocked on waiting an object. Unfortunately
>>> >> I'm unable to reproduce this in unittests.
>>> >>
>>> >> My guess is that the outer synchronized(writer) {} block is causing
>>> >> the problem, but can't figure out why. Any idea?
>>> >>
>>> >> Chengdu
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> >> For additional commands, e-mail: java-user-help@lucene.apache.org
>>> >>
>>> >>
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: deadlock in indexing
Posted by Chengdu Huang <ch...@patterninsight.com>.
Thanks Uwe!
I was looking at javadoc of IndexWriter in Lucene 2.4.0 and didn't
find anything about thread safety. The wiki page does have that
though. Thanks.
Chengdu
On Tue, Jul 28, 2009 at 2:20 PM, Uwe Schindler<uw...@thetaphi.de> wrote:
>> By "IndexWriter is threadsafe" do you mean that I can have to two
>> threads, one calls IndexWriter.addDocument(), the other calls
>> IndexWriter.deleteDocuments() & IndexWriter.optimize(), without any
>> synchronization?
>
> Exactly.
>
>> For the deadlock, I also think it has something to do with merging.
>> Below are stacktraces of some of the relevant threads:
>
>
> ...
>
>> It seems that the last thread has locked the ConcurrentMergeScheduler,
>> but then wait on it too. Is that the problem?
>
> The problem is that IndexWriter does its own synchronization. By adding
> extra synchronized() blocks around these code parts you break the complete
> locking logic inside IW. You can fix this by either:
>
> a) use another object as monitor (not the IndexWriter), if your own code
> needs some locking.
> b) remove extra locking at all, as IndexWriter is threadsafe (which is
> explained n IndexReaders/Writers JavaDocs and Wiki)
>
>> Chengdu
>>
>> On Tue, Jul 28, 2009 at 12:28 AM, Simon
>> Willnauer<si...@googlemail.com> wrote:
>> > I can not help you to figure out your exact problem but you can use an
>> > the same indexwriter instance without synchronization. IndexWriter is
>> > threadsafe so you synchronized block seems obsolet.
>> > I could imagine that there is a backgroud merge going on while you try
>> > to access the critical section ( you synchronized block) which could
>> > block you code for a while until the merge has finished. Can you
>> > figure out if a merger-thread is running? The threads name should be
>> > set to something like "Lucene Merge Thread #n"
>> >
>> > simon
>> >
>> > On Tue, Jul 28, 2009 at 6:27 AM, Chengdu
>> > Huang<ch...@patterninsight.com> wrote:
>> >> Hi,
>> >>
>> >> I have an application in which documents are added upon receiving a
>> >> user request and a background thread is needed to remove old
>> >> documents. I have an IndexWriter opened on a Directory that adds
>> >> documents and commits but never closes. The background thread that
>> >> removes documents uses the same instance of IndexWriter. So the code
>> >> looks like
>> >>
>> >> // Thread to add document:
>> >> synchronized(writer) {
>> >> try {
>> >> Document doc = new Document();
>> >> doc.add();
>> >> ...
>> >> writer.commit();
>> >> } catch (Exception e) {
>> >> writer.rollback();
>> >> }
>> >> }
>> >>
>> >> Now looks like I run into some kind of deadlock here even *WITHOUT*
>> >> the background thread of removing documents. The symptom is that the
>> >> whole java process is on sleeping state and jstack shows that the
>> >> thread to add document is blocked on waiting an object. Unfortunately
>> >> I'm unable to reproduce this in unittests.
>> >>
>> >> My guess is that the outer synchronized(writer) {} block is causing
>> >> the problem, but can't figure out why. Any idea?
>> >>
>> >> Chengdu
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: java-user-help@lucene.apache.org
>> >>
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: deadlock in indexing
Posted by Uwe Schindler <uw...@thetaphi.de>.
> By "IndexWriter is threadsafe" do you mean that I can have to two
> threads, one calls IndexWriter.addDocument(), the other calls
> IndexWriter.deleteDocuments() & IndexWriter.optimize(), without any
> synchronization?
Exactly.
> For the deadlock, I also think it has something to do with merging.
> Below are stacktraces of some of the relevant threads:
...
> It seems that the last thread has locked the ConcurrentMergeScheduler,
> but then wait on it too. Is that the problem?
The problem is that IndexWriter does its own synchronization. By adding
extra synchronized() blocks around these code parts you break the complete
locking logic inside IW. You can fix this by either:
a) use another object as monitor (not the IndexWriter), if your own code
needs some locking.
b) remove extra locking at all, as IndexWriter is threadsafe (which is
explained n IndexReaders/Writers JavaDocs and Wiki)
> Chengdu
>
> On Tue, Jul 28, 2009 at 12:28 AM, Simon
> Willnauer<si...@googlemail.com> wrote:
> > I can not help you to figure out your exact problem but you can use an
> > the same indexwriter instance without synchronization. IndexWriter is
> > threadsafe so you synchronized block seems obsolet.
> > I could imagine that there is a backgroud merge going on while you try
> > to access the critical section ( you synchronized block) which could
> > block you code for a while until the merge has finished. Can you
> > figure out if a merger-thread is running? The threads name should be
> > set to something like "Lucene Merge Thread #n"
> >
> > simon
> >
> > On Tue, Jul 28, 2009 at 6:27 AM, Chengdu
> > Huang<ch...@patterninsight.com> wrote:
> >> Hi,
> >>
> >> I have an application in which documents are added upon receiving a
> >> user request and a background thread is needed to remove old
> >> documents. I have an IndexWriter opened on a Directory that adds
> >> documents and commits but never closes. The background thread that
> >> removes documents uses the same instance of IndexWriter. So the code
> >> looks like
> >>
> >> // Thread to add document:
> >> synchronized(writer) {
> >> try {
> >> Document doc = new Document();
> >> doc.add();
> >> ...
> >> writer.commit();
> >> } catch (Exception e) {
> >> writer.rollback();
> >> }
> >> }
> >>
> >> Now looks like I run into some kind of deadlock here even *WITHOUT*
> >> the background thread of removing documents. The symptom is that the
> >> whole java process is on sleeping state and jstack shows that the
> >> thread to add document is blocked on waiting an object. Unfortunately
> >> I'm unable to reproduce this in unittests.
> >>
> >> My guess is that the outer synchronized(writer) {} block is causing
> >> the problem, but can't figure out why. Any idea?
> >>
> >> Chengdu
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: deadlock in indexing
Posted by Michael McCandless <lu...@mikemccandless.com>.
On Tue, Jul 28, 2009 at 5:10 PM, Chengdu
Huang<ch...@patterninsight.com> wrote:
> For the deadlock, I also think it has something to do with merging.
Right, what's happening here is the ConcurrentMergeScheduler, which
runs BG threads to do merging, has a max thread count (default 3) that
it's allowed to run at once.
Because you had sync'd on writer, each of these threads was blocked on
trying to start its merge (they invoke a synchronized method on
IndexWriter).
Once the max number of threads is outstanding,
ConcurrentMergeScheduler forcefully idles the main threads, waiting
for the BG thread count to drop back down, which in your case will
never happen.
Synchronizing on your own private Object will fix it.
And I agree: we should update the javadocs to spell out IndexWriter's
thread safety, including "you shouldn't synchronize on IndexWriter
externally". Looks like Simon just opened the issue for this --
thanks!
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: deadlock in indexing
Posted by Chengdu Huang <ch...@patterninsight.com>.
Hi Simon,
By "IndexWriter is threadsafe" do you mean that I can have to two
threads, one calls IndexWriter.addDocument(), the other calls
IndexWriter.deleteDocuments() & IndexWriter.optimize(), without any
synchronization?
For the deadlock, I also think it has something to do with merging.
Below are stacktraces of some of the relevant threads:
"Lucene Merge Thread #2" daemon prio=10 tid=0x086be000 nid=0x4a9
waiting for monitor entry [0x52dad000..0x52dadf20]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3964)
- waiting to lock <0x58a745d0> (a org.apache.lucene.index.IndexWriter)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3872)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
"Lucene Merge Thread #1" daemon prio=10 tid=0x517c3400 nid=0x4a7
waiting for monitor entry [0x52dfe000..0x52dfeea0]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3964)
- waiting to lock <0x58a745d0> (a org.apache.lucene.index.IndexWriter)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3872)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
"Lucene Merge Thread #0" daemon prio=10 tid=0x08714c00 nid=0x4a1
waiting for monitor entry [0x52ff7000..0x52ff7e20]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3964)
- waiting to lock <0x58a745d0> (a org.apache.lucene.index.IndexWriter)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3872)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
"Add Document Thread" prio=10 tid=0x084ef000 nid=0x4985 in
Object.wait() [0x5315c000..0x5315d020]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x58af4548> (a
org.apache.lucene.index.ConcurrentMergeScheduler)
at java.lang.Object.wait(Object.java:485)
at org.apache.lucene.index.ConcurrentMergeScheduler.merge(ConcurrentMergeScheduler.java:182)
- locked <0x58af4548> (a
org.apache.lucene.index.ConcurrentMergeScheduler)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2402)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2397)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2393)
at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3443)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1922)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1880)
at com.patterninsight.indexing.IndexTask.indexDocuments(IndexTask.java:143)
- locked <0x58b422c0> (a org.apache.lucene.index.IndexWriter)
It seems that the last thread has locked the ConcurrentMergeScheduler,
but then wait on it too. Is that the problem?
Chengdu
On Tue, Jul 28, 2009 at 12:28 AM, Simon
Willnauer<si...@googlemail.com> wrote:
> I can not help you to figure out your exact problem but you can use an
> the same indexwriter instance without synchronization. IndexWriter is
> threadsafe so you synchronized block seems obsolet.
> I could imagine that there is a backgroud merge going on while you try
> to access the critical section ( you synchronized block) which could
> block you code for a while until the merge has finished. Can you
> figure out if a merger-thread is running? The threads name should be
> set to something like "Lucene Merge Thread #n"
>
> simon
>
> On Tue, Jul 28, 2009 at 6:27 AM, Chengdu
> Huang<ch...@patterninsight.com> wrote:
>> Hi,
>>
>> I have an application in which documents are added upon receiving a
>> user request and a background thread is needed to remove old
>> documents. I have an IndexWriter opened on a Directory that adds
>> documents and commits but never closes. The background thread that
>> removes documents uses the same instance of IndexWriter. So the code
>> looks like
>>
>> // Thread to add document:
>> synchronized(writer) {
>> try {
>> Document doc = new Document();
>> doc.add();
>> ...
>> writer.commit();
>> } catch (Exception e) {
>> writer.rollback();
>> }
>> }
>>
>> Now looks like I run into some kind of deadlock here even *WITHOUT*
>> the background thread of removing documents. The symptom is that the
>> whole java process is on sleeping state and jstack shows that the
>> thread to add document is blocked on waiting an object. Unfortunately
>> I'm unable to reproduce this in unittests.
>>
>> My guess is that the outer synchronized(writer) {} block is causing
>> the problem, but can't figure out why. Any idea?
>>
>> Chengdu
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: deadlock in indexing
Posted by Simon Willnauer <si...@googlemail.com>.
I can not help you to figure out your exact problem but you can use an
the same indexwriter instance without synchronization. IndexWriter is
threadsafe so you synchronized block seems obsolet.
I could imagine that there is a backgroud merge going on while you try
to access the critical section ( you synchronized block) which could
block you code for a while until the merge has finished. Can you
figure out if a merger-thread is running? The threads name should be
set to something like "Lucene Merge Thread #n"
simon
On Tue, Jul 28, 2009 at 6:27 AM, Chengdu
Huang<ch...@patterninsight.com> wrote:
> Hi,
>
> I have an application in which documents are added upon receiving a
> user request and a background thread is needed to remove old
> documents. I have an IndexWriter opened on a Directory that adds
> documents and commits but never closes. The background thread that
> removes documents uses the same instance of IndexWriter. So the code
> looks like
>
> // Thread to add document:
> synchronized(writer) {
> try {
> Document doc = new Document();
> doc.add();
> ...
> writer.commit();
> } catch (Exception e) {
> writer.rollback();
> }
> }
>
> Now looks like I run into some kind of deadlock here even *WITHOUT*
> the background thread of removing documents. The symptom is that the
> whole java process is on sleeping state and jstack shows that the
> thread to add document is blocked on waiting an object. Unfortunately
> I'm unable to reproduce this in unittests.
>
> My guess is that the outer synchronized(writer) {} block is causing
> the problem, but can't figure out why. Any idea?
>
> Chengdu
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org