You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Sergiusz Urbaniak <se...@gmail.com> on 2013/05/15 17:50:04 UTC
Fwd: Deadlock in DocumentsWriterFlushControl
Hi all,
We have an obvious deadlock between a "MaybeRefreshIndexJob" thread
calling ReferenceManager.maybeRefresh(ReferenceManager.java:204) and a
"RebuildIndexJob" thread calling
IndexWriter.deleteAll(IndexWriter.java:2065).
Lucene tries to flush in the "MaybeRefreshIndexJob" thread while
simultaneously waiting for the flush in the "RebuildIndexJob" thread
deadlocked by the intrinsic lock on the IndexWriter instance.
May this be a bug or is it wrong usage? We are not explicitly locking
on the IndexWriter instance (following the documentation).
Thread-Dump:
"Thread - 1" daemon prio=10 tid=0x00007f8fe4006000 nid=0x1ac2 waiting
for monitor entry [0x00007f8fa7bf7000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.lucene.index.IndexWriter.useCompoundFile(IndexWriter.java:2223)
- waiting to lock <0x00000000f1c00438> (a
org.apache.lucene.index.IndexWriter)
at
org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:563)
at
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:533)
at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:422)
at
org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:559)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:365)
- locked <0x00000000f1c007d0> (a java.lang.Object)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:270)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:245)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:235)
at
org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:170)
at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:118)
at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:58)
at
org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:155)
at
org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:204)
at jobs.MaybeRefreshIndexJob.timeout(MaybeRefreshIndexJob.java:47)
"Thread - 2" prio=10 tid=0x00007f903000a000 nid=0x1a38 in
Object.wait() [0x00007f9037dd6000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000f1c0c240> (a
org.apache.lucene.index.DocumentsWriterFlushControl)
at java.lang.Object.wait(Object.java:503)
at
org.apache.lucene.index.DocumentsWriterFlushControl.waitForFlush(DocumentsWriterFlushControl.java:245)
- locked <0x00000000f1c0c240> (a
org.apache.lucene.index.DocumentsWriterFlushControl)
at org.apache.lucene.index.DocumentsWriter.abort(DocumentsWriter.java:235)
- locked <0x00000000f1c05370> (a
org.apache.lucene.index.DocumentsWriter)
at org.apache.lucene.index.IndexWriter.deleteAll(IndexWriter.java:2065)
- locked <0x00000000f1c00438> (a org.apache.lucene.index.IndexWriter)
at jobs.RebuildIndexJob.buildIndex(RebuildIndexJob.java:102)
Thanks for any help,
Sergiusz
Re: Deadlock in DocumentsWriterFlushControl
Posted by Sergiusz Urbaniak <se...@gmail.com>.
Hi Simon,
Thanks for the quick response! I just created
https://issues.apache.org/jira/browse/LUCENE-5002.
Sergiusz
On Wed, May 15, 2013 at 11:49 PM, Simon Willnauer
<si...@gmail.com> wrote:
> This seems like a bug caused due to the fact that we moved the CFS
> building into DWPT. Can you open an issue for this?
>
> simon
>
> On Wed, May 15, 2013 at 5:50 PM, Sergiusz Urbaniak
> <se...@gmail.com> wrote:
>> Hi all,
>>
>> We have an obvious deadlock between a "MaybeRefreshIndexJob" thread
>> calling ReferenceManager.maybeRefresh(ReferenceManager.java:204) and a
>> "RebuildIndexJob" thread calling
>> IndexWriter.deleteAll(IndexWriter.java:2065).
>>
>> Lucene tries to flush in the "MaybeRefreshIndexJob" thread while
>> simultaneously waiting for the flush in the "RebuildIndexJob" thread
>> deadlocked by the intrinsic lock on the IndexWriter instance.
>>
>> May this be a bug or is it wrong usage? We are not explicitly locking
>> on the IndexWriter instance (following the documentation).
>>
>> Thread-Dump:
>>
>> "Thread - 1" daemon prio=10 tid=0x00007f8fe4006000 nid=0x1ac2 waiting
>> for monitor entry [0x00007f8fa7bf7000]
>> java.lang.Thread.State: BLOCKED (on object monitor)
>> at
>> org.apache.lucene.index.IndexWriter.useCompoundFile(IndexWriter.java:2223)
>> - waiting to lock <0x00000000f1c00438> (a
>> org.apache.lucene.index.IndexWriter)
>> at
>> org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:563)
>> at
>> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:533)
>> at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:422)
>> at
>> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:559)
>> at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:365)
>> - locked <0x00000000f1c007d0> (a java.lang.Object)
>> at
>> org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:270)
>> at
>> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:245)
>> at
>> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:235)
>> at
>> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:170)
>> at
>> org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:118)
>> at
>> org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:58)
>> at
>> org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:155)
>> at
>> org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:204)
>> at jobs.MaybeRefreshIndexJob.timeout(MaybeRefreshIndexJob.java:47)
>>
>> "Thread - 2" prio=10 tid=0x00007f903000a000 nid=0x1a38 in
>> Object.wait() [0x00007f9037dd6000]
>> java.lang.Thread.State: WAITING (on object monitor)
>> at java.lang.Object.wait(Native Method)
>> - waiting on <0x00000000f1c0c240> (a
>> org.apache.lucene.index.DocumentsWriterFlushControl)
>> at java.lang.Object.wait(Object.java:503)
>> at
>> org.apache.lucene.index.DocumentsWriterFlushControl.waitForFlush(DocumentsWriterFlushControl.java:245)
>> - locked <0x00000000f1c0c240> (a
>> org.apache.lucene.index.DocumentsWriterFlushControl)
>> at org.apache.lucene.index.DocumentsWriter.abort(DocumentsWriter.java:235)
>> - locked <0x00000000f1c05370> (a
>> org.apache.lucene.index.DocumentsWriter)
>> at org.apache.lucene.index.IndexWriter.deleteAll(IndexWriter.java:2065)
>> - locked <0x00000000f1c00438> (a org.apache.lucene.index.IndexWriter)
>> at jobs.RebuildIndexJob.buildIndex(RebuildIndexJob.java:102)
>>
>> Thanks for any help,
>> Sergiusz
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Deadlock in DocumentsWriterFlushControl
Posted by Simon Willnauer <si...@gmail.com>.
This seems like a bug caused due to the fact that we moved the CFS
building into DWPT. Can you open an issue for this?
simon
On Wed, May 15, 2013 at 5:50 PM, Sergiusz Urbaniak
<se...@gmail.com> wrote:
> Hi all,
>
> We have an obvious deadlock between a "MaybeRefreshIndexJob" thread
> calling ReferenceManager.maybeRefresh(ReferenceManager.java:204) and a
> "RebuildIndexJob" thread calling
> IndexWriter.deleteAll(IndexWriter.java:2065).
>
> Lucene tries to flush in the "MaybeRefreshIndexJob" thread while
> simultaneously waiting for the flush in the "RebuildIndexJob" thread
> deadlocked by the intrinsic lock on the IndexWriter instance.
>
> May this be a bug or is it wrong usage? We are not explicitly locking
> on the IndexWriter instance (following the documentation).
>
> Thread-Dump:
>
> "Thread - 1" daemon prio=10 tid=0x00007f8fe4006000 nid=0x1ac2 waiting
> for monitor entry [0x00007f8fa7bf7000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.lucene.index.IndexWriter.useCompoundFile(IndexWriter.java:2223)
> - waiting to lock <0x00000000f1c00438> (a
> org.apache.lucene.index.IndexWriter)
> at
> org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:563)
> at
> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:533)
> at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:422)
> at
> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:559)
> at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:365)
> - locked <0x00000000f1c007d0> (a java.lang.Object)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:270)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:245)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:235)
> at
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:170)
> at
> org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:118)
> at
> org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:58)
> at
> org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:155)
> at
> org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:204)
> at jobs.MaybeRefreshIndexJob.timeout(MaybeRefreshIndexJob.java:47)
>
> "Thread - 2" prio=10 tid=0x00007f903000a000 nid=0x1a38 in
> Object.wait() [0x00007f9037dd6000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000f1c0c240> (a
> org.apache.lucene.index.DocumentsWriterFlushControl)
> at java.lang.Object.wait(Object.java:503)
> at
> org.apache.lucene.index.DocumentsWriterFlushControl.waitForFlush(DocumentsWriterFlushControl.java:245)
> - locked <0x00000000f1c0c240> (a
> org.apache.lucene.index.DocumentsWriterFlushControl)
> at org.apache.lucene.index.DocumentsWriter.abort(DocumentsWriter.java:235)
> - locked <0x00000000f1c05370> (a
> org.apache.lucene.index.DocumentsWriter)
> at org.apache.lucene.index.IndexWriter.deleteAll(IndexWriter.java:2065)
> - locked <0x00000000f1c00438> (a org.apache.lucene.index.IndexWriter)
> at jobs.RebuildIndexJob.buildIndex(RebuildIndexJob.java:102)
>
> Thanks for any help,
> Sergiusz
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org