You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Doug Sale (JIRA)" <ji...@apache.org> on 2009/02/19 20:22:02 UTC
[jira] Created: (LUCENE-1544) Deadlock:
IndexWriter.addIndexes(IndexReader[])
Deadlock: IndexWriter.addIndexes(IndexReader[])
-----------------------------------------------
Key: LUCENE-1544
URL: https://issues.apache.org/jira/browse/LUCENE-1544
Project: Lucene - Java
Issue Type: Bug
Components: Index
Affects Versions: 2.4
Environment: should be ubiquitous... but, Win XP on a MacBookPro (Intel) under Eclipse (Ganymede) using default compiler/runtime
Reporter: Doug Sale
A deadlock issue occurs under the following circumstances
- IndexWriter.autoCommit == true
- IndexWriter.directory contains multiple segments
- IndexWriter.AddIndex(IndexReader[]) is invoked
I put together a JUnit test that recreates the deadlock, which I've attached. It is the first test method, 'testAddIndexByIndexReader()'.
In a nutshell, here is what happens:
// 1) AddIndexes(IndexReader[]) acquires the write lock,
// then begins optimization of destination index (this is
// prior to adding any external segments).
//
// 2) Main thread starts a ConcurrentMergeScheduler.MergeThread
// to merge the 2 segments.
//
// 3) Merging thread tries to acquire the read lock at
// IndexWriter.blockAddIndexes(boolean) in
// IndexWriter.StartCommit(), but cannot as...
//
// 4) Main thread still holds the write lock, and is
// waiting for the IndexWriter.runningMerges data structure
// to be devoid of merges with their optimize flag
// set (IndexWriter.optimizeMergesPending()).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1544) Deadlock:
IndexWriter.addIndexes(IndexReader[])
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-1544:
---------------------------------------
Attachment: LUCENE-1544.patch
Patch attached.
I changed the up-front acquireWrite inside addIndexes(IndexReader[])
to an acquireRead followed by upgradeReadToWrite.
Once we remove autoCommit=true case we can greatly simplify IW -- I'm
looking forward to it!!
I plan to commit in a day or so, and back port to 2.4.1.
> Deadlock: IndexWriter.addIndexes(IndexReader[])
> -----------------------------------------------
>
> Key: LUCENE-1544
> URL: https://issues.apache.org/jira/browse/LUCENE-1544
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4
> Environment: should be ubiquitous... but, Win XP on a MacBookPro (Intel) under Eclipse (Ganymede) using default compiler/runtime
> Reporter: Doug Sale
> Assignee: Michael McCandless
> Fix For: 2.4.1, 2.9
>
> Attachments: LUCENE-1544.patch, TestAddIndexes.java
>
>
> A deadlock issue occurs under the following circumstances
> - IndexWriter.autoCommit == true
> - IndexWriter.directory contains multiple segments
> - IndexWriter.AddIndex(IndexReader[]) is invoked
> I put together a JUnit test that recreates the deadlock, which I've attached. It is the first test method, 'testAddIndexByIndexReader()'.
> In a nutshell, here is what happens:
> // 1) AddIndexes(IndexReader[]) acquires the write lock,
> // then begins optimization of destination index (this is
> // prior to adding any external segments).
> //
> // 2) Main thread starts a ConcurrentMergeScheduler.MergeThread
> // to merge the 2 segments.
> //
> // 3) Merging thread tries to acquire the read lock at
> // IndexWriter.blockAddIndexes(boolean) in
> // IndexWriter.StartCommit(), but cannot as...
> //
> // 4) Main thread still holds the write lock, and is
> // waiting for the IndexWriter.runningMerges data structure
> // to be devoid of merges with their optimize flag
> // set (IndexWriter.optimizeMergesPending()).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-1544) Deadlock:
IndexWriter.addIndexes(IndexReader[])
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless reassigned LUCENE-1544:
------------------------------------------
Assignee: Michael McCandless
> Deadlock: IndexWriter.addIndexes(IndexReader[])
> -----------------------------------------------
>
> Key: LUCENE-1544
> URL: https://issues.apache.org/jira/browse/LUCENE-1544
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4
> Environment: should be ubiquitous... but, Win XP on a MacBookPro (Intel) under Eclipse (Ganymede) using default compiler/runtime
> Reporter: Doug Sale
> Assignee: Michael McCandless
> Attachments: TestAddIndexes.java
>
>
> A deadlock issue occurs under the following circumstances
> - IndexWriter.autoCommit == true
> - IndexWriter.directory contains multiple segments
> - IndexWriter.AddIndex(IndexReader[]) is invoked
> I put together a JUnit test that recreates the deadlock, which I've attached. It is the first test method, 'testAddIndexByIndexReader()'.
> In a nutshell, here is what happens:
> // 1) AddIndexes(IndexReader[]) acquires the write lock,
> // then begins optimization of destination index (this is
> // prior to adding any external segments).
> //
> // 2) Main thread starts a ConcurrentMergeScheduler.MergeThread
> // to merge the 2 segments.
> //
> // 3) Merging thread tries to acquire the read lock at
> // IndexWriter.blockAddIndexes(boolean) in
> // IndexWriter.StartCommit(), but cannot as...
> //
> // 4) Main thread still holds the write lock, and is
> // waiting for the IndexWriter.runningMerges data structure
> // to be devoid of merges with their optimize flag
> // set (IndexWriter.optimizeMergesPending()).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1544) Deadlock:
IndexWriter.addIndexes(IndexReader[])
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-1544:
---------------------------------------
Fix Version/s: 2.9
2.4.1
> Deadlock: IndexWriter.addIndexes(IndexReader[])
> -----------------------------------------------
>
> Key: LUCENE-1544
> URL: https://issues.apache.org/jira/browse/LUCENE-1544
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4
> Environment: should be ubiquitous... but, Win XP on a MacBookPro (Intel) under Eclipse (Ganymede) using default compiler/runtime
> Reporter: Doug Sale
> Assignee: Michael McCandless
> Fix For: 2.4.1, 2.9
>
> Attachments: LUCENE-1544.patch, TestAddIndexes.java
>
>
> A deadlock issue occurs under the following circumstances
> - IndexWriter.autoCommit == true
> - IndexWriter.directory contains multiple segments
> - IndexWriter.AddIndex(IndexReader[]) is invoked
> I put together a JUnit test that recreates the deadlock, which I've attached. It is the first test method, 'testAddIndexByIndexReader()'.
> In a nutshell, here is what happens:
> // 1) AddIndexes(IndexReader[]) acquires the write lock,
> // then begins optimization of destination index (this is
> // prior to adding any external segments).
> //
> // 2) Main thread starts a ConcurrentMergeScheduler.MergeThread
> // to merge the 2 segments.
> //
> // 3) Merging thread tries to acquire the read lock at
> // IndexWriter.blockAddIndexes(boolean) in
> // IndexWriter.StartCommit(), but cannot as...
> //
> // 4) Main thread still holds the write lock, and is
> // waiting for the IndexWriter.runningMerges data structure
> // to be devoid of merges with their optimize flag
> // set (IndexWriter.optimizeMergesPending()).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1544) Deadlock:
IndexWriter.addIndexes(IndexReader[])
Posted by "Doug Sale (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Sale updated LUCENE-1544:
------------------------------
Attachment: TestAddIndexes.java
JUnit test.
method/testcase that recreates deadlock: testAddIndexesByIndexReader()
> Deadlock: IndexWriter.addIndexes(IndexReader[])
> -----------------------------------------------
>
> Key: LUCENE-1544
> URL: https://issues.apache.org/jira/browse/LUCENE-1544
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4
> Environment: should be ubiquitous... but, Win XP on a MacBookPro (Intel) under Eclipse (Ganymede) using default compiler/runtime
> Reporter: Doug Sale
> Attachments: TestAddIndexes.java
>
>
> A deadlock issue occurs under the following circumstances
> - IndexWriter.autoCommit == true
> - IndexWriter.directory contains multiple segments
> - IndexWriter.AddIndex(IndexReader[]) is invoked
> I put together a JUnit test that recreates the deadlock, which I've attached. It is the first test method, 'testAddIndexByIndexReader()'.
> In a nutshell, here is what happens:
> // 1) AddIndexes(IndexReader[]) acquires the write lock,
> // then begins optimization of destination index (this is
> // prior to adding any external segments).
> //
> // 2) Main thread starts a ConcurrentMergeScheduler.MergeThread
> // to merge the 2 segments.
> //
> // 3) Merging thread tries to acquire the read lock at
> // IndexWriter.blockAddIndexes(boolean) in
> // IndexWriter.StartCommit(), but cannot as...
> //
> // 4) Main thread still holds the write lock, and is
> // waiting for the IndexWriter.runningMerges data structure
> // to be devoid of merges with their optimize flag
> // set (IndexWriter.optimizeMergesPending()).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1544) Deadlock:
IndexWriter.addIndexes(IndexReader[])
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-1544.
----------------------------------------
Resolution: Fixed
Committed revision 746662 on 2.4 branch.
Thanks Doug!
> Deadlock: IndexWriter.addIndexes(IndexReader[])
> -----------------------------------------------
>
> Key: LUCENE-1544
> URL: https://issues.apache.org/jira/browse/LUCENE-1544
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4
> Environment: should be ubiquitous... but, Win XP on a MacBookPro (Intel) under Eclipse (Ganymede) using default compiler/runtime
> Reporter: Doug Sale
> Assignee: Michael McCandless
> Fix For: 2.4.1, 2.9
>
> Attachments: LUCENE-1544.patch, TestAddIndexes.java
>
>
> A deadlock issue occurs under the following circumstances
> - IndexWriter.autoCommit == true
> - IndexWriter.directory contains multiple segments
> - IndexWriter.AddIndex(IndexReader[]) is invoked
> I put together a JUnit test that recreates the deadlock, which I've attached. It is the first test method, 'testAddIndexByIndexReader()'.
> In a nutshell, here is what happens:
> // 1) AddIndexes(IndexReader[]) acquires the write lock,
> // then begins optimization of destination index (this is
> // prior to adding any external segments).
> //
> // 2) Main thread starts a ConcurrentMergeScheduler.MergeThread
> // to merge the 2 segments.
> //
> // 3) Merging thread tries to acquire the read lock at
> // IndexWriter.blockAddIndexes(boolean) in
> // IndexWriter.StartCommit(), but cannot as...
> //
> // 4) Main thread still holds the write lock, and is
> // waiting for the IndexWriter.runningMerges data structure
> // to be devoid of merges with their optimize flag
> // set (IndexWriter.optimizeMergesPending()).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1544) Deadlock:
IndexWriter.addIndexes(IndexReader[])
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675645#action_12675645 ]
Michael McCandless commented on LUCENE-1544:
--------------------------------------------
Committed revision 746661 on trunk.
> Deadlock: IndexWriter.addIndexes(IndexReader[])
> -----------------------------------------------
>
> Key: LUCENE-1544
> URL: https://issues.apache.org/jira/browse/LUCENE-1544
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4
> Environment: should be ubiquitous... but, Win XP on a MacBookPro (Intel) under Eclipse (Ganymede) using default compiler/runtime
> Reporter: Doug Sale
> Assignee: Michael McCandless
> Fix For: 2.4.1, 2.9
>
> Attachments: LUCENE-1544.patch, TestAddIndexes.java
>
>
> A deadlock issue occurs under the following circumstances
> - IndexWriter.autoCommit == true
> - IndexWriter.directory contains multiple segments
> - IndexWriter.AddIndex(IndexReader[]) is invoked
> I put together a JUnit test that recreates the deadlock, which I've attached. It is the first test method, 'testAddIndexByIndexReader()'.
> In a nutshell, here is what happens:
> // 1) AddIndexes(IndexReader[]) acquires the write lock,
> // then begins optimization of destination index (this is
> // prior to adding any external segments).
> //
> // 2) Main thread starts a ConcurrentMergeScheduler.MergeThread
> // to merge the 2 segments.
> //
> // 3) Merging thread tries to acquire the read lock at
> // IndexWriter.blockAddIndexes(boolean) in
> // IndexWriter.StartCommit(), but cannot as...
> //
> // 4) Main thread still holds the write lock, and is
> // waiting for the IndexWriter.runningMerges data structure
> // to be devoid of merges with their optimize flag
> // set (IndexWriter.optimizeMergesPending()).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org