You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Tim Smith (JIRA)" <ji...@apache.org> on 2011/08/12 17:19:27 UTC
[jira] [Created] (LUCENE-3373) waitForMerges deadlocks if
background merge fails
waitForMerges deadlocks if background merge fails
-------------------------------------------------
Key: LUCENE-3373
URL: https://issues.apache.org/jira/browse/LUCENE-3373
Project: Lucene - Java
Issue Type: Bug
Components: core/index
Affects Versions: 3.0.3
Reporter: Tim Smith
waitForMerges can deadlock if a merge fails for ConcurrentMergeScheduler
this is because the merge thread will die, but pending merges are still available
normally, the merge thread will pick up the next merge once it finishes the previous merge, but in the event of a merge exception, the pending work is not resumed, but waitForMerges won't complete until all pending work is complete
i worked around this by overriding doMerge() like so:
{code}
protected final void doMerge(MergePolicy.OneMerge merge) throws IOException {
try {
super.doMerge(merge);
} catch (Throwable exc) {
// Just logging the exception and not rethrowing
// insert logging code here
}
}
{code}
Here's the rough steps i used to reproduce this issue:
override doMerge like so
{code}
protected final void doMerge(MergePolicy.OneMerge merge) throws IOException {
try {Thread.sleep(500L);} catch (InterruptedException e) { }
super.doMerge(merge);
throw new IOException("fail");
}
{code}
then, if you do the following:
loop 50 times:
addDocument // any doc
commit
waitForMerges // This will deadlock sometimes
SOLR-2017 may be related to this (stack trace for deadlock looked related)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3373) waitForMerges deadlocks if
background merge fails
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089662#comment-13089662 ]
Michael McCandless commented on LUCENE-3373:
--------------------------------------------
Hmm nice catch!
Any ides on how should we fix this...? What do we want to happen? Should waitForMerges() return if any merge hits an exc? It could return a boolean indicating an error occurred?
> waitForMerges deadlocks if background merge fails
> -------------------------------------------------
>
> Key: LUCENE-3373
> URL: https://issues.apache.org/jira/browse/LUCENE-3373
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/index
> Affects Versions: 3.0.3
> Reporter: Tim Smith
>
> waitForMerges can deadlock if a merge fails for ConcurrentMergeScheduler
> this is because the merge thread will die, but pending merges are still available
> normally, the merge thread will pick up the next merge once it finishes the previous merge, but in the event of a merge exception, the pending work is not resumed, but waitForMerges won't complete until all pending work is complete
> i worked around this by overriding doMerge() like so:
> {code}
> protected final void doMerge(MergePolicy.OneMerge merge) throws IOException {
> try {
> super.doMerge(merge);
> } catch (Throwable exc) {
> // Just logging the exception and not rethrowing
> // insert logging code here
> }
> }
> {code}
> Here's the rough steps i used to reproduce this issue:
> override doMerge like so
> {code}
> protected final void doMerge(MergePolicy.OneMerge merge) throws IOException {
> try {Thread.sleep(500L);} catch (InterruptedException e) { }
> super.doMerge(merge);
> throw new IOException("fail");
> }
> {code}
> then, if you do the following:
> loop 50 times:
> addDocument // any doc
> commit
> waitForMerges // This will deadlock sometimes
> SOLR-2017 may be related to this (stack trace for deadlock looked related)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3373) waitForMerges deadlocks if
background merge fails
Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089674#comment-13089674 ]
Tim Smith commented on LUCENE-3373:
-----------------------------------
waitForMerges should continue to wait until all merges are complete (regardless of if they all end up failing)
i would suggest updating the MergeThread to catch all exceptions and allow processing the next merge. right now, any merge failure results in a ThreadDeath, which seems rather nasty. should probably just catch the exception and log a index trace message
> waitForMerges deadlocks if background merge fails
> -------------------------------------------------
>
> Key: LUCENE-3373
> URL: https://issues.apache.org/jira/browse/LUCENE-3373
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/index
> Affects Versions: 3.0.3
> Reporter: Tim Smith
>
> waitForMerges can deadlock if a merge fails for ConcurrentMergeScheduler
> this is because the merge thread will die, but pending merges are still available
> normally, the merge thread will pick up the next merge once it finishes the previous merge, but in the event of a merge exception, the pending work is not resumed, but waitForMerges won't complete until all pending work is complete
> i worked around this by overriding doMerge() like so:
> {code}
> protected final void doMerge(MergePolicy.OneMerge merge) throws IOException {
> try {
> super.doMerge(merge);
> } catch (Throwable exc) {
> // Just logging the exception and not rethrowing
> // insert logging code here
> }
> }
> {code}
> Here's the rough steps i used to reproduce this issue:
> override doMerge like so
> {code}
> protected final void doMerge(MergePolicy.OneMerge merge) throws IOException {
> try {Thread.sleep(500L);} catch (InterruptedException e) { }
> super.doMerge(merge);
> throw new IOException("fail");
> }
> {code}
> then, if you do the following:
> loop 50 times:
> addDocument // any doc
> commit
> waitForMerges // This will deadlock sometimes
> SOLR-2017 may be related to this (stack trace for deadlock looked related)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org