You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@lucene.apache.org by "Stefan Heidrich (JIRA)" <ji...@apache.org> on 2009/03/10 09:52:50 UTC

[jira] Created: (LUCENE-1555) Deadlock while optimize

Deadlock while optimize
-----------------------

                 Key: LUCENE-1555
                 URL: https://issues.apache.org/jira/browse/LUCENE-1555
             Project: Lucene - Java
          Issue Type: Bug
          Components: Index
    Affects Versions: 2.4
         Environment: ubuntu 8.04, java 1.6 update 07, Lucene 2.4.0
            Reporter: Stefan Heidrich


Sometimes after starting the thread with the indexer, the thread will hang in the following threads.

Thread [Lucene Merge Thread #0] (Ausgesetzt)	
	IndexWriter.commitMerge(MergePolicy$OneMerge, SegmentMerger, int) Line: 3751	
	IndexWriter.mergeMiddle(MergePolicy$OneMerge) Line: 4240	
	IndexWriter.merge(MergePolicy$OneMerge) Line: 3877	
	ConcurrentMergeScheduler.doMerge(MergePolicy$OneMerge) Line: 205	
	ConcurrentMergeScheduler$MergeThread.run() Line: 260	

Thread [Indexer] (Ausgesetzt)	
	Object.wait(long) Line: not available [native Methode]	
	IndexWriter.doWait() Line: 4491	
	IndexWriter.optimize(int, boolean) Line: 2268	
	IndexWriter.optimize(boolean) Line: 2203	
	IndexWriter.optimize() Line: 2183	
	Indexer.run() Line: 263	


If you need more informations, please let me know.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: [Mira] Commented: (LUCENE-1555) Deadlock while optimize

Posted by pd...@yahoo.com.

-----Original Message-----
From: "Michael McCandless (JIRA)" <ji...@apache.org>

Date: Wed, 11 Mar 2009 05:56:50 
To: <ja...@lucene.apache.org>
Subject: [jira] Commented: (LUCENE-1555) Deadlock while optimize

    [ https://issues.apache.org/jira/browse/LUCENE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12680864#action_12680864 ] 

Michael McCandless commented on LUCENE-1555:
--------------------------------------------

bq. First of all, I could solve the problem by using IndexWriter.close(true) instead of IndexWriter.close().

Did you mean false instead of true?  (waitForMerges=true is what
close() does by default).

This (false) is dangerous to do, always, since it could result in merge
starvation whereby long-running merges never have a chance to
complete (I just updated 2.9 javadocs to state this).

{quote}
The indexer thread will index new data periodically. But if the indexer threat is started, it will perform an optimize as well. So it could happen that

((ConcurrentMergeScheduler)indexWriter.getMergeScheduler()).sync();
indexWriter.close(true);

and

indexWriter = new IndexWriter(indexDirectory, analyser , false, MaxFieldLength.UNLIMITED);
indexWriter.optimize(MAX_MERGED_FILES);
((ConcurrentMergeScheduler)indexWriter.getMergeScheduler()).sync();
indexWriter.close(true);

will be executed one after another. With close() this could result in a lock.
{quote}

I don't really follow what you're saying here.  Are you saying calling
ConcurrentMergeScheduler.sync() leads to the deadlock, or, prevents
it?

By indexer thread, do you mean you are launching your own threads to
do indexing, or are you talking about ConcurrentMergeScheduler's
threads?

Are you doing any external synchronization on IndexWriter yourself?

Are you calling IndexWriter.close() from multiple threads?

Can you post every thread in your thread dump?  (Or, is it really the
case that you only have 2 threads?).

bq. I've only changed the maxFieldLength of the writer to 50000.

OK that's fine.

> Deadlock while optimize
> -----------------------
>
>                 Key: LUCENE-1555
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1555
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.4
>         Environment: ubuntu 8.04, java 1.6 update 07, Lucene 2.4.0
>            Reporter: Stefan Heidrich
>            Assignee: Michael McCandless
>
> Sometimes after starting the thread with the indexer, the thread will hang in the following threads.
> Thread [Lucene Merge Thread #0] (Ausgesetzt)	
> 	IndexWriter.commitMerge(MergePolicy$OneMerge, SegmentMerger, int) Line: 3751	
> 	IndexWriter.mergeMiddle(MergePolicy$OneMerge) Line: 4240	
> 	IndexWriter.merge(MergePolicy$OneMerge) Line: 3877	
> 	ConcurrentMergeScheduler.doMerge(MergePolicy$OneMerge) Line: 205	
> 	ConcurrentMergeScheduler$MergeThread.run() Line: 260	
> Thread [Indexer] (Ausgesetzt)	
> 	Object.wait(long) Line: not available [native Methode]	
> 	IndexWriter.doWait() Line: 4491	
> 	IndexWriter.optimize(int, boolean) Line: 2268	
> 	IndexWriter.optimize(boolean) Line: 2203	
> 	IndexWriter.optimize() Line: 2183	
> 	Indexer.run() Line: 263	
> If you need more informations, please let me know.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org