You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Dawid Weiss <da...@gmail.com> on 2012/03/01 23:20:04 UTC
toString on Thread
Overriding toString on a Thread is not a good idea. Can I remove it or
at least make it simpler in ConcurrentMergeScheduler? This override
caused a fantastic deadlock -- an interesting possibility I didn't
think of -- again, when dumping threads (for the exception string)
Thread.toString was invoked from what I thought was an isolated
monitor (and it was); only toString had its own monitors underneath
and here's what happened (simplified):
"Lucene Merge Thread #1":
at org.apache.lucene.index.IndexWriter.segString(IndexWriter.java:3764)
- waiting to lock <L5> (a org.apache.lucene.index.IndexWriter)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.toString(ConcurrentMergeScheduler.java:499)
...
at org.apache.lucene.util.LuceneTestCase.getRandom(LuceneTestCase.java:276)
at org.apache.lucene.index.TestTransactions.access$100(TestTransactions.java:33)
at org.apache.lucene.index.TestTransactions$RandomFailure.eval(TestTransactions.java:40)
at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:688)
- locked <L4> (a org.apache.lucene.store.MockDirectoryWrapper)
at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:415)
- locked <L4> (a org.apache.lucene.store.MockDirectoryWrapper)
at org.apache.lucene.codecs.lucene40.Lucene40FieldInfosWriter.write(Lucene40FieldInfosWriter.java:56)
at org.apache.lucene.index.SegmentMerger.mergeFieldInfos(SegmentMerger.java:194)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3623)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3257)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:382)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:451)
"Lucene Merge Thread #0":
at org.apache.lucene.store.MockDirectoryWrapper.listAll(MockDirectoryWrapper.java:695)
- waiting to lock <L4> (a org.apache.lucene.store.MockDirectoryWrapper)
at org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:345)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3272)
- locked <L5> (a org.apache.lucene.index.IndexWriter)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:382)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:451)
A classic, isn't it?
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: toString on Thread
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
> Now I've got to go think if we've done anything like that in Solr...
I did a quick check via Eclipse's Java search and it seems nothing
else overrides Thread#toString() or Thread#getName. Can't guarantee
anything, but 99% sure we're safe from this one.
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: toString on Thread
Posted by Uwe Schindler <uw...@thetaphi.de>.
I like the Clowns! Whenever I see IndexWriter internals I think about those clowns...
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: dawid.weiss@gmail.com [mailto:dawid.weiss@gmail.com] On Behalf Of
> Dawid Weiss
> Sent: Thursday, March 01, 2012 11:30 PM
> To: dev@lucene.apache.org; yonik@lucidimagination.com
> Subject: Re: toString on Thread
>
> > Ouch!
> > Now I've got to go think if we've done anything like that in Solr...
>
> Yeah... I honestly never thought about such possibility and I don't think any sane
> person would ;)
>
> I think this qualifies as a hack similar to the solution to this puzzler:
> http://wouter.coekaerts.be/2012/puzzle-clowns
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: toString on Thread
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Thu, Mar 1, 2012 at 5:30 PM, Dawid Weiss
<da...@cs.put.poznan.pl> wrote:
>> Ouch!
>> Now I've got to go think if we've done anything like that in Solr...
>
> Yeah... I honestly never thought about such possibility and I don't
> think any sane person would ;)
>
> I think this qualifies as a hack similar to the solution to this puzzler:
> http://wouter.coekaerts.be/2012/puzzle-clowns
Cute! Hadn't seen that. My first thought to a solution matched the
first comment on the solutions page (no spoilers!).
I missed the more elegant official solution.
-Yonik
lucenerevolution.com - Lucene/Solr Open Source Search Conference.
Boston May 7-10
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: toString on Thread
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
> Ouch!
> Now I've got to go think if we've done anything like that in Solr...
Yeah... I honestly never thought about such possibility and I don't
think any sane person would ;)
I think this qualifies as a hack similar to the solution to this puzzler:
http://wouter.coekaerts.be/2012/puzzle-clowns
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: toString on Thread
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Thu, Mar 1, 2012 at 5:20 PM, Dawid Weiss <da...@gmail.com> wrote:
> Overriding toString on a Thread is not a good idea. Can I remove it or
> at least make it simpler in ConcurrentMergeScheduler? This override
> caused a fantastic deadlock -- an interesting possibility I didn't
> think of -- again, when dumping threads (for the exception string)
> Thread.toString was invoked from what I thought was an isolated
> monitor (and it was); only toString had its own monitors underneath
> and here's what happened (simplified):
Ouch!
Now I've got to go think if we've done anything like that in Solr...
-Yonik
lucenerevolution.com - Lucene/Solr Open Source Search Conference.
Boston May 7-10
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: toString on Thread
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
> Though it is nice to see what segments are being merged by the
> thread... but this risk is awful. App can turn on IW's infoStream to
> see it too...
This could be possible by updating a volatile string somewhere and
only exposing it in toString override, but I don't know if this is
worth the effort. Volatile will impose an additional happens-before,
etc.
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: toString on Thread
Posted by Michael McCandless <lu...@mikemccandless.com>.
Scary! I think remove it?
Though it is nice to see what segments are being merged by the
thread... but this risk is awful. App can turn on IW's infoStream to
see it too...
Mike McCandless
http://blog.mikemccandless.com
On Thu, Mar 1, 2012 at 5:20 PM, Dawid Weiss <da...@gmail.com> wrote:
> Overriding toString on a Thread is not a good idea. Can I remove it or
> at least make it simpler in ConcurrentMergeScheduler? This override
> caused a fantastic deadlock -- an interesting possibility I didn't
> think of -- again, when dumping threads (for the exception string)
> Thread.toString was invoked from what I thought was an isolated
> monitor (and it was); only toString had its own monitors underneath
> and here's what happened (simplified):
>
> "Lucene Merge Thread #1":
> at org.apache.lucene.index.IndexWriter.segString(IndexWriter.java:3764)
> - waiting to lock <L5> (a org.apache.lucene.index.IndexWriter)
> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.toString(ConcurrentMergeScheduler.java:499)
> ...
> at org.apache.lucene.util.LuceneTestCase.getRandom(LuceneTestCase.java:276)
> at org.apache.lucene.index.TestTransactions.access$100(TestTransactions.java:33)
> at org.apache.lucene.index.TestTransactions$RandomFailure.eval(TestTransactions.java:40)
> at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:688)
> - locked <L4> (a org.apache.lucene.store.MockDirectoryWrapper)
> at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:415)
> - locked <L4> (a org.apache.lucene.store.MockDirectoryWrapper)
> at org.apache.lucene.codecs.lucene40.Lucene40FieldInfosWriter.write(Lucene40FieldInfosWriter.java:56)
> at org.apache.lucene.index.SegmentMerger.mergeFieldInfos(SegmentMerger.java:194)
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3623)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3257)
> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:382)
> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:451)
> "Lucene Merge Thread #0":
> at org.apache.lucene.store.MockDirectoryWrapper.listAll(MockDirectoryWrapper.java:695)
> - waiting to lock <L4> (a org.apache.lucene.store.MockDirectoryWrapper)
> at org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:345)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3272)
> - locked <L5> (a org.apache.lucene.index.IndexWriter)
> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:382)
> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:451)
>
> A classic, isn't it?
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org