You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Mani EZZAT <ma...@arisem.com> on 2009/10/02 10:41:47 UTC

TimeLimitedCollector hang on, VM process doesn't die (TOMCAT)

Hello everyone.
I'm using solrJ for an application deployed in Tomcat (6.x). It's base 
on lucene 2.9
when I use the catalina stop command, the VM doesn't die.

The problem seems to be the TimeLimitedCollector thread.
Here a jstack trace of the JVM (I get the same thing with jconsole too) :


Attaching to process ID 32430, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 1.5.0_17-b04
Thread 32430: (state = BLOCKED)


Thread 32530: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
 - org.apache.lucene.search.TimeLimitedCollector$TimerThread.run() 
@bci=15, line=79 (Interpreted frame)


Thread 32481: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 
(Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=41, line=1841 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=359 
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=47, line=470 
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, 
line=674 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)


Thread 32480: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 
(Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=41, line=1841 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=359 
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=47, line=470 
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, 
line=674 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)


Thread 32470: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=120 
(Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove() @bci=2, line=136 (Interpreted 
frame)
 - org.apache.commons.io.FileCleaner$Reaper.run() @bci=8, line=206 
(Interpreted frame)


Thread 32468: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=474 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, 
line=416 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.run() @bci=11, line=442 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)


Thread 32466: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=474 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, 
line=416 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.run() @bci=11, line=442 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)


Thread 32462: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=474 (Interpreted frame)
 - 
org.apache.jk.common.ChannelSocket.accept(org.apache.jk.core.MsgContext) 
@bci=20, line=300 (Interpreted frame)
 - org.apache.jk.common.ChannelSocket.acceptConnections() @bci=72, 
line=660 (Interpreted frame)
 - 
org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(java.lang.Object[]) 
@bci=4, line=870 (Interpreted frame)
 - org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run() 
@bci=167, line=690 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)


Thread 32458: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run() @bci=23, 
line=302 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)


Thread 32444: (state = IN_NATIVE)
 - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 
(Interpreted frame)
 - java.net.PlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=384 
(Interpreted frame)
 - java.net.ServerSocket.implAccept(java.net.Socket) @bci=50, line=450 
(Interpreted frame)
 - java.net.ServerSocket.accept() @bci=48, line=421 (Interpreted frame)
 - sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept() 
@bci=1, line=31 (Interpreted frame)
 - sun.rmi.transport.tcp.TCPTransport.run() @bci=59, line=340 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)


Thread 32443: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=474 (Interpreted frame)
 - java.util.TimerThread.mainLoop() @bci=28, line=483 (Interpreted frame)
 - java.util.TimerThread.run() @bci=1, line=462 (Interpreted frame)


Thread 32438: (state = BLOCKED)


Thread 32437: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=120 
(Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove() @bci=2, line=136 (Interpreted 
frame)
 - java.lang.ref.Finalizer$FinalizerThread.run() @bci=3, line=159 
(Interpreted frame)


Thread 32436: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=474 (Interpreted frame)
 - java.lang.ref.Reference$ReferenceHandler.run() @bci=46, line=116 
(Interpreted frame)


When I manually kill the lucene thread, the jvm exit.

I'm using java 1.5.0.17 and debian lenny.

I'm not sure if it's a Solr or Lucene problem...

Best regards.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: TimeLimitedCollector hang on, VM process doesn't die (TOMCAT)

Posted by Mani EZZAT <ma...@arisem.com>.
Mark Miller wrote:
> Mani EZZAT wrote:
>   
>> Mark Miller wrote:
>>     
>>> That thread will only be stopped if its interrupted. So it would appear
>>> there is a not a path that leads to it being interrupted ... why that is
>>> would be the next question ...
>>>
>>>   
>>>       
>> I found someone (a japanese) who had the same problem
>>
>> http://d.hatena.ne.jp/knaka20blue/20090608
>>
>> Do you think I should look at the SolR side ?
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>     
> Actually, it may not be the TimeLimitedCollector thats keeping those
> open. Since its a daemon thread, it shouldn't prevent shutdown even if
> its not interrupted.
>
> See that java.util.Timer way below? It's probably that. Know where it
> comes from? By default they are not daemon threads, so its a likely
> candidate ...
>
>   
This thread is also a deamon.
But I know which thread cause this and I don't think it's lucene related :)
Thank you for your time

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: TimeLimitedCollector hang on, VM process doesn't die (TOMCAT)

Posted by Mark Miller <ma...@gmail.com>.
Mani EZZAT wrote:
> Mark Miller wrote:
>> That thread will only be stopped if its interrupted. So it would appear
>> there is a not a path that leads to it being interrupted ... why that is
>> would be the next question ...
>>
>>   
> I found someone (a japanese) who had the same problem
>
> http://d.hatena.ne.jp/knaka20blue/20090608
>
> Do you think I should look at the SolR side ?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
Actually, it may not be the TimeLimitedCollector thats keeping those
open. Since its a daemon thread, it shouldn't prevent shutdown even if
its not interrupted.

See that java.util.Timer way below? It's probably that. Know where it
comes from? By default they are not daemon threads, so its a likely
candidate ...

-- 
- Mark

http://www.lucidimagination.com




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: TimeLimitedCollector hang on, VM process doesn't die (TOMCAT)

Posted by Mani EZZAT <ma...@arisem.com>.
Mark Miller wrote:
> That thread will only be stopped if its interrupted. So it would appear
> there is a not a path that leads to it being interrupted ... why that is
> would be the next question ...
>
>   
I found someone (a japanese) who had the same problem

http://d.hatena.ne.jp/knaka20blue/20090608

Do you think I should look at the SolR side ?

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: TimeLimitedCollector hang on, VM process doesn't die (TOMCAT)

Posted by Mark Miller <ma...@gmail.com>.
That thread will only be stopped if its interrupted. So it would appear
there is a not a path that leads to it being interrupted ... why that is
would be the next question ...

-- 
- Mark

http://www.lucidimagination.com



Mani EZZAT wrote:
> Hello everyone.
> I'm using solrJ for an application deployed in Tomcat (6.x). It's base
> on lucene 2.9
> when I use the catalina stop command, the VM doesn't die.
>
> The problem seems to be the TimeLimitedCollector thread.
> Here a jstack trace of the JVM (I get the same thing with jconsole too) :
>
>
> Attaching to process ID 32430, please wait...
> Debugger attached successfully.
> Server compiler detected.
> JVM version is 1.5.0_17-b04
> Thread 32430: (state = BLOCKED)
>
>
> Thread 32530: (state = BLOCKED)
> - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
> - org.apache.lucene.search.TimeLimitedCollector$TimerThread.run()
> @bci=15, line=79 (Interpreted frame)
>
>
> Thread 32481: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118
> (Interpreted frame)
> -
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await()
> @bci=41, line=1841 (Interpreted frame)
> - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=359
> (Interpreted frame)
> - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=47, line=470
> (Interpreted frame)
> - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18,
> line=674 (Interpreted frame)
> - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)
>
>
> Thread 32480: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118
> (Interpreted frame)
> -
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await()
> @bci=41, line=1841 (Interpreted frame)
> - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=359
> (Interpreted frame)
> - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=47, line=470
> (Interpreted frame)
> - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18,
> line=674 (Interpreted frame)
> - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)
>
>
> Thread 32470: (state = BLOCKED)
> - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
> - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=120
> (Interpreted frame)
> - java.lang.ref.ReferenceQueue.remove() @bci=2, line=136 (Interpreted
> frame)
> - org.apache.commons.io.FileCleaner$Reaper.run() @bci=8, line=206
> (Interpreted frame)
>
>
> Thread 32468: (state = BLOCKED)
> - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
> - java.lang.Object.wait() @bci=2, line=474 (Interpreted frame)
> - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8,
> line=416 (Interpreted frame)
> - org.apache.tomcat.util.net.JIoEndpoint$Worker.run() @bci=11,
> line=442 (Interpreted frame)
> - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)
>
>
> Thread 32466: (state = BLOCKED)
> - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
> - java.lang.Object.wait() @bci=2, line=474 (Interpreted frame)
> - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8,
> line=416 (Interpreted frame)
> - org.apache.tomcat.util.net.JIoEndpoint$Worker.run() @bci=11,
> line=442 (Interpreted frame)
> - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)
>
>
> Thread 32462: (state = BLOCKED)
> - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
> - java.lang.Object.wait() @bci=2, line=474 (Interpreted frame)
> -
> org.apache.jk.common.ChannelSocket.accept(org.apache.jk.core.MsgContext)
> @bci=20, line=300 (Interpreted frame)
> - org.apache.jk.common.ChannelSocket.acceptConnections() @bci=72,
> line=660 (Interpreted frame)
> -
> org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(java.lang.Object[])
> @bci=4, line=870 (Interpreted frame)
> - org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
> @bci=167, line=690 (Interpreted frame)
> - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)
>
>
> Thread 32458: (state = BLOCKED)
> - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
> - org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run() @bci=23,
> line=302 (Interpreted frame)
> - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)
>
>
> Thread 32444: (state = IN_NATIVE)
> - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0
> (Interpreted frame)
> - java.net.PlainSocketImpl.accept(java.net.SocketImpl) @bci=7,
> line=384 (Interpreted frame)
> - java.net.ServerSocket.implAccept(java.net.Socket) @bci=50, line=450
> (Interpreted frame)
> - java.net.ServerSocket.accept() @bci=48, line=421 (Interpreted frame)
> - sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept()
> @bci=1, line=31 (Interpreted frame)
> - sun.rmi.transport.tcp.TCPTransport.run() @bci=59, line=340
> (Interpreted frame)
> - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)
>
>
> Thread 32443: (state = BLOCKED)
> - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
> - java.lang.Object.wait() @bci=2, line=474 (Interpreted frame)
> - java.util.TimerThread.mainLoop() @bci=28, line=483 (Interpreted frame)
> - java.util.TimerThread.run() @bci=1, line=462 (Interpreted frame)
>
>
> Thread 32438: (state = BLOCKED)
>
>
> Thread 32437: (state = BLOCKED)
> - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
> - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=120
> (Interpreted frame)
> - java.lang.ref.ReferenceQueue.remove() @bci=2, line=136 (Interpreted
> frame)
> - java.lang.ref.Finalizer$FinalizerThread.run() @bci=3, line=159
> (Interpreted frame)
>
>
> Thread 32436: (state = BLOCKED)
> - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
> - java.lang.Object.wait() @bci=2, line=474 (Interpreted frame)
> - java.lang.ref.Reference$ReferenceHandler.run() @bci=46, line=116
> (Interpreted frame)
>
>
> When I manually kill the lucene thread, the jvm exit.
>
> I'm using java 1.5.0.17 and debian lenny.
>
> I'm not sure if it's a Solr or Lucene problem...
>
> Best regards.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>





---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org