You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by devoss ind <de...@gmail.com> on 2014/05/15 18:00:48 UTC
Re: tomcat6 thread locked
On 7 May 2014 17:37, "devoss ind" <de...@gmail.com> wrote:
>
> Hi,
>
> Am using tomcat 6, java6 in production environment to run our web
application. We found that recently, cpu in top is showing 100% nearly.
Then we dig into each thread cpu usage by -H option in top. We found only
one thread is taking 100%usage and identified the thread using thread dump.
The thread dump for that particular thread which is using 100% usage is as
follows...
>
> "http-8080-exec-3687" daemon prio=10 tid=0x7336f400 nid=0x41a1 runnable
[0x619ad000..0x619ade30]
> java.lang.Thread.State: RUNNABLE
> at
org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:616)
> - locked <0x967b22f0> (a
org.apache.coyote.http11.InternalNioOutputBuffer)
> at
org.apache.coyote.http11.InternalNioOutputBuffer.commit(InternalNioOutputBuffer.java:608)
> at
org.apache.coyote.http11.Http11NioProcessor.action(Http11NioProcessor.java:1024)
> at org.apache.coyote.Response.action(Response.java:183)
> at org.apache.coyote.Response.sendHeaders(Response.java:379)
> at
org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:305)
> at
org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:288)
> at
org.apache.catalina.connector.Response.flushBuffer(Response.java:548)
> at
org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:345)
> at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:144)
> at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> at
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:879)
> at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:719)
> at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2080)
> at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
>
> We can see that above thread is locked due to some problem in NIO.
>
> We have verified in tomcat manager also for such thread. We found that a
particular thread is taking 14140517ms time and the request in that line is
of POST request which is not related to our application.
>
> That particular thread is not releasing till we restart tomcat.
>
> This is happening frequently.
>
> Please help me to find out the root cause of this problem. How to avoid
restarts.
>
> Our server.xml connector configuration is as follows...
>
> <Connector port="${http.port}"
protocol="org.apache.coyote.http11.Http11NioProtocol"
> connectionTimeout="5000"
> acceptCount="1500"
> maxThreads="10000"
> redirectPort="8443" />
>
> Please help in this thread locking problem.
>
Re: tomcat6 thread locked
Posted by "Terence M. Bandoian" <te...@tmbsw.com>.
On 5/20/2014 8:28 PM, devoss ind wrote:
> Hi Christopher,
>
> Can you suggest stable tomcat and jvm versions.
>
> Regards,
> Devoss.
> On 20 May 2014 21:32, "Christopher Schultz" <ch...@christopherschultz.net>
> wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> Devoss,
>>
>> On 5/19/14, 2:43 AM, devoss ind wrote:
>>> Exact Tomcat version is 6.0.16
>> This version of Tomcat is profoundly old: more than 6 years. There are
>> documented and well-understood vulnerabilities in that (and later)
>> versions of Tomcat 6.0.x that will allow unauthenticated remote
>> attackers to subvert certain authentication mechanisms, mount XSS and
>> CSRF attacks, and trivially launch non-distributed denial-of-service
>> attacks against your server(s).
>>
>> Continued use of it is reckless to be polite. You absolutely
>> positively need to upgrade immediately.
>>
>>> OS Version is RHEL Server release 5.3(Tikanga) Java version is
>>> 1.6.0_10
>> All that stuff is fairly old as well.
>>
>> Upgrade Tomcat and your JVM and re-test, please.
>>
>> - -chris
>>
>>> On Fri, May 16, 2014 at 9:15 PM, Christopher Schultz <
>>> chris@christopherschultz.net> wrote:
>>>
>>> devoss,
>>>
>>> Exact Tomcat version? OS and version? Java version?
>>>
>>> On 5/15/14, 12:00 PM, devoss ind wrote:
>>>>>> On 7 May 2014 17:37, "devoss ind" <de...@gmail.com>
>>>>>> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> Am using tomcat 6, java6 in production environment to run
>>>>>>> our web
>>>>>> application. We found that recently, cpu in top is showing
>>>>>> 100% nearly. Then we dig into each thread cpu usage by -H
>>>>>> option in top. We found only one thread is taking 100%usage
>>>>>> and identified the thread using thread dump. The thread dump
>>>>>> for that particular thread which is using 100% usage is as
>>>>>> follows...
>>>>>>> "http-8080-exec-3687" daemon prio=10 tid=0x7336f400
>>>>>>> nid=0x41a1 runnable
>>>>>> [0x619ad000..0x619ade30]
>>>>>>> java.lang.Thread.State: RUNNABLE at
>> org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:616)
>>>
>> - - locked <0x967b22f0> (a
>>>>>> org.apache.coyote.http11.InternalNioOutputBuffer)
>>>>>>> at
>> org.apache.coyote.http11.InternalNioOutputBuffer.commit(InternalNioOutputBuffer.java:608)
>>>
>> at
>> org.apache.coyote.http11.Http11NioProcessor.action(Http11NioProcessor.java:1024)
>>>
>> at org.apache.coyote.Response.action(Response.java:183)
>>>>>>> at
>>>>>>> org.apache.coyote.Response.sendHeaders(Response.java:379)
>>>>>>> at
>> org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:305)
>> at
>> org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:288)
>> at
>>>>>> org.apache.catalina.connector.Response.flushBuffer(Response.java:548)
>> at
>> org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:345)
>>>
>> at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:144)
>>>
>> at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>>>
>> at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>>>
>> at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
>>>
>> at
>> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:879)
>>>
>> at
>> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:719)
>>>
>> at
>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2080)
>>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>
>> at java.lang.Thread.run(Thread.java:619)
>>>>>>> We can see that above thread is locked due to some problem
>>>>>>> in NIO.
>>>>>>>
>>>>>>> We have verified in tomcat manager also for such thread. We
>>>>>>> found that a
>>>>>> particular thread is taking 14140517ms time and the request
>>>>>> in that line is of POST request which is not related to our
>>>>>> application.
>>>>>>> That particular thread is not releasing till we restart
>>>>>>> tomcat.
>>>>>>>
>>>>>>> This is happening frequently.
>>>>>>>
>>>>>>> Please help me to find out the root cause of this problem.
>>>>>>> How to avoid
>>>>>> restarts.
>>>>>>> Our server.xml connector configuration is as follows...
>>>>>>>
>>>>>>> <Connector port="${http.port}"
>>>>>> protocol="org.apache.coyote.http11.Http11NioProtocol"
>>>>>>> connectionTimeout="5000" acceptCount="1500"
>>>>>>> maxThreads="10000" redirectPort="8443" />
>>>>>>>
>>>>>>> Please help in this thread locking problem.
http://tomcat.apache.org/whichversion.html
Also, the convention on the Tomcat users list is to respond below the
previous message. Please see 6. in:
http://tomcat.apache.org/lists.html#tomcat-users
-Terence Bandoian
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: tomcat6 thread locked
Posted by Jeffrey Janner <Je...@PolyDyne.com>.
> -----Original Message-----
> From: devoss ind [mailto:devoss.ind@gmail.com]
> Sent: Tuesday, May 20, 2014 8:29 PM
> To: Tomcat Users List
> Subject: Re: tomcat6 thread locked
>
> Hi Christopher,
>
> Can you suggest stable tomcat and jvm versions.
>
> Regards,
> Devoss.
> >
I can: Java 7u55 (1.7.0_55) and Tomcat 7.0.53
Just so happens these are the latest versions of both product.
Jeff
Re: tomcat6 thread locked
Posted by devoss ind <de...@gmail.com>.
Hi Christopher,
Can you suggest stable tomcat and jvm versions.
Regards,
Devoss.
On 20 May 2014 21:32, "Christopher Schultz" <ch...@christopherschultz.net>
wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Devoss,
>
> On 5/19/14, 2:43 AM, devoss ind wrote:
> > Exact Tomcat version is 6.0.16
>
> This version of Tomcat is profoundly old: more than 6 years. There are
> documented and well-understood vulnerabilities in that (and later)
> versions of Tomcat 6.0.x that will allow unauthenticated remote
> attackers to subvert certain authentication mechanisms, mount XSS and
> CSRF attacks, and trivially launch non-distributed denial-of-service
> attacks against your server(s).
>
> Continued use of it is reckless to be polite. You absolutely
> positively need to upgrade immediately.
>
> > OS Version is RHEL Server release 5.3(Tikanga) Java version is
> > 1.6.0_10
>
> All that stuff is fairly old as well.
>
> Upgrade Tomcat and your JVM and re-test, please.
>
> - -chris
>
> > On Fri, May 16, 2014 at 9:15 PM, Christopher Schultz <
> > chris@christopherschultz.net> wrote:
> >
> > devoss,
> >
> > Exact Tomcat version? OS and version? Java version?
> >
> > On 5/15/14, 12:00 PM, devoss ind wrote:
> >>>> On 7 May 2014 17:37, "devoss ind" <de...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> Am using tomcat 6, java6 in production environment to run
> >>>>> our web
> >>>> application. We found that recently, cpu in top is showing
> >>>> 100% nearly. Then we dig into each thread cpu usage by -H
> >>>> option in top. We found only one thread is taking 100%usage
> >>>> and identified the thread using thread dump. The thread dump
> >>>> for that particular thread which is using 100% usage is as
> >>>> follows...
> >>>>>
> >>>>> "http-8080-exec-3687" daemon prio=10 tid=0x7336f400
> >>>>> nid=0x41a1 runnable
> >>>> [0x619ad000..0x619ade30]
> >>>>> java.lang.Thread.State: RUNNABLE at
> >>>>
> >
> org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:616)
> >>>>>
> >>>>
> >
> >
> - - locked <0x967b22f0> (a
> >>>> org.apache.coyote.http11.InternalNioOutputBuffer)
> >>>>> at
> >>>>
> >
> org.apache.coyote.http11.InternalNioOutputBuffer.commit(InternalNioOutputBuffer.java:608)
> >>>>>
> >>>>
> >
> >
> at
> >>>>
> >
> org.apache.coyote.http11.Http11NioProcessor.action(Http11NioProcessor.java:1024)
> >>>>>
> >>>>
> >
> >
> at org.apache.coyote.Response.action(Response.java:183)
> >>>>> at
> >>>>> org.apache.coyote.Response.sendHeaders(Response.java:379)
> >>>>> at
> >>>>
> org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:305)
> >>>>>
> >>>>
> >
> >>>>
> at
> >>>>
> org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:288)
> >>>>>
> >>>>
> >
> >>>>
> at
> >>>> org.apache.catalina.connector.Response.flushBuffer(Response.java:548)
> >>>>>
> >>>>
> >
> >>>>
> at
> >>>>
> >
> org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:345)
> >>>>>
> >>>>
> >
> >
> at
> >>>>
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:144)
> >>>>>
> >>>>
> >
> >
> at
> >>>>
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> >>>>>
> >>>>
> >
> >
> at
> >>>>
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> >>>>>
> >>>>
> >
> >
> at
> >>>>
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> >>>>>
> >>>>
> >
> >
> at
> >>>>
> >
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:879)
> >>>>>
> >>>>
> >
> >
> at
> >>>>
> >
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:719)
> >>>>>
> >>>>
> >
> >
> at
> >>>>
> >
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2080)
> >>>>>
> >>>>
> >
> >
> at
> >>>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >>>>>
> >>>>
> >
> >
> at
> >>>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >>>>>
> >>>>
> >
> >
> at java.lang.Thread.run(Thread.java:619)
> >>>>>
> >>>>> We can see that above thread is locked due to some problem
> >>>>> in NIO.
> >>>>>
> >>>>> We have verified in tomcat manager also for such thread. We
> >>>>> found that a
> >>>> particular thread is taking 14140517ms time and the request
> >>>> in that line is of POST request which is not related to our
> >>>> application.
> >>>>>
> >>>>> That particular thread is not releasing till we restart
> >>>>> tomcat.
> >>>>>
> >>>>> This is happening frequently.
> >>>>>
> >>>>> Please help me to find out the root cause of this problem.
> >>>>> How to avoid
> >>>> restarts.
> >>>>>
> >>>>> Our server.xml connector configuration is as follows...
> >>>>>
> >>>>> <Connector port="${http.port}"
> >>>> protocol="org.apache.coyote.http11.Http11NioProtocol"
> >>>>> connectionTimeout="5000" acceptCount="1500"
> >>>>> maxThreads="10000" redirectPort="8443" />
> >>>>>
> >>>>> Please help in this thread locking problem.
> >>>>>
> >>>>
> >>
> >> ---------------------------------------------------------------------
> >>
> >>
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> >> For additional commands, e-mail: users-help@tomcat.apache.org
> >>
> >>
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJTe3x+AAoJEBzwKT+lPKRYRpUQALslmRSD3el2YLLrM89+qS+J
> +Hd297e6scPSSho2QyAA3WqkeqbTEWtgOyD7k9n757Q0uvgjb4kCe6NPRuhqSYV7
> dp1YVl80UgYbFG7vM7h0UBbAxSCNuUUBH2Y4RL5gzAlpOQfpUFjhAMcdUQWApQGr
> ErxxuxYH8uIce8HJrROlK8Zety4o1lSOJOO+VNp/QNGyuXkg8tq2JiW+95/1DZGT
> Zwr6yLKBFMyyJXBClrabAttZ7xpRFVEg8c/SdHBjE+18D6WRxRxreyZuWw/90Gpj
> CEnc08BnFuCXIHVH68dMJ8UYCvA4smDdl5wI6GJkI3Bf1vx/Ilu1BLJs3gc4acrb
> L/u2MWTw/jx/sRWWZxL5hh0LJPoyogZo7F9kbrekTCqc8RvCGtrdgAvnb327ZOOl
> may9YXLN0PVAI2A6+3v5KgpBURwoQbfZXm1pjqLQRKwvmYaILEmxu32eHXpqMcuy
> tk6D9mG4juJ4A6k6OX1r8DYr158wDNG5dVNx4OCdQxylrVNxUeKbWzjHHDiMYdA7
> ZSeXYLWyGJCmqg8PxFXt+3sFZWsjtPj4JEIdYKK9mjIijx2cEy73BYtveQJox0fS
> /4l43l/YlnxmPYiPjaHPphHt3+8dMUE/jGqVMpyLdXaM2iZfy3hOlish2mjL3et0
> WgpsxjRAqz5uD48VMScA
> =yd5W
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: tomcat6 thread locked
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Devoss,
On 5/19/14, 2:43 AM, devoss ind wrote:
> Exact Tomcat version is 6.0.16
This version of Tomcat is profoundly old: more than 6 years. There are
documented and well-understood vulnerabilities in that (and later)
versions of Tomcat 6.0.x that will allow unauthenticated remote
attackers to subvert certain authentication mechanisms, mount XSS and
CSRF attacks, and trivially launch non-distributed denial-of-service
attacks against your server(s).
Continued use of it is reckless to be polite. You absolutely
positively need to upgrade immediately.
> OS Version is RHEL Server release 5.3(Tikanga) Java version is
> 1.6.0_10
All that stuff is fairly old as well.
Upgrade Tomcat and your JVM and re-test, please.
- -chris
> On Fri, May 16, 2014 at 9:15 PM, Christopher Schultz <
> chris@christopherschultz.net> wrote:
>
> devoss,
>
> Exact Tomcat version? OS and version? Java version?
>
> On 5/15/14, 12:00 PM, devoss ind wrote:
>>>> On 7 May 2014 17:37, "devoss ind" <de...@gmail.com>
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Am using tomcat 6, java6 in production environment to run
>>>>> our web
>>>> application. We found that recently, cpu in top is showing
>>>> 100% nearly. Then we dig into each thread cpu usage by -H
>>>> option in top. We found only one thread is taking 100%usage
>>>> and identified the thread using thread dump. The thread dump
>>>> for that particular thread which is using 100% usage is as
>>>> follows...
>>>>>
>>>>> "http-8080-exec-3687" daemon prio=10 tid=0x7336f400
>>>>> nid=0x41a1 runnable
>>>> [0x619ad000..0x619ade30]
>>>>> java.lang.Thread.State: RUNNABLE at
>>>>
> org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:616)
>>>>>
>>>>
>
>
- - locked <0x967b22f0> (a
>>>> org.apache.coyote.http11.InternalNioOutputBuffer)
>>>>> at
>>>>
> org.apache.coyote.http11.InternalNioOutputBuffer.commit(InternalNioOutputBuffer.java:608)
>>>>>
>>>>
>
>
at
>>>>
> org.apache.coyote.http11.Http11NioProcessor.action(Http11NioProcessor.java:1024)
>>>>>
>>>>
>
>
at org.apache.coyote.Response.action(Response.java:183)
>>>>> at
>>>>> org.apache.coyote.Response.sendHeaders(Response.java:379)
>>>>> at
>>>> org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:305)
>>>>>
>>>>
>
>>>>
at
>>>> org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:288)
>>>>>
>>>>
>
>>>>
at
>>>> org.apache.catalina.connector.Response.flushBuffer(Response.java:548)
>>>>>
>>>>
>
>>>>
at
>>>>
> org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:345)
>>>>>
>>>>
>
>
at
>>>>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:144)
>>>>>
>>>>
>
>
at
>>>>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>>>>>
>>>>
>
>
at
>>>>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>>>>>
>>>>
>
>
at
>>>>
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
>>>>>
>>>>
>
>
at
>>>>
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:879)
>>>>>
>>>>
>
>
at
>>>>
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:719)
>>>>>
>>>>
>
>
at
>>>>
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2080)
>>>>>
>>>>
>
>
at
>>>>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>
>>>>
>
>
at
>>>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>
>>>>
>
>
at java.lang.Thread.run(Thread.java:619)
>>>>>
>>>>> We can see that above thread is locked due to some problem
>>>>> in NIO.
>>>>>
>>>>> We have verified in tomcat manager also for such thread. We
>>>>> found that a
>>>> particular thread is taking 14140517ms time and the request
>>>> in that line is of POST request which is not related to our
>>>> application.
>>>>>
>>>>> That particular thread is not releasing till we restart
>>>>> tomcat.
>>>>>
>>>>> This is happening frequently.
>>>>>
>>>>> Please help me to find out the root cause of this problem.
>>>>> How to avoid
>>>> restarts.
>>>>>
>>>>> Our server.xml connector configuration is as follows...
>>>>>
>>>>> <Connector port="${http.port}"
>>>> protocol="org.apache.coyote.http11.Http11NioProtocol"
>>>>> connectionTimeout="5000" acceptCount="1500"
>>>>> maxThreads="10000" redirectPort="8443" />
>>>>>
>>>>> Please help in this thread locking problem.
>>>>>
>>>>
>>
>> ---------------------------------------------------------------------
>>
>>
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTe3x+AAoJEBzwKT+lPKRYRpUQALslmRSD3el2YLLrM89+qS+J
+Hd297e6scPSSho2QyAA3WqkeqbTEWtgOyD7k9n757Q0uvgjb4kCe6NPRuhqSYV7
dp1YVl80UgYbFG7vM7h0UBbAxSCNuUUBH2Y4RL5gzAlpOQfpUFjhAMcdUQWApQGr
ErxxuxYH8uIce8HJrROlK8Zety4o1lSOJOO+VNp/QNGyuXkg8tq2JiW+95/1DZGT
Zwr6yLKBFMyyJXBClrabAttZ7xpRFVEg8c/SdHBjE+18D6WRxRxreyZuWw/90Gpj
CEnc08BnFuCXIHVH68dMJ8UYCvA4smDdl5wI6GJkI3Bf1vx/Ilu1BLJs3gc4acrb
L/u2MWTw/jx/sRWWZxL5hh0LJPoyogZo7F9kbrekTCqc8RvCGtrdgAvnb327ZOOl
may9YXLN0PVAI2A6+3v5KgpBURwoQbfZXm1pjqLQRKwvmYaILEmxu32eHXpqMcuy
tk6D9mG4juJ4A6k6OX1r8DYr158wDNG5dVNx4OCdQxylrVNxUeKbWzjHHDiMYdA7
ZSeXYLWyGJCmqg8PxFXt+3sFZWsjtPj4JEIdYKK9mjIijx2cEy73BYtveQJox0fS
/4l43l/YlnxmPYiPjaHPphHt3+8dMUE/jGqVMpyLdXaM2iZfy3hOlish2mjL3et0
WgpsxjRAqz5uD48VMScA
=yd5W
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: tomcat6 thread locked
Posted by devoss ind <de...@gmail.com>.
Hi Christopher,
Exact Tomcat version is 6.0.16
OS Version is RHEL Server release 5.3(Tikanga)
Java version is 1.6.0_10
Regards,
Devoss
On Fri, May 16, 2014 at 9:15 PM, Christopher Schultz <
chris@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> devoss,
>
> Exact Tomcat version? OS and version? Java version?
>
> On 5/15/14, 12:00 PM, devoss ind wrote:
> > On 7 May 2014 17:37, "devoss ind" <de...@gmail.com> wrote:
> >>
> >> Hi,
> >>
> >> Am using tomcat 6, java6 in production environment to run our
> >> web
> > application. We found that recently, cpu in top is showing 100%
> > nearly. Then we dig into each thread cpu usage by -H option in top.
> > We found only one thread is taking 100%usage and identified the
> > thread using thread dump. The thread dump for that particular
> > thread which is using 100% usage is as follows...
> >>
> >> "http-8080-exec-3687" daemon prio=10 tid=0x7336f400 nid=0x41a1
> >> runnable
> > [0x619ad000..0x619ade30]
> >> java.lang.Thread.State: RUNNABLE at
> >
> org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:616)
> >>
> >
> - - locked <0x967b22f0> (a
> > org.apache.coyote.http11.InternalNioOutputBuffer)
> >> at
> >
> org.apache.coyote.http11.InternalNioOutputBuffer.commit(InternalNioOutputBuffer.java:608)
> >>
> >
> at
> >
> org.apache.coyote.http11.Http11NioProcessor.action(Http11NioProcessor.java:1024)
> >>
> >
> at org.apache.coyote.Response.action(Response.java:183)
> >> at org.apache.coyote.Response.sendHeaders(Response.java:379) at
> > org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:305)
> >>
> >
> at
> > org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:288)
> >>
> >
> at
> > org.apache.catalina.connector.Response.flushBuffer(Response.java:548)
> >>
> >
> at
> >
> org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:345)
> >>
> >
> at
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:144)
> >>
> >
> at
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> >>
> >
> at
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> >>
> >
> at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> >>
> >
> at
> >
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:879)
> >>
> >
> at
> >
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:719)
> >>
> >
> at
> >
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2080)
> >>
> >
> at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >>
> >
> at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >>
> >
> at java.lang.Thread.run(Thread.java:619)
> >>
> >> We can see that above thread is locked due to some problem in
> >> NIO.
> >>
> >> We have verified in tomcat manager also for such thread. We found
> >> that a
> > particular thread is taking 14140517ms time and the request in that
> > line is of POST request which is not related to our application.
> >>
> >> That particular thread is not releasing till we restart tomcat.
> >>
> >> This is happening frequently.
> >>
> >> Please help me to find out the root cause of this problem. How to
> >> avoid
> > restarts.
> >>
> >> Our server.xml connector configuration is as follows...
> >>
> >> <Connector port="${http.port}"
> > protocol="org.apache.coyote.http11.Http11NioProtocol"
> >> connectionTimeout="5000" acceptCount="1500" maxThreads="10000"
> >> redirectPort="8443" />
> >>
> >> Please help in this thread locking problem.
> >>
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJTdjKHAAoJEBzwKT+lPKRYoPsQAKx0UqJ8eVP9gXJ+dluvUBii
> RmbIkljyBiVjCLj3uQlPoMNu6B9LRjcwQM0cBXh9MmXakoYVNJ9fxSv1bc8iI/4X
> EjKxRdYcupuGxgzPXrM2onhdhfAXgfsjYUKiFspPv+RKjG+wVrfULdkcLW4FgQV9
> XnTnZD32W7R3LprCYksk4WGpU4NCJOjCx2VXIXT39jl00pyPqTZ1StTmzXa/9WZP
> w1xwytW+sUHt1hgFsLyVbKJzL/61e1Fkk4M/avyBVqWHm3sCdi8b2qpCfwoAmEZX
> mhBEC1diNhvi7+w0n7K3iXN6f+iloECUpHWiXcVOn9Q1p8t1cNXN6vFCNIpdrmzL
> T4Doa4S4aomnFWRIVqg8DsO1FI5v0ZJnAsI9NEHVdd7RDMZpEYdnvkVFG2QN6rnb
> SwVVjFbmhZVB1bmPMFzPb+OPRTeQUivdgLvnpXRCegVyxTxpPaoDJeykeahZqueh
> 0owe7fOFJBiG9dgDD+EDb87p3CipHGQA6sxxsQVTJ5kO9hzAthhTUd47J6Xy2bCa
> Fkz6MAQ7//G5GwPrYfZHQG1DIIWg+CYwZIkr4YTP83zD8P9oUGgt1HHYXhvq8hCq
> I910AxAg8r0l9X1c6QXQ1C+20FjG2wYRcMjU/LItrZY2ra7U//oT3L8YX0Gx4UrB
> SM7oBj/HEFDcTzU/4ACv
> =jW9p
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: tomcat6 thread locked
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
devoss,
Exact Tomcat version? OS and version? Java version?
On 5/15/14, 12:00 PM, devoss ind wrote:
> On 7 May 2014 17:37, "devoss ind" <de...@gmail.com> wrote:
>>
>> Hi,
>>
>> Am using tomcat 6, java6 in production environment to run our
>> web
> application. We found that recently, cpu in top is showing 100%
> nearly. Then we dig into each thread cpu usage by -H option in top.
> We found only one thread is taking 100%usage and identified the
> thread using thread dump. The thread dump for that particular
> thread which is using 100% usage is as follows...
>>
>> "http-8080-exec-3687" daemon prio=10 tid=0x7336f400 nid=0x41a1
>> runnable
> [0x619ad000..0x619ade30]
>> java.lang.Thread.State: RUNNABLE at
> org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:616)
>>
>
- - locked <0x967b22f0> (a
> org.apache.coyote.http11.InternalNioOutputBuffer)
>> at
> org.apache.coyote.http11.InternalNioOutputBuffer.commit(InternalNioOutputBuffer.java:608)
>>
>
at
> org.apache.coyote.http11.Http11NioProcessor.action(Http11NioProcessor.java:1024)
>>
>
at org.apache.coyote.Response.action(Response.java:183)
>> at org.apache.coyote.Response.sendHeaders(Response.java:379) at
> org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:305)
>>
>
at
> org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:288)
>>
>
at
> org.apache.catalina.connector.Response.flushBuffer(Response.java:548)
>>
>
at
> org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:345)
>>
>
at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:144)
>>
>
at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>>
>
at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>>
>
at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
>>
>
at
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:879)
>>
>
at
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:719)
>>
>
at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2080)
>>
>
at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>
>
at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>
>
at java.lang.Thread.run(Thread.java:619)
>>
>> We can see that above thread is locked due to some problem in
>> NIO.
>>
>> We have verified in tomcat manager also for such thread. We found
>> that a
> particular thread is taking 14140517ms time and the request in that
> line is of POST request which is not related to our application.
>>
>> That particular thread is not releasing till we restart tomcat.
>>
>> This is happening frequently.
>>
>> Please help me to find out the root cause of this problem. How to
>> avoid
> restarts.
>>
>> Our server.xml connector configuration is as follows...
>>
>> <Connector port="${http.port}"
> protocol="org.apache.coyote.http11.Http11NioProtocol"
>> connectionTimeout="5000" acceptCount="1500" maxThreads="10000"
>> redirectPort="8443" />
>>
>> Please help in this thread locking problem.
>>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTdjKHAAoJEBzwKT+lPKRYoPsQAKx0UqJ8eVP9gXJ+dluvUBii
RmbIkljyBiVjCLj3uQlPoMNu6B9LRjcwQM0cBXh9MmXakoYVNJ9fxSv1bc8iI/4X
EjKxRdYcupuGxgzPXrM2onhdhfAXgfsjYUKiFspPv+RKjG+wVrfULdkcLW4FgQV9
XnTnZD32W7R3LprCYksk4WGpU4NCJOjCx2VXIXT39jl00pyPqTZ1StTmzXa/9WZP
w1xwytW+sUHt1hgFsLyVbKJzL/61e1Fkk4M/avyBVqWHm3sCdi8b2qpCfwoAmEZX
mhBEC1diNhvi7+w0n7K3iXN6f+iloECUpHWiXcVOn9Q1p8t1cNXN6vFCNIpdrmzL
T4Doa4S4aomnFWRIVqg8DsO1FI5v0ZJnAsI9NEHVdd7RDMZpEYdnvkVFG2QN6rnb
SwVVjFbmhZVB1bmPMFzPb+OPRTeQUivdgLvnpXRCegVyxTxpPaoDJeykeahZqueh
0owe7fOFJBiG9dgDD+EDb87p3CipHGQA6sxxsQVTJ5kO9hzAthhTUd47J6Xy2bCa
Fkz6MAQ7//G5GwPrYfZHQG1DIIWg+CYwZIkr4YTP83zD8P9oUGgt1HHYXhvq8hCq
I910AxAg8r0l9X1c6QXQ1C+20FjG2wYRcMjU/LItrZY2ra7U//oT3L8YX0Gx4UrB
SM7oBj/HEFDcTzU/4ACv
=jW9p
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org