You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@geronimo.apache.org by Stella Lok <oi...@gmail.com> on 2009/02/26 11:58:01 UTC
Web server stopped responding to requests until restart
Hi,
I am using WAS CE 2.0.0.2 with ActiveMQ 5.2.0. While doing load
testing, I encountered situations when the web server (which is the
Tomcat embedded in WAS CE) permanently (until restart) stopped
responding to any HTTP requests (the catalina log stops logging new
requests), and none of the apps nor the admin console can be accessed
anymore. When that happens, the WAS CE java process' CPU utilization
would invariably be permanently stuck at around the 50% level, even if
I completely stop accessing the server.
I did a thread dump whenever this occurred, and most of the time I do
not see any blocked threads, only runnable, parked and
waiting-on-condition threads. Once, I was able to see that there were
4 blocked ActiveMQ threads, though I'm not sure if this is related.
I would be most grateful if someone could help guide me to possible
causes of the problem.
Thank you
Re: Web server stopped responding to requests until restart
Posted by viola lu <vi...@gmail.com>.
from my side, it's http server configuration limitation.After peformance
tune, it works well now.
On Tue, Mar 3, 2009 at 10:10 PM, Stella Lok <oi...@gmail.com>wrote:
> Hi,
>
> After
> doing much testing, I am able to recreate the "frozen server"
> situation on demand.
>
> Strangely enough, it seems like the problem is caused by my use of
> org.apache.commons.lang.builder.ToStringBuilder in a (non-infinite)
> loop. When I remove the usage, the problem disappears. I'll have to do
> more investigation before I can reach a conclusion though
>
> On Fri, Feb 27, 2009 at 12:42 AM, Stella Lok <oi...@gmail.com>
> wrote:
> > Hi Kevan,
> >
> > The machine has one Xeon processor which is a dual core. I am running
> > WAS CE on IBM JSE 5.
> >
> > I do have the thread dump showing the blocked MQ threads, which I have
> > just forwarded to you, if you would like to take a look (if anyone
> > else is interested in taking a look, please let me know)
> >
> > Btw, the activemq used is not the one embedded in WASCE, but an external
> one.
> >
> > Thank you very much
> >
> > On Thu, Feb 26, 2009 at 11:13 PM, Kevan Miller <ke...@gmail.com>
> wrote:
> >>
> >> On Feb 26, 2009, at 5:58 AM, Stella Lok wrote:
> >>
> >>> Hi,
> >>>
> >>> I am using WAS CE 2.0.0.2 with ActiveMQ 5.2.0. While doing load
> >>> testing, I encountered situations when the web server (which is the
> >>> Tomcat embedded in WAS CE) permanently (until restart) stopped
> >>> responding to any HTTP requests (the catalina log stops logging new
> >>> requests), and none of the apps nor the admin console can be accessed
> >>> anymore. When that happens, the WAS CE java process' CPU utilization
> >>> would invariably be permanently stuck at around the 50% level, even if
> >>> I completely stop accessing the server.
> >>>
> >>> I did a thread dump whenever this occurred, and most of the time I do
> >>> not see any blocked threads, only runnable, parked and
> >>> waiting-on-condition threads. Once, I was able to see that there were
> >>> 4 blocked ActiveMQ threads, though I'm not sure if this is related.
> >>>
> >>> I would be most grateful if someone could help guide me to possible
> >>> causes of the problem.
> >>
> >> Hmm. Do you still have the thread dumps? Esp. the one showing the
> blocked
> >> AMQ threads? Is this a 2 CPU machine (e.g. a single dual core
> processor?)
> >> Sounds like you have a single thread spinning. A thread dump should help
> >> identify the culprit... Are you running on Sun or IBM JSE?
> >>
> >> --kevan
> >>
> >
>
--
viola
Re: Web server stopped responding to requests until restart
Posted by Stella Lok <oi...@gmail.com>.
Hi,
After
doing much testing, I am able to recreate the "frozen server"
situation on demand.
Strangely enough, it seems like the problem is caused by my use of
org.apache.commons.lang.builder.ToStringBuilder in a (non-infinite)
loop. When I remove the usage, the problem disappears. I'll have to do
more investigation before I can reach a conclusion though
On Fri, Feb 27, 2009 at 12:42 AM, Stella Lok <oi...@gmail.com> wrote:
> Hi Kevan,
>
> The machine has one Xeon processor which is a dual core. I am running
> WAS CE on IBM JSE 5.
>
> I do have the thread dump showing the blocked MQ threads, which I have
> just forwarded to you, if you would like to take a look (if anyone
> else is interested in taking a look, please let me know)
>
> Btw, the activemq used is not the one embedded in WASCE, but an external one.
>
> Thank you very much
>
> On Thu, Feb 26, 2009 at 11:13 PM, Kevan Miller <ke...@gmail.com> wrote:
>>
>> On Feb 26, 2009, at 5:58 AM, Stella Lok wrote:
>>
>>> Hi,
>>>
>>> I am using WAS CE 2.0.0.2 with ActiveMQ 5.2.0. While doing load
>>> testing, I encountered situations when the web server (which is the
>>> Tomcat embedded in WAS CE) permanently (until restart) stopped
>>> responding to any HTTP requests (the catalina log stops logging new
>>> requests), and none of the apps nor the admin console can be accessed
>>> anymore. When that happens, the WAS CE java process' CPU utilization
>>> would invariably be permanently stuck at around the 50% level, even if
>>> I completely stop accessing the server.
>>>
>>> I did a thread dump whenever this occurred, and most of the time I do
>>> not see any blocked threads, only runnable, parked and
>>> waiting-on-condition threads. Once, I was able to see that there were
>>> 4 blocked ActiveMQ threads, though I'm not sure if this is related.
>>>
>>> I would be most grateful if someone could help guide me to possible
>>> causes of the problem.
>>
>> Hmm. Do you still have the thread dumps? Esp. the one showing the blocked
>> AMQ threads? Is this a 2 CPU machine (e.g. a single dual core processor?)
>> Sounds like you have a single thread spinning. A thread dump should help
>> identify the culprit... Are you running on Sun or IBM JSE?
>>
>> --kevan
>>
>
Re: Web server stopped responding to requests until restart
Posted by "viola.lu" <vi...@gmail.com>.
HI, I seems met the same question.If i just set up http server+wasce without
activemq, http requests are served well, but after add activemq, http
requests success percent became lower.
stella lok wrote:
>
> Hi Kevan,
>
> The machine has one Xeon processor which is a dual core. I am running
> WAS CE on IBM JSE 5.
>
> I do have the thread dump showing the blocked MQ threads, which I have
> just forwarded to you, if you would like to take a look (if anyone
> else is interested in taking a look, please let me know)
>
> Btw, the activemq used is not the one embedded in WASCE, but an external
> one.
>
> Thank you very much
>
> On Thu, Feb 26, 2009 at 11:13 PM, Kevan Miller <ke...@gmail.com>
> wrote:
>>
>> On Feb 26, 2009, at 5:58 AM, Stella Lok wrote:
>>
>>> Hi,
>>>
>>> I am using WAS CE 2.0.0.2 with ActiveMQ 5.2.0. While doing load
>>> testing, I encountered situations when the web server (which is the
>>> Tomcat embedded in WAS CE) permanently (until restart) stopped
>>> responding to any HTTP requests (the catalina log stops logging new
>>> requests), and none of the apps nor the admin console can be accessed
>>> anymore. When that happens, the WAS CE java process' CPU utilization
>>> would invariably be permanently stuck at around the 50% level, even if
>>> I completely stop accessing the server.
>>>
>>> I did a thread dump whenever this occurred, and most of the time I do
>>> not see any blocked threads, only runnable, parked and
>>> waiting-on-condition threads. Once, I was able to see that there were
>>> 4 blocked ActiveMQ threads, though I'm not sure if this is related.
>>>
>>> I would be most grateful if someone could help guide me to possible
>>> causes of the problem.
>>
>> Hmm. Do you still have the thread dumps? Esp. the one showing the blocked
>> AMQ threads? Is this a 2 CPU machine (e.g. a single dual core processor?)
>> Sounds like you have a single thread spinning. A thread dump should help
>> identify the culprit... Are you running on Sun or IBM JSE?
>>
>> --kevan
>>
>
>
--
View this message in context: http://www.nabble.com/Web-server-stopped-responding-to-requests-until-restart-tp22221898s134p22238065.html
Sent from the Apache Geronimo - Users mailing list archive at Nabble.com.
Re: Web server stopped responding to requests until restart
Posted by Stella Lok <oi...@gmail.com>.
Hi Kevan,
The machine has one Xeon processor which is a dual core. I am running
WAS CE on IBM JSE 5.
I do have the thread dump showing the blocked MQ threads, which I have
just forwarded to you, if you would like to take a look (if anyone
else is interested in taking a look, please let me know)
Btw, the activemq used is not the one embedded in WASCE, but an external one.
Thank you very much
On Thu, Feb 26, 2009 at 11:13 PM, Kevan Miller <ke...@gmail.com> wrote:
>
> On Feb 26, 2009, at 5:58 AM, Stella Lok wrote:
>
>> Hi,
>>
>> I am using WAS CE 2.0.0.2 with ActiveMQ 5.2.0. While doing load
>> testing, I encountered situations when the web server (which is the
>> Tomcat embedded in WAS CE) permanently (until restart) stopped
>> responding to any HTTP requests (the catalina log stops logging new
>> requests), and none of the apps nor the admin console can be accessed
>> anymore. When that happens, the WAS CE java process' CPU utilization
>> would invariably be permanently stuck at around the 50% level, even if
>> I completely stop accessing the server.
>>
>> I did a thread dump whenever this occurred, and most of the time I do
>> not see any blocked threads, only runnable, parked and
>> waiting-on-condition threads. Once, I was able to see that there were
>> 4 blocked ActiveMQ threads, though I'm not sure if this is related.
>>
>> I would be most grateful if someone could help guide me to possible
>> causes of the problem.
>
> Hmm. Do you still have the thread dumps? Esp. the one showing the blocked
> AMQ threads? Is this a 2 CPU machine (e.g. a single dual core processor?)
> Sounds like you have a single thread spinning. A thread dump should help
> identify the culprit... Are you running on Sun or IBM JSE?
>
> --kevan
>
Re: Web server stopped responding to requests until restart
Posted by Kevan Miller <ke...@gmail.com>.
On Feb 26, 2009, at 5:58 AM, Stella Lok wrote:
> Hi,
>
> I am using WAS CE 2.0.0.2 with ActiveMQ 5.2.0. While doing load
> testing, I encountered situations when the web server (which is the
> Tomcat embedded in WAS CE) permanently (until restart) stopped
> responding to any HTTP requests (the catalina log stops logging new
> requests), and none of the apps nor the admin console can be accessed
> anymore. When that happens, the WAS CE java process' CPU utilization
> would invariably be permanently stuck at around the 50% level, even if
> I completely stop accessing the server.
>
> I did a thread dump whenever this occurred, and most of the time I do
> not see any blocked threads, only runnable, parked and
> waiting-on-condition threads. Once, I was able to see that there were
> 4 blocked ActiveMQ threads, though I'm not sure if this is related.
>
> I would be most grateful if someone could help guide me to possible
> causes of the problem.
Hmm. Do you still have the thread dumps? Esp. the one showing the
blocked AMQ threads? Is this a 2 CPU machine (e.g. a single dual core
processor?) Sounds like you have a single thread spinning. A thread
dump should help identify the culprit... Are you running on Sun or IBM
JSE?
--kevan