You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by "Hadole, Nishant IN BOM SISL" <Ni...@siemens.com> on 2009/12/04 10:29:04 UTC
Tomcat Crashes out of continuous servicing of stuck request
I am using Apache HTTP Server 2.0.61, Apache Tomcat Server 6.0.14.0 and mod_jk 2.0.46 (AJP V 1.3).
Scenario - Client call for heavy Post request from JSP. Tomcat receives the request and starts processing. Before receiving the response, client closes JSP window. Thus there is no one to receive the output.
Issue - Tomcat continues processing request indefinitely, causing loss of free heap memory and eventually crashes. When checked in Tomcat Monitor, under header jk-8009, the stage for stuck request is SERVICE and time goes on increasing.
I have checked several Time-outs setting for tomcat and AJP connectors, but still of no use.
Kindly help. Also let me know if any specific parameterization is to ne mentioned here for this.
Note: We cannot avoid client closing window while request processing is in progress.
With best regards,
Nishant Hadole
Tel.: +91 22 2495 7816
Fax: +91 22 6660 8521
Mailto: Nishant.Hadole@siemens.com
Re: Tomcat Crashes out of continuous servicing of stuck request
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Nishant,
Mailing the list is fine: there's no need to mail me separately.
On 12/6/2009 11:19 PM, Hadole, Nishant IN BOM SISL wrote:
> Tomcat crashes means, the free memory starts declining dramatically
> to zero, and server stops responding to new requests.
Does the JVM and/or Tomcat actually crash? That's an important
difference: if Tomcat simply stops responding to requests, that's likely
to be a different problem than an actual crash.
> I am sure with little modifications, this can be handled in code
> itself, and this is not a concern at all.
Eh... okay, if you say so.
> I am more interested in knowing whether there exists any
> configuration for such cases, which stops processing after some
> predefined time-out.
No.
> I know that this can be done for KEEPALIVE requests. What @ requests
> in SERVICE stage.
I'm not sure what you mean, here.
> The screenshot for Tomcat Manager is attached for the same.
The screenshot indicates that you have 756MiB free of a 1016MiB heap.
That looks good to me. It also indicates that your "jvehelp" servlet
seems to be taking forever to do its work. Perhaps you should take a
thread dump and figure out what your servlet is doing when it stalls?
Only one thread is in the "keepalive wait" state, and it has been there
for 6.5 seconds. What is your keepalive timeout? Is it set to an
appropriate amount?
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAksecS8ACgkQ9CaO5/Lv0PCH+gCffSPXVFy7KAdXKCrS3hCOr+GW
IvkAn1gSQR6dt5JFIWi2JXMXX3fAPBH7
=2YaM
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat Crashes out of continuous servicing of stuck request
Posted by "Hadole, Nishant IN BOM SISL" <Ni...@siemens.com>.
Dear Chris,
Tomcat crashes means, the free memory starts declining dramatically to zero, and server stops responding to new requests. I am sure with little modifications, this can be handled in code itself, and this is not a concern at all.
I am more interested in knowing whether there exists any configuration for such cases, which stops processing after some predefined time-out. I know that this can be done for KEEPALIVE requests. What @ requests in SERVICE stage.
The screenshot for Tomcat Manager is attached for the same.
With best regards,
Nishant Hadole
Siemens IT Solutions and Services
SIS PRO SI-I
Tel.: +91 22 2495 7816
Fax: +91 22 6660 8521
Mailto: Nishant.Hadole@siemens.com
www.siemens.co.in
-----Original Message-----
From: Christopher Schultz [mailto:chris@christopherschultz.net]
Sent: Saturday, 05 December, 2009 03:34 AM
To: Tomcat Users List
Cc: Hadole, Nishant IN BOM SISL
Subject: Re: Tomcat Crashes out of continuous servicing of stuck request
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Nishant,
On 12/4/2009 4:29 AM, Hadole, Nishant IN BOM SISL wrote:
> Tomcat continues processing request indefinitely, causing loss of
> free heap memory and eventually crashes.
When you say "crashes", what exactly to you mean? OOME? JVM failure?
> When checked in Tomcat Monitor, under header jk-8009, the stage for
> stuck request is SERVICE and time goes on increasing.
As others have said, without attempting to send data to the client, you
can't know that they have disappeared. :(
My question is why your code causes a "crash" when the client
disappears, but works just fine when the client gets a proper response.
That suggests a mismanagement of resources by your webapp. You might
consider reviewing your code to find out why your "loss of free heap
memory" is occurring, because Tomcat surely isn't causing that to happen.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAksZh0sACgkQ9CaO5/Lv0PBGfwCfVjYr8P9A0iFm6hLKkG7gxKx6
zsoAn2s5Box8os9g0dE6uFgB4TXJWPdr
=ssOC
-----END PGP SIGNATURE-----
Re: Tomcat Crashes out of continuous servicing of stuck request
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Nishant,
On 12/4/2009 4:29 AM, Hadole, Nishant IN BOM SISL wrote:
> Tomcat continues processing request indefinitely, causing loss of
> free heap memory and eventually crashes.
When you say "crashes", what exactly to you mean? OOME? JVM failure?
> When checked in Tomcat Monitor, under header jk-8009, the stage for
> stuck request is SERVICE and time goes on increasing.
As others have said, without attempting to send data to the client, you
can't know that they have disappeared. :(
My question is why your code causes a "crash" when the client
disappears, but works just fine when the client gets a proper response.
That suggests a mismanagement of resources by your webapp. You might
consider reviewing your code to find out why your "loss of free heap
memory" is occurring, because Tomcat surely isn't causing that to happen.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAksZh0sACgkQ9CaO5/Lv0PBGfwCfVjYr8P9A0iFm6hLKkG7gxKx6
zsoAn2s5Box8os9g0dE6uFgB4TXJWPdr
=ssOC
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat Crashes out of continuous servicing of stuck request
Posted by "Looijmans, Mike" <mi...@oce.com>.
Just an idea: What happens if you change your DB call into a "Sleep(30)"
or something similar? Does tomcat still misbehave then? (the 'retry'
could be related to something else than tomcat).
M
> -----Original Message-----
> From: Hadole, Nishant IN BOM SISL [mailto:Nishant.Hadole@siemens.com]
> Sent: vrijdag 04 december 2009 11:06
> To: 'Rainer Jung'; Tomcat Users List
> Subject: RE: Tomcat Crashes out of continuous servicing of
> stuck request
>
> Dear Rainer,
>
> Thanks for explanation. In this particular case, when client
> press a button on JSP, it initiates a Database search
> operation, which may take time up to 30-45 seconds.
> Meanwhile, we are showing a screen which tell user that his /
> her request is being processed and no to close the window.
>
> But, sometimes users are impatient and still close the
> window. Yes, as you suggested, it is possible to handle close
> event / stop processing by some notification, but application
> is full of such utilities, and it is too much of efforts.
>
> I am interested in some parameterization, which detects
> broken connection and automatically drops stuck request. I
> have even checked this with requests with STAGE as KEEPALIVE,
> but not working with STAGE as SERVICE. Also, I am not able to
> figure out, why the processing is repeated.
>
> With best regards,
> Nishant Hadole
> Mailto: Nishant.Hadole@siemens.com
This message and attachment(s) are intended solely for use by the addressee and may contain information that is privileged, confidential or otherwise exempt from disclosure under applicable law.
If you are not the intended recipient or agent thereof responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited.
If you have received this communication in error, please notify the sender immediately by telephone and with a 'reply' message.
Thank you for your co-operation.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat Crashes out of continuous servicing of stuck request
Posted by "Hadole, Nishant IN BOM SISL" <Ni...@siemens.com>.
Dear Rainer,
Thanks for explanation. In this particular case, when client press a button on JSP, it initiates a Database search operation, which may take time up to 30-45 seconds. Meanwhile, we are showing a screen which tell user that his / her request is being processed and no to close the window.
But, sometimes users are impatient and still close the window. Yes, as you suggested, it is possible to handle close event / stop processing by some notification, but application is full of such utilities, and it is too much of efforts.
I am interested in some parameterization, which detects broken connection and automatically drops stuck request. I have even checked this with requests with STAGE as KEEPALIVE, but not working with STAGE as SERVICE. Also, I am not able to figure out, why the processing is repeated.
With best regards,
Nishant Hadole
Mailto: Nishant.Hadole@siemens.com
-----Original Message-----
From: Rainer Jung [mailto:rainer.jung@kippdata.de]
Sent: Friday, 04 December, 2009 03:14 PM
To: Tomcat Users List
Cc: Hadole, Nishant IN BOM SISL
Subject: Re: Tomcat Crashes out of continuous servicing of stuck request
On 04.12.2009 10:29, Hadole, Nishant IN BOM SISL wrote:
> I am using Apache HTTP Server 2.0.61, Apache Tomcat Server 6.0.14.0 and mod_jk 2.0.46 (AJP V 1.3).
mod_jk 2.0.46 does not exist.
> Scenario - Client call for heavy Post request from JSP. Tomcat receives the request and starts processing. Before receiving the response, client closes JSP window. Thus there is no one to receive the output.
>
> Issue - Tomcat continues processing request indefinitely, causing loss of free heap memory and eventually crashes. When checked in Tomcat Monitor, under header jk-8009, the stage for stuck request is SERVICE and time goes on increasing.
>
> I have checked several Time-outs setting for tomcat and AJP connectors, but still of no use.
Without trying to send something back to the client, there is no way
telling the client closed the window (or pressed reload or switched to
another URL). So in order to be able to stop processing long running
stuff, you need to try sending something to the client every now and
then, and your code working on producing the real response content needs
to be stopped by some notification. You will need to implement this
yourself.
Maybe someone can provide some example code?
> Kindly help. Also let me know if any specific parameterization is to ne mentioned here for this.
>
>
> Note: We cannot avoid client closing window while request processing is in progress.
>
> With best regards,
> Nishant Hadole
> Tel.: +91 22 2495 7816
> Fax: +91 22 6660 8521
> Mailto: Nishant.Hadole@siemens.com
Regards,
Rainer
Re: Tomcat Crashes out of continuous servicing of stuck request
Posted by Rainer Jung <ra...@kippdata.de>.
On 04.12.2009 11:41, Looijmans, Mike wrote:
>
> ...
>> Without trying to send something back to the client, there is
>> no way telling the client closed the window (or pressed
>> reload or switched to another URL).
>
> I would expect the socket to be closed, which can be detected at the
> server side. The exceptions I can think of are the client crashing or a
> network disconnect.
>
> Though apache probably detects the socket's close, it has little means
It detects the close only when trying to send something via the socket.
There's no monitoring of unused sockets. Although the server received
the FIN or RST packets and changes the state of the socket internally,
there'*s no application (=Apache) code checking that state when not
actually trying to use the socket. You could write such code, but it's
not there. The closed socket is detected once the server tries to read
from or write to it.
> of informing the associated servlet because that is blocked waiting for
> the response from the database.
Exactly. Even if the web server knew, you would still have to forward
the information to the naxt hop, e.g. Tomcat (and then also the
database). The communication between Apache and Tomcat (either via http
or via ajp) doesn't have any notification facility of the form "don't
proceed working on this request". It can only detect an error on top of
request and response communication. So here, once the app actually tries
to send something back, Apache will notice the closed socket to the
client, and then close the socket to the backend itself (at least in the
case of mod_jk) and then Tomcat notices the closed socket to the web
server and throws an error itself.
> Depending on the database, it is usually also no use to try and stop -
> the query will continue its work even though the requesting user is gone
> on most DBMSes. So taking a slot in the webserver is not a big issue,
> the DB is wasting far more resources on that user.
>
> Other options to explore are dividing the big query into multiple
> smaller ones, so that you can abort sooner. Use "INTO TEMP" to store
> intermediates. That would give you the opportiunity to check whether the
> client is still listening - and you could even give the client some
> updates on progress, which may be considered a nice to have feature as
> well.
>
> Best of all would be to optimize the database and make those queries
> faster, but I guess you must have valid reasons for not doing so.
>
> M.
>
> -- My reply ends here --
>
> This message and attachment(s) are intended solely for use by the addressee and may contain information that is privileged, confidential or otherwise exempt from disclosure under applicable law.
>
> If you are not the intended recipient or agent thereof responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited.
>
> If you have received this communication in error, please notify the sender immediately by telephone and with a 'reply' message.
>
> Thank you for your co-operation.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat Crashes out of continuous servicing of stuck request
Posted by "Looijmans, Mike" <mi...@oce.com>.
...
> Without trying to send something back to the client, there is
> no way telling the client closed the window (or pressed
> reload or switched to another URL).
I would expect the socket to be closed, which can be detected at the
server side. The exceptions I can think of are the client crashing or a
network disconnect.
Though apache probably detects the socket's close, it has little means
of informing the associated servlet because that is blocked waiting for
the response from the database.
Depending on the database, it is usually also no use to try and stop -
the query will continue its work even though the requesting user is gone
on most DBMSes. So taking a slot in the webserver is not a big issue,
the DB is wasting far more resources on that user.
Other options to explore are dividing the big query into multiple
smaller ones, so that you can abort sooner. Use "INTO TEMP" to store
intermediates. That would give you the opportiunity to check whether the
client is still listening - and you could even give the client some
updates on progress, which may be considered a nice to have feature as
well.
Best of all would be to optimize the database and make those queries
faster, but I guess you must have valid reasons for not doing so.
M.
-- My reply ends here --
This message and attachment(s) are intended solely for use by the addressee and may contain information that is privileged, confidential or otherwise exempt from disclosure under applicable law.
If you are not the intended recipient or agent thereof responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited.
If you have received this communication in error, please notify the sender immediately by telephone and with a 'reply' message.
Thank you for your co-operation.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Crashes out of continuous servicing of stuck request
Posted by Rainer Jung <ra...@kippdata.de>.
On 04.12.2009 10:29, Hadole, Nishant IN BOM SISL wrote:
> I am using Apache HTTP Server 2.0.61, Apache Tomcat Server 6.0.14.0 and mod_jk 2.0.46 (AJP V 1.3).
mod_jk 2.0.46 does not exist.
> Scenario - Client call for heavy Post request from JSP. Tomcat receives the request and starts processing. Before receiving the response, client closes JSP window. Thus there is no one to receive the output.
>
> Issue - Tomcat continues processing request indefinitely, causing loss of free heap memory and eventually crashes. When checked in Tomcat Monitor, under header jk-8009, the stage for stuck request is SERVICE and time goes on increasing.
>
> I have checked several Time-outs setting for tomcat and AJP connectors, but still of no use.
Without trying to send something back to the client, there is no way
telling the client closed the window (or pressed reload or switched to
another URL). So in order to be able to stop processing long running
stuff, you need to try sending something to the client every now and
then, and your code working on producing the real response content needs
to be stopped by some notification. You will need to implement this
yourself.
Maybe someone can provide some example code?
> Kindly help. Also let me know if any specific parameterization is to ne mentioned here for this.
>
>
> Note: We cannot avoid client closing window while request processing is in progress.
>
> With best regards,
> Nishant Hadole
> Tel.: +91 22 2495 7816
> Fax: +91 22 6660 8521
> Mailto: Nishant.Hadole@siemens.com
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org