You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Edward Quick <Ed...@iggroup.com> on 2011/07/13 17:10:11 UTC
mod_jk question about lingering close_waits
Hi,
Apologies in advance if this is the wrong mailing list. I was unable to find one specific to mod_jk and this looked the most relevant.
I have the following apache tomcat server configuration set up: Apache 2.2.3 , mod_jk 1.2.32 , and tomcat 6.0.32 running HelloWorld.war
If I send a request to http://remotehost/HelloWorld/index.jsp, I observe the following behaviour:
1 ) My tomcat AJP connector is listening on port 8010
TEST [root@remotehost]$ netstat -an | grep 8010
tcp 0 0 0.0.0.0:8010 0.0.0.0:* LISTEN
2) I make a request to http://remotehost/HelloWorld/index.jsp and see the apache establish a connection to tomcat. The response comes back straightaway.
TEST [root@remotehost]$ netstat -an | grep 8010
tcp 0 0 0.0.0.0:8010 0.0.0.0:* LISTEN
tcp 0 0 10.34.2.140:8010 10.34.2.140:46792 ESTABLISHED
tcp 0 0 10.34.2.140:46792 10.34.2.140:8010 ESTABLISHED
3) After reaching KeepAliveTimeout on the Tomcat AJP connector, the connections change state:
TEST [root@ remotehost]$ netstat -an | grep 8010
tcp 0 0 0.0.0.0:8010 0.0.0.0:* LISTEN
tcp 0 0 10.34.2.140:8010 10.34.2.140:46792 FIN_WAIT2
tcp 1 0 10.34.2.140:46792 10.34.2.140:8010 CLOSE_WAIT
4) After about another minute, the FIN_WAIT2 is cleared, but the CLOSE_WAIT remains (on the apache) until reaching net.ipv4.tcp_keepalive_time (which is set to 7200 seconds)
TEST [root@ remotehost]$ netstat -an | grep 8010
tcp 0 0 0.0.0.0:8010 0.0.0.0:* LISTEN
tcp 1 0 10.34.2.140:46792 10.34.2.140:8010 CLOSE_WAIT
Everything there looks fine except for the CLOSE_WAIT which lingers around for 2 hours until it hits tcp_keepalive_time.
Adding JkOptions +DisableReuse will prevent the CLOSE_WAITs staying, but that is also said to have a performance impact.
My question is if this is the expected tcpip state after the thread has finished, because in JDBC I see the connections permanently ESTABLISHED or in TIME_WAIT.
Thanks for any help.
Ed.
________________________________
The information contained in this email is strictly confidential and for the use of the addressee only, unless otherwise indicated. If you are not the intended recipient, please do not read, copy, use or disclose to others this message or any attachment. Please also notify the sender by replying to this email or by telephone (+44 (0)20 7896 0011) and then delete the email and any copies of it. Opinions, conclusions (etc) that do not relate to the official business of this company shall be understood as neither given nor endorsed by it. IG Group Holdings plc is a company registered in England and Wales under number 01190902. VAT registration number 761 2978 07. Registered Office: Cannon Bridge House, 25 Dowgate Hill, London EC4R 2YA. Authorised and regulated by the Financial Services Authority. FSA Register number 114059.
Re: mod_jk question about lingering close_waits
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
André,
On 7/13/2011 5:16 PM, André Warnier wrote:
> 1) this is the right list for Apache/Tomcat connectors (mod_jk among
> them)
Yes. When the question is about mod_proxy_ajp, we usually try our best
but defer to the Apache httpd user's list when things get too hairy.
This /is/ the right place for mod_jk questions.
> 2) a question : do these CLOSE_WAIT sockets bother you for some
> specific reason ? In a totally different context, I have had problems
> with Linux systems when hundreds of such sockets accumulated over
> time (entire TCP stack becoming unresponsive at some point), but they
> do not seem to have an impact when the number remains "reasonable"
> (below one hundred or so).
+1
The question is why the AJP connections are being terminated in the
first place. AJP expects to have a persistent connection, and usually
doesn't shut them down. Do you have a firewall between the web server
and Tomcat?
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk4e+F0ACgkQ9CaO5/Lv0PDbTQCgpIBnfoETXAqz77YBzX0SOkQo
HxIAoI4kwXcvaQmarcaVNVlHzGq+9LO9
=NjPY
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk question about lingering close_waits
Posted by Rainer Jung <ra...@kippdata.de>.
On 15.07.2011 12:09, Edward Quick wrote:
> Thanks for your replies. I found the solution was to set JkWatchdogInterval (by default this is not on). I set it to 30, and observed the FIN_WAIT disappears after 60 seconds (this could be the worker.maintain setting?) and the CLOSE_WAIT thread disappeared when it reached the connection_pool_timeout.
>
> Does this mean connection_pool_timeout only works if JkWatchdogInterval is swithed on?
No, it would also be checked, if a process gets a rquest to handle. So
if there is traffic it usually works fine without the watchdog thread.
But since pools are per process and we can't assume that each of the
processes with idle connections will get new requests soon, t is usually
better to switch on the watchdog.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: mod_jk question about lingering close_waits
Posted by Edward Quick <Ed...@iggroup.com>.
Thanks for your replies. I found the solution was to set JkWatchdogInterval (by default this is not on). I set it to 30, and observed the FIN_WAIT disappears after 60 seconds (this could be the worker.maintain setting?) and the CLOSE_WAIT thread disappeared when it reached the connection_pool_timeout.
Does this mean connection_pool_timeout only works if JkWatchdogInterval is swithed on?
Ed.
-----Original Message-----
From: Rainer Jung [mailto:rainer.jung@kippdata.de]
Sent: 13 July 2011 23:23
To: users@tomcat.apache.org
Subject: Re: mod_jk question about lingering close_waits
On 13.07.2011 23:16, André Warnier wrote:
> Hi.
>
> I am not the one who can really answer your question, but
>
> 1) this is the right list for Apache/Tomcat connectors (mod_jk among them)
>
> 2) a question : do these CLOSE_WAIT sockets bother you for some specific
> reason ?
> In a totally different context, I have had problems with Linux systems
> when hundreds of such sockets accumulated over time (entire TCP stack
> becoming unresponsive at some point), but they do not seem to have an
> impact when the number remains "reasonable" (below one hundred or so).
> In this case, as I understand things, there will be a maximum of N such
> sockets, where N is the maximum number of threads which Tomcat may have,
> at some point, running simultaneously from this AJP Connector. (And it
> is also the number of connections which mod_jk will automatically
> configure in its pool).
> On many of the sites which I take care of, I regularly see 25-50 such
> mod_jk sockets in CLOSE_WAIT state for extended periods of time, and
> they do not seem to have any negative consequences on the system. They
> just clutter the netstat displays, but you can always "grep -v" them out.
3) Look at the extensive sample configuration provided by the mod_jk
1.2.32 source download. There are many interesting worker properties,
for example idle timeouts, which should reduce the CLOSE_WAIT counts.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
The information contained in this email is strictly confidential and for the use of the addressee only, unless otherwise indicated. If you are not the intended recipient, please do not read, copy, use or disclose to others this message or any attachment. Please also notify the sender by replying to this email or by telephone (+44 (0)20 7896 0011) and then delete the email and any copies of it. Opinions, conclusions (etc) that do not relate to the official business of this company shall be understood as neither given nor endorsed by it. IG Group Holdings plc is a company registered in England and Wales under number 01190902. VAT registration number 761 2978 07. Registered Office: Cannon Bridge House, 25 Dowgate Hill, London EC4R 2YA. Authorised and regulated by the Financial Services Authority. FSA Register number 114059.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk question about lingering close_waits
Posted by Rainer Jung <ra...@kippdata.de>.
On 13.07.2011 23:16, André Warnier wrote:
> Hi.
>
> I am not the one who can really answer your question, but
>
> 1) this is the right list for Apache/Tomcat connectors (mod_jk among them)
>
> 2) a question : do these CLOSE_WAIT sockets bother you for some specific
> reason ?
> In a totally different context, I have had problems with Linux systems
> when hundreds of such sockets accumulated over time (entire TCP stack
> becoming unresponsive at some point), but they do not seem to have an
> impact when the number remains "reasonable" (below one hundred or so).
> In this case, as I understand things, there will be a maximum of N such
> sockets, where N is the maximum number of threads which Tomcat may have,
> at some point, running simultaneously from this AJP Connector. (And it
> is also the number of connections which mod_jk will automatically
> configure in its pool).
> On many of the sites which I take care of, I regularly see 25-50 such
> mod_jk sockets in CLOSE_WAIT state for extended periods of time, and
> they do not seem to have any negative consequences on the system. They
> just clutter the netstat displays, but you can always "grep -v" them out.
3) Look at the extensive sample configuration provided by the mod_jk
1.2.32 source download. There are many interesting worker properties,
for example idle timeouts, which should reduce the CLOSE_WAIT counts.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk question about lingering close_waits
Posted by André Warnier <aw...@ice-sa.com>.
Hi.
I am not the one who can really answer your question, but
1) this is the right list for Apache/Tomcat connectors (mod_jk among them)
2) a question : do these CLOSE_WAIT sockets bother you for some specific reason ?
In a totally different context, I have had problems with Linux systems when hundreds of
such sockets accumulated over time (entire TCP stack becoming unresponsive at some point),
but they do not seem to have an impact when the number remains "reasonable" (below one
hundred or so).
In this case, as I understand things, there will be a maximum of N such sockets, where N
is the maximum number of threads which Tomcat may have, at some point, running
simultaneously from this AJP Connector. (And it is also the number of connections which
mod_jk will automatically configure in its pool).
On many of the sites which I take care of, I regularly see 25-50 such mod_jk sockets in
CLOSE_WAIT state for extended periods of time, and they do not seem to have any negative
consequences on the system. They just clutter the netstat displays, but you can always
"grep -v" them out.
Edward Quick wrote:
> Hi,
>
> Apologies in advance if this is the wrong mailing list. I was unable to find one specific to mod_jk and this looked the most relevant.
>
> I have the following apache tomcat server configuration set up: Apache 2.2.3 , mod_jk 1.2.32 , and tomcat 6.0.32 running HelloWorld.war
> If I send a request to http://remotehost/HelloWorld/index.jsp, I observe the following behaviour:
>
>
> 1 ) My tomcat AJP connector is listening on port 8010
>
> TEST [root@remotehost]$ netstat -an | grep 8010
> tcp 0 0 0.0.0.0:8010 0.0.0.0:* LISTEN
>
>
> 2) I make a request to http://remotehost/HelloWorld/index.jsp and see the apache establish a connection to tomcat. The response comes back straightaway.
>
> TEST [root@remotehost]$ netstat -an | grep 8010
> tcp 0 0 0.0.0.0:8010 0.0.0.0:* LISTEN
> tcp 0 0 10.34.2.140:8010 10.34.2.140:46792 ESTABLISHED
> tcp 0 0 10.34.2.140:46792 10.34.2.140:8010 ESTABLISHED
>
> 3) After reaching KeepAliveTimeout on the Tomcat AJP connector, the connections change state:
>
> TEST [root@ remotehost]$ netstat -an | grep 8010
> tcp 0 0 0.0.0.0:8010 0.0.0.0:* LISTEN
> tcp 0 0 10.34.2.140:8010 10.34.2.140:46792 FIN_WAIT2
> tcp 1 0 10.34.2.140:46792 10.34.2.140:8010 CLOSE_WAIT
>
> 4) After about another minute, the FIN_WAIT2 is cleared, but the CLOSE_WAIT remains (on the apache) until reaching net.ipv4.tcp_keepalive_time (which is set to 7200 seconds)
>
> TEST [root@ remotehost]$ netstat -an | grep 8010
> tcp 0 0 0.0.0.0:8010 0.0.0.0:* LISTEN
> tcp 1 0 10.34.2.140:46792 10.34.2.140:8010 CLOSE_WAIT
>
>
> Everything there looks fine except for the CLOSE_WAIT which lingers around for 2 hours until it hits tcp_keepalive_time.
> Adding JkOptions +DisableReuse will prevent the CLOSE_WAITs staying, but that is also said to have a performance impact.
>
> My question is if this is the expected tcpip state after the thread has finished, because in JDBC I see the connections permanently ESTABLISHED or in TIME_WAIT.
>
> Thanks for any help.
>
> Ed.
>
> ________________________________
> The information contained in this email is strictly confidential and for the use of the addressee only, unless otherwise indicated. If you are not the intended recipient, please do not read, copy, use or disclose to others this message or any attachment. Please also notify the sender by replying to this email or by telephone (+44 (0)20 7896 0011) and then delete the email and any copies of it. Opinions, conclusions (etc) that do not relate to the official business of this company shall be understood as neither given nor endorsed by it. IG Group Holdings plc is a company registered in England and Wales under number 01190902. VAT registration number 761 2978 07. Registered Office: Cannon Bridge House, 25 Dowgate Hill, London EC4R 2YA. Authorised and regulated by the Financial Services Authority. FSA Register number 114059.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org