number of established connections keep growing

[Victor Granic <vi...@yahoo.com>]

Hello!

I'm running Tomcat 5.0.28 on a Windows 2000 server.  Tomcat is wrapped
using the Java Service Wrapper.  All connections to the Tomcat service
come from a proxy running apache+mod_jk on linux.

Problem:  The number of ESTABLISHED connections continue to grow on the
Windows server until users are unable to access the web site.  I can
see that at any given time on the proxy there are really only around 20
established connections to the Tomcat service but running netstat on
the Windows machine displays close to 200 connections with the proxy in
the established state.

I found this issue posted to the list in the past but no definitive
solutions were mentioned.  I don't want to increase some TCP connection
limit or tomcat cache size, I'd like to understand what's preventing
those connections from being torn down properly.  If it's a bug that's
fixed in 5.5, that would be good to know.  I even noticed one thread
where the same problem was experience on a linux machine so it may be
that it's not specific to Tomcat on Windows.

Any insight is greatly appreciated.

Thanks,

Victor


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: number of established connections keep growing

[Victor Granic <vi...@yahoo.com>]

--- Mladen Turk <ml...@jboss.com> wrote:

> Victor Granic wrote:
> > Hello!
> > 
> > I'm running Tomcat 5.0.28 on a Windows 2000 server.  Tomcat is
> wrapped
> > using the Java Service Wrapper.  All connections to the Tomcat
> service
> > come from a proxy running apache+mod_jk on linux.
> > 
> > Problem:  The number of ESTABLISHED connections continue to grow on
> the
> > Windows server until users are unable to access the web site.
> 
> 
> This is because mod_jk uses constant connection pool, and once
> when the connection is established it stays open for the
> server lifetime.
> The reason why connections are growing is because the Apache will
> create up to MaxClient connections.

That's true.  Most of the time the proxy has around 20 established
connections to Tomcat.  The number of connections is probably growing
during times of high traffic and they close when they become unused. 
But, why aren't the sockets closing on the Tomcat server?


> Also if you have a firewall between mod_jk and Tomcat that tends
> to cut the inactive connections, the Tomcat connection will be
> half-closed. Apache will reconnect and your connection count will
> rise.
> 

I do have a firewall between mod_jk and Tomcat but it has been
configured with very high inactivity timeouts so I don't think it is
interfering with the persistent connections from the proxy.  On the
proxy server I see fully established connections to Tomcat and on the
Tomcat server I see those same fully established connections and an
additional 150 or more stale established connections.  They could very
well be in a half open state.  But why?  Why didn't they close when the
FIN was received/sent?  A 18 hour tcpdump on either side of the
firewall shows that FIN/ACKs are being sent and received by both the
proxy and Tomcat server.


> The solution is to set the connectionTimeout in server.xml
> that will close the inactive connections. This number should be
> large, like 10 ... 30 minutes, so that performance doesn't suffer.
> 

It looks like this parameter is not set for the active connector. 
Tomcat documentation states that the default is 60 seconds.  I'm
wondering if it's possible that in reality, if it's not set then a
timeout will never be reached which could be the cause of my problem.  

I'm a little confused by you're recommendation since it is a far
greater timeout than the default and the main issue is that these
half-closed connections are lingering and multiplying, effectively
rendering the service unavailable after a few days.  Would a lower
timeout be a better cure?

> Regards,
> Mladen.
> 

Thanks for the helpful response Mladen!

Victor


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: number of established connections keep growing

[Mladen Turk <ml...@jboss.com>]

Victor Granic wrote:
> Hello!
> 
> I'm running Tomcat 5.0.28 on a Windows 2000 server.  Tomcat is wrapped
> using the Java Service Wrapper.  All connections to the Tomcat service
> come from a proxy running apache+mod_jk on linux.
> 
> Problem:  The number of ESTABLISHED connections continue to grow on the
> Windows server until users are unable to access the web site.


This is because mod_jk uses constant connection pool, and once
when the connection is established it stays open for the
server lifetime.
The reason why connections are growing is because the Apache will
create up to MaxClient connections.
Also if you have a firewall between mod_jk and Tomcat that tends
to cut the inactive connections, the Tomcat connection will be
half-closed. Apache will reconnect and your connection count will
rise.

The solution is to set the connectionTimeout in server.xml
that will close the inactive connections. This number should be
large, like 10 ... 30 minutes, so that performance doesn't suffer.

Regards,
Mladen.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org