FW: Apache + mod_jk + Tomcat Configuration Help

[Emre <em...@teneke.com>]

I need serious help with Tomcat and mod_jk configuration.

Currently I am running a website that gets about 1 million page hits a day.
Lately, there has been serious issues with my Tomcat servers. I believe this
is due to higher than average traffic. This prompts me to better configure
and fine tune my Tomcat and Apache servers.

When the usage is low, I observer sub 1 second processing times per jsp page
(per my mod_jk log file).

I am using Apache 1.3.30, mod_jk 1.2.5, and Tomcat 4.1.30

Currently I am running 4 Tomcat instances on one server and the Apache HTTP
server on another.
The following is the Connector setting I have for each Tomcat instance:
  
<Connector className="org.apache.coyote.tomcat4.CoyoteConnector"
acceptCount="100" bufferSize="2048" compression="off" connectionLinger="-1"
connectionTimeout="0" connectionUploadTimeout="300000" debug="0"
disableUploadTimeout="false" enableLookups="true" maxKeepAliveRequests="400"
maxProcessors="1000" minProcessors="50" address="192.168.0.4" port="8009"
protocolHandlerClassName="org.apache.jk.server.JkCoyoteHandler"
proxyPort="0" redirectPort="8443" scheme="http" secure="false"
serverSocketTimeout="0" tcpNoDelay="true" tomcatAuthentication="true"
useBodyEncodingForURI="true" useURIValidationHack="false">
      <Factory
className="org.apache.catalina.net.DefaultServerSocketFactory"/>
    </Connector>

The machine that is running Tomcat servers has Dual Xeon 2.4GHz processors
with 4GB of RAM.

At peak hours, we have about 3000 simultaneous users logged in.

I already have my Apache MaxConnections set to 5000, therefore I know there
is no problem there.

Can someone please give me advise on how I should be configuring my
Connectors?

Also, I noticed that load balancing lbfactor in mod_jk is not working
properly, given each worker has the same lbfactor value, the first one in
the workers list gets many more requests than the second one. And the second
one gets more than the third one, and so on.

Bottom line: I am looking for recommendations to better configure my app
servers.

Thanx in advance



---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-user-help@jakarta.apache.org

Re: FW: Apache + mod_jk + Tomcat Configuration Help

[Graham Bleach <gr...@darkskills.org.uk>]

On Wed, Jun 02, 2004 at 03:42:24PM -0400, Emre wrote:
> 
> I need serious help with Tomcat and mod_jk configuration.
> 
> Currently I am running a website that gets about 1 million page hits a day.
> Lately, there has been serious issues with my Tomcat servers. I believe this
> is due to higher than average traffic. This prompts me to better configure
> and fine tune my Tomcat and Apache servers.
> 
> When the usage is low, I observer sub 1 second processing times per jsp page
> (per my mod_jk log file).
> 
> I am using Apache 1.3.30, mod_jk 1.2.5, and Tomcat 4.1.30
> 
> Currently I am running 4 Tomcat instances on one server and the Apache HTTP
> server on another.
> The following is the Connector setting I have for each Tomcat instance:
>   
> <Connector className="org.apache.coyote.tomcat4.CoyoteConnector"
> acceptCount="100" bufferSize="2048" compression="off" connectionLinger="-1"
> connectionTimeout="0" connectionUploadTimeout="300000" debug="0"
> disableUploadTimeout="false" enableLookups="true" maxKeepAliveRequests="400"
> maxProcessors="1000" minProcessors="50" address="192.168.0.4" port="8009"
> protocolHandlerClassName="org.apache.jk.server.JkCoyoteHandler"
> proxyPort="0" redirectPort="8443" scheme="http" secure="false"
> serverSocketTimeout="0" tcpNoDelay="true" tomcatAuthentication="true"
> useBodyEncodingForURI="true" useURIValidationHack="false">
>       <Factory
> className="org.apache.catalina.net.DefaultServerSocketFactory"/>
>     </Connector>
> 
> The machine that is running Tomcat servers has Dual Xeon 2.4GHz processors
> with 4GB of RAM.

How busy is this machine? Memory usage? CPU usage? Are you not sure that
you are just overloading these boxes? Or that there is some contention
within your app?

How many connections are there to port 8009?

Check that you are not running out of AJP processors and consider
reducing the acceptCount, as it might be causing users to wait if they
get directed to a busy tomcat instance. Look in the catalina_log.* files
for the messages that indicate that all your processors are busy.

> At peak hours, we have about 3000 simultaneous users logged in.
> 
> I already have my Apache MaxConnections set to 5000, therefore I know there
> is no problem there.

I've never heard of MaxConnections, do you mean MaxClients? If so, note
that it could mean that you don't have enough AJP processors, since with
4000 tomcat connectors, they could easily all be busy. Note also that,
dependent on the value of socket_timeout/cache_timeout in your
worker.properties file, each apache process could have a connection open
to each tomcat server.

> Can someone please give me advise on how I should be configuring my
> Connectors?

It would be better if you also posted your worker.properties and
relevant sections of your httpd.conf.

> Also, I noticed that load balancing lbfactor in mod_jk is not working
> properly, given each worker has the same lbfactor value, the first one in
> the workers list gets many more requests than the second one. And the second
> one gets more than the third one, and so on.

I'm afraid this is seen with apache running in pre-fork mode (I asked a
similar question a while ago). Each apache process always starts with the
first worker in the list and work its way through. I would look at
doing one or more of the following:

- Decreasing StartServers to reduce the initial high skew towards the
first worker when tomcat is first started.
- Increasing MaxRequestsPerChild if it is set to a low value, or
removing it completely, as this can cause a very skewed bias towards the
first server in the list as apache processes get respawned and hit the
first tomcat server again.
- Fiddle with socket_timeout and possibly reduce it, so that connections
to the servers get cleaned up quickly.

G
-- 
The illegal we do immediately. The unconstitutional takes a little longer.
        -- Henry Kissinger

---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-user-help@jakarta.apache.org