[users@httpd] slowness with mod_proxy

[Matthew Pressly <mp...@claborn.net>]

I'm having trouble with a mod_proxy/mod_rewrite reverse
proxy being very slow.  Any help in diagnosing the
cause of the slowness would be greatly appreciated.

The setup is as follows.  There are two servers (Dell 2450)
each running apache 1.3.27.  The front end server
is configured to proxy certain requests through
to the backend server over a separate network
connection (10.0.0.*).

The setup works well under very light load and even
under heavy load generated by apache bench (ab), but
slows down dramatically when I switch over the DNS
to put it into production if there is light to
moderate load.  I have not come up with any reliable
way to reproduce the slowness without using real
load generated by people using the site.
It makes me think the slowness could be somehow
related to DNS lookups, but I do have HostnameLookups off.

When it gets slow, pages take anywhere from about 5 to
30 seconds longer to load through the proxy than 
they do directly from the backend server.

I have not been able to strace or ltrace httpd because
of a security fix on SuSE 8.2 that broke ptrace for
processes that change their effective uid.

Here are relavent portions of httpd.conf on the proxy:

# Apache 1.3.27
MinSpareServers 50
MaxSpareServers 150
StartServers 150
MaxClients 150

HostnameLookups Off
<VirtualHost *:80>
  ServerName www4.myserver1.com
  ServerAlias www.myserver1.com
  ServerAlias myserver1.com
  ServerAlias myserver2.com
  ServerAlias www4.myserver2.com
  DocumentRoot /home/prod/webhome/myserver2/site
  ErrorLog /var/log/httpd/error_log
  CustomLog /var/log/httpd/access_log combined
  RewriteEngine on
  RewriteLog /var/log/httpd/rewrite_log
  RewriteLogLevel 0
  ProxyReceiveBufferSize 16384
  RewriteMap servermap /home/prod/webhome/project/conf/myserver1_server_map.txt 
  # index.html - redirect to a more sensible place
  RewriteRule ^/$ http://${servermap:static}/w1/index.html [R,L]
  RewriteRule ^/index.html$ http://${servermap:static}/w1/index.html [R,L]

  # Handle all static content locally
  RewriteRule ^(/surveyimages|/respondentreport|/reports.*)$ $1 [PT,L]
  RewriteCond %{REQUEST_URI} \.(jpg|gif|pdf|png|html|js|html|css)$ 
  RewriteRule ^(.*)$ $1 [PT,L]

  # Send http:...admin through https
  RewriteCond %{SERVER_PORT} !^443$
  RewriteRule ^/admin(.*) https://www.myserver2.com/admin$1 [R,L]

  # Proxy all admin access
  RewriteRule ^(/admin/.*)$ http://${servermap:admin}$1 [P,L]
  RewriteRule ^(/admin)$ http://${servermap:admin}$1/ [P,L]

  # Proxy all client access
  RewriteRule ^(/client/.*)$ http://${servermap:client}$1 [P,L]
  RewriteRule ^(/client)$ http://${servermap:client}$1/ [P,L]
  # All that should be left are mod perl requests that are not admin
  # are client, so proxy them through to a modperl frontend server
  # - nonrotating, but subject to change later.
  RewriteRule ^(.*)$ http://${servermap:modperlfrontend}$1 [P,L]

  # Now, anything that is left is an error - return forbidden
  RewriteRule ^(.*)$ http://www.myserver1.com$1 [F,L]
</VirtualHost>
<VirtualHost *:443>
  SSLEngine on
  SSLCertificateFile /etc/certfile
  SSLCertificateKeyFile /etc/certkeyfile
  <Files ~ "\.(cgi|shtml)$">
    SSLOptions +StdEnvVars
  </Files> 
  CustomLog /var/log/httpd/ssl-access_log combined

  ServerName www4.myserver1.com:443
  ServerAlias www.myserver1.com:443
  ServerAlias myserver1.com:443
  ServerAlias myserver2.com:443
  ServerAlias www4.myserver2.com:443
  DocumentRoot /home/prod/webhome/project/site
  ErrorLog /var/log/httpd/error_log
  CustomLog combined /var/log/httpd/access_log
  RewriteEngine on
  RewriteLog /var/log/httpd/rewrite_log
  RewriteLogLevel 0
  ProxyReceiveBufferSize 16384
  RewriteMap servermap /home/prod/webhome/project/conf/myserver1_server_map.txt 

  # index.html - redirect to a more sensible place
  RewriteRule ^/$ http://${servermap:static}/w1/index.html [R,L]
  RewriteRule ^/index.html$ http://${servermap:static}/w1/index.html [R,L]

  # Handle all static content locally
  RewriteRule ^(/surveyimages|/respondentreport|/reports.*)$ $1 [PT,L]
  RewriteCond %{REQUEST_URI} \.(jpg|gif|pdf|png|html|js|html|css)$ 
  RewriteRule ^(.*)$ $1 [PT,L]

  # Proxy all admin access
  RewriteRule ^(/admin/.*)$ http://${servermap:admin}$1 [P,L]
  RewriteRule ^(/admin)$ http://${servermap:admin}$1/ [P,L]

  # Proxy all client access
  RewriteRule ^(/client/.*)$ http://${servermap:client}$1 [P,L]
  RewriteRule ^(/client)$ http://${servermap:client}$1/ [P,L]

  # All that should be left are mod perl requests that are not admin
  # are client, so proxy them through to a modperl frontend server
  # - nonrotating, but subject to change later.
  RewriteRule ^(.*)$ http://${servermap:modperlfrontend}$1 [P,L]

  # Now, anything that is left is an error - return forbidden
  RewriteRule ^(.*)$ http://www.myserver1.com$1 [F,L]
</VirtualHost>

Here are the contents of the map file:
# Access the backend servers through a separate 10.* network
# This are in this file so that I can change them on the fly if
# needed without having to restart the proxy httpd.
static www4.myserver1.com
admin 10.0.0.19:8001 
client 10.0.0.19:8001
modperlfrontend 10.0.0.19:8001

The backend server is listening on port 8001.

Any help or pointers would be greatly appreciated.

--
Matthew


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org