os-linux/6960: httpd processes stop accepting connections

[Don Owens <do...@conru.com>]

>Number:         6960
>Category:       os-linux
>Synopsis:       httpd processes stop accepting connections
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    apache
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   apache
>Arrival-Date:   Mon Dec 11 22:40:00 PST 2000
>Closed-Date:
>Last-Modified:
>Originator:     don@conru.com
>Release:        1.3.12
>Organization:
apache
>Environment:
Linux e42.friendfinder.com 2.2.14-VA.2.1smp #1 SMP Mon Jul 31 21:53:20 PDT 2000 i686 unknown
>Description:
  I have run into a problem with some Apache processes hanging on Linux.
The webserver is fine for a little while, but then at least one child
(and likely many) stops accepting connections properly.  We have a
watchdog cron job that runs every 3 minutes to check the webserver.
About 14% of the time, the watchdog is unable to connect to the
webserver (times out after 5 minutes).  But, during this same time,
clients are able to connect to other child processes and retrieve web
pages.  The MaxClients has not been reached.

  The only changes to the code we have made is to raise the
HARD_SERVER_LIMIT to 1024, as it comes as 255 by default for Linux.
This seems to work fine for several of our servers.  But for two of our
servers, we see this problem.

Server version: Apache/1.3.12 (Unix)

Compiled-in modules:
  http_core.c
  mod_mmap_static.c
  mod_vhost_alias.c
  mod_env.c
  mod_log_config.c
  mod_log_agent.c
  mod_log_referer.c
  mod_mime_magic.c
  mod_mime.c
  mod_negotiation.c
  mod_status.c
  mod_info.c
  mod_include.c
  mod_autoindex.c
  mod_dir.c
  mod_cgi.c
  mod_asis.c
  mod_imap.c
  mod_actions.c
  mod_speling.c
  mod_userdir.c
  mod_alias.c
  mod_rewrite.c
  mod_access.c
  mod_auth.c
  mod_auth_anon.c
  mod_auth_dbm.c
  mod_auth_db.c
  mod_digest.c
  mod_proxy.c
  mod_cern_meta.c
  mod_expires.c
  mod_headers.c
  mod_usertrack.c
  mod_example.c
  mod_unique_id.c
  mod_so.c
  mod_setenvif.c
  mod_perl.c

Apache Conf:

MaxClients 525
StartServer 200
MinSpareServers 30
MaxSpareServers 80
KeepAlive Off
MaxRequestsPerChild 0
Timeout 300

Server Box:

Red Hat Linux release 6.2 (Piglet)
VA Linux Release 6.2.3 08/01/00
Kernel 2.2.14-VA.2.1smp on a 4-processor i686

Quad Xeon 700 1MB
4G Ram
Mylex 1100 Raid
100Mb intel EEPro ethernet card

>How-To-Repeat:
Go to http://adultfriendfinder.com/ and walk around the site.  After a little
while, you should see that your web browser will stall.
>Fix:

>Release-Note:
>Audit-Trail:
>Unformatted:
 [In order for any reply to be added to the PR database, you need]
 [to include <ap...@Apache.Org> in the Cc line and make sure the]
 [subject line starts with the report component and number, with ]
 [or without any 'Re:' prefixes (such as "general/1098:" or      ]
 ["Re: general/1098:").  If the subject doesn't match this       ]
 [pattern, your message will be misfiled and ignored.  The       ]
 ["apbugs" address is not added to the Cc line of messages from  ]
 [the database automatically because of the potential for mail   ]
 [loops.  If you do not include this Cc, your reply may be ig-   ]
 [nored unless you are responding to an explicit request from a  ]
 [developer.  Reply only with text; DO NOT SEND ATTACHMENTS!     ]