[users@httpd] Unresponsive apache webserver, memory issue

[james <ja...@wintercastle.net>]

I'm having an issue with an apache web server running on CentOS5. After a 
few days/weeks of
running the server will become unresponsive and will require a physical
reboot in order to come back online. The system is so unresponsive when
the issue occurs that login at console is not even possible. 

I have atop installed and have looked back before the crash to see what
happened process wise and I can see the http starts using a lot of
memory and CPU usage. The vmcommit jumps from 1.8 GB to 4.8GB in a
matter of a few minutes. The VSIZE of the httpd process jumps from 8.1GB
to 36.9GB. So apache is doing something -- but how can I get historical
data for this? I also see that paging is very active, probably why the
server is unresponsive. I have looked through the apache logs and system
logs and there is nothing
obvious that is consuming all that memory. I know of the server-status
module for apache but that is only useful if you can get to the server
during the crash (I can't) and doesn't have any historical data. 

Once thing I have seen from server-status while the server is responding is 
the presence of many of these type of requests to the server from the 
server itself (always to "example.com" which is the first virtualhost 
listed in the apache config). I have researched this and on the apache wiki 
they say this is NOT a problem, however I don't see this at all on my other 
webservers. There are around 50-60 entries like the below at any one time 
-- meanwhile the server is handling between 2-5 requests. 


    
        
            8-5	 -	 0/0/403	 .
            	 0.00	 2237	 0	 0.0	 0.00	 7.07
            	 ::1	 example.com 	 OPTIONS * HTTP/1.0	 
        

The unresponsive server issue occurs seemingly randomly, last time in the 
middle of the night with little or no user traffic. 

James