[users@httpd] Problem with gcache socket

[Dave R <da...@gmail.com>]

Hello,

I'm running Apache/1.3.33 Ben-SSL/1.55 , as included with Debian Sarge, and
I'm having a strange problem that happens infrequently (roughly once per
month). The server chugs merrily along serving requests, but will suddenly
cease to serve pages (but keeps accepting the TCP connections), on both port
80 and port 443. Here is a clip from the error log at the time this occurs:

----- BEGIN PASTE
[Sun Feb  5 21:17:49 2006] [info] server seems busy, (you may need to
increase StartServers, or Min/MaxSpareServers), spawning 8 children, there
are (chopped)
[Sun Feb  5 21:17:50 2006] [info] server seems busy, (you may need to
increase StartServers, or Min/MaxSpareServers), spawning 16 children, there
are (chopped)
[Sun Feb  5 21:19:14 2006] [error] PHP Notice:  Undefined index:
HTTP_USER_AGENT... (unrelated php notice)
Failed to connect to socket: /var/run/gcache_port
connect: Connection refused
apache-ssl: gcacheclient.c:118: OpenServer: Assertion `!"couldn't connect to
socket"' failed.
Failed to connect to socket: /var/run/gcache_port
connect: Connection refused
apache-ssl: gcacheclient.c:118: OpenServer: Assertion `!"couldn't connect to
socket"' failed.
----- END PASTE

Notice how the "server seems busy" lines seem to portend the coming doom,
but the actual error does not appear until a few minutes after.

A simple 'apache-sslctl' restart does not bring the server back up, I have
to 'killall apache-ssl' and 'apache-sslctl start'. Running the
'apache-sslctl' restart is, I think, responsible for this error log entry:

----- BEGIN PASTE
[Mon Feb  6 08:39:20 2006] [crit] (98)Address already in use: make_sock:
could not bind to port 443
----- END PASTE

The timestamps are correct, the server was down overnight, and the
'apache-ssl restart' was run that following morning.

I should note that this does not seem to be related to any other activity on
the server, usually no one is logged in at the time.

There is no particularly high load accompanying these problems, and the
access logs show no irregular traffic at the times of the problem. The
server has 1 GB of RAM, 2 GB of swap, and a P4 3.2 GHz CPU.

Here are some possibly relevant lines from httpd.conf:
MinSpareServers 10
MaxSpareServers 70
StartServers 20
MaxClients 300
MaxRequestsPerChild 100

Info from Google that may be related:
- this is *not* the "my gcache is not working help me set it up" problem,
which is common. This server works great 99% of the time.
- we can rule out permission problems which give trouble with gcache, for
the same reason, that it works almost all of the time.
- possibly related to this
issue,<http://lists.debian.org/debian-apache/2005/04/msg00113.html>where
log rotation caused problems forcing a restart. However, since my
problem occurs irregularly on different day of week, day of month, time of
day, etc, I think it's safe to rule out log rotation issues. Also in my
case, a simple apache restart does not fix the problem.
- the erratic times/days of the error also seem to preclude the
/var/run/gcache_port socket being screwed up by a cron job.

Any advice is appreciated. If there's any further information that may help
to solve this problem, I'd be happy to supply it.

Dave