MaxClients ignored ?

[Rob Hartill <ro...@imdb.com>]

Was any bug introduced recently that would cause apache to
ignore MaxClients and plow on spawning more and more ? this was
with a Nov 1st 1.3b-dev.

I just lost 4 servers due to exhausted swap space. I was half-watching
'top' on one when it spawned itslef to death :-(

.. had to drag out remote support to reboot the suckers (I'm a few
thousand miles too far awy to reach).

I've used this morning's cvs to build a new httpd and have crossed
my fingers that it won't happen again.

Strange thing is they were all working fine for the last 8 days then
went bang within a few short hours of each other.


--
Rob Hartill                              Internet Movie Database (Ltd)
http://www.moviedatabase.com/   .. a site for sore eyes.

Re: MaxClients ignored ?

[Dean Gaudet <dg...@arctic.org>]

On Mon, 10 Nov 1997, Rob Hartill wrote:

> It could be that the spawning of httpds that I saw was caused by them
> dumping core and the parent replacing them before they had disapeared
> completely.

Well the parent counts anything still marked as STARTING as "idle" for the
purposes of deciding to spawn another.  So it shouldn't have been
overlapping startups, it should have waited until it had the pid back from
wait().

But computers do suck ;)

Dean

Re: MaxClients ignored ?

[Rob Hartill <ro...@imdb.com>]

On Mon, 10 Nov 1997, Dean Gaudet wrote:

> Bleh.  Lemme think about it,

thanks, but I now suspect the problem was caused by the OS/httpd-process
running out of resources to create new files.

sysctl -a |grep files
kern.maxfiles: 8232

I think I was in that range. doh!.

Our servers build huge caches of pregenerated html over the week and
they were happy until the 7th and last day, then bang... I suppose
on the bright side it showed the round-robin worked as advertised :-/

It could be that the spawning of httpds that I saw was caused by them
dumping core and the parent replacing them before they had disapeared
completely.

I hate computers.
--
Rob Hartill                              Internet Movie Database (Ltd)
http://www.moviedatabase.com/   .. a site for sore eyes.

Re: MaxClients ignored ?

[Dean Gaudet <dg...@arctic.org>]

On Mon, 10 Nov 1997, Rob Hartill wrote:

> On Sun, 9 Nov 1997, Dean Gaudet wrote:
> 
> > Did you get any of the "server seems busy" error messages? 
> 
> I wasn't looking for it. I have to set MaxClients very low (20)
> so this has been common in the past.

Bleh.

> > Did you get the "server reached MaxClients setting" error message? 
> 
> ditto

Well this is a rated limited message ... it only occurs once; even if you
use a USR1 or HUP you shouldn't get it more than once per lifetime of the
parent. 

> 5 out of 6 servers lasted the night (well, the 4 hours of sleep I
> got). One didn't and is now unreachable by me :-(

Bleh.  Lemme think about it, maybe there's something really wrong with how
my code works when it is consistantly at MaxClients.  Maybe you could put
in some debugging code?  Something like this after the main loop in
perform_idle_server_maintenance():

{
    static counter = 0;

    if (counter == 0) {
	aplog_error(APLOG_MARK, APLOG_NOERRNO|APLOG_DEBUG, NULL,
	    "total_non_dead = %u, max_daemons_limit = %u, idle_count = %u, free_length = %u",
	    total_non_dead, max_daemons_limit, idle_count, free_length);
	counter = 60;
    }
    --counter;
}

Dean

Re: MaxClients ignored ?

[Rob Hartill <ro...@imdb.com>]

On Sun, 9 Nov 1997, Dean Gaudet wrote:

> Not that I'm aware of ...  The code in perform_idle_server_maintenance
> does make a few assumptions about various pieces of state ... which can
> make it a little non-obvious if you're trying to understand it.  I think
> it's all documented.
> 
> Did you get any of the "server seems busy" error messages? 

I wasn't looking for it. I have to set MaxClients very low (20)
so this has been common in the past.

> Did you get the "server reached MaxClients setting" error message? 

ditto

> These are FreeBSD servers with mod_perl, right?

correct

> Do you use any of the tuning #defines? 

no

> Do you use USR1 or HUP? 

neither. I always kill -15 before restarting


5 out of 6 servers lasted the night (well, the 4 hours of sleep I
got). One didn't and is now unreachable by me :-(


--
Rob Hartill                              Internet Movie Database (Ltd)
http://www.moviedatabase.com/   .. a site for sore eyes.

Re: MaxClients ignored ?

[Dean Gaudet <dg...@arctic.org>]

Not that I'm aware of ...  The code in perform_idle_server_maintenance
does make a few assumptions about various pieces of state ... which can
make it a little non-obvious if you're trying to understand it.  I think
it's all documented.

Did you get any of the "server seems busy" error messages? 

Did you get the "server reached MaxClients setting" error message? 

These are FreeBSD servers with mod_perl, right?  Do you use any of the
tuning #defines? 

Do you use USR1 or HUP? 

Dean

On Mon, 10 Nov 1997, Rob Hartill wrote:

> 
> Was any bug introduced recently that would cause apache to
> ignore MaxClients and plow on spawning more and more ? this was
> with a Nov 1st 1.3b-dev.
> 
> I just lost 4 servers due to exhausted swap space. I was half-watching
> 'top' on one when it spawned itslef to death :-(
> 
> .. had to drag out remote support to reboot the suckers (I'm a few
> thousand miles too far awy to reach).
> 
> I've used this morning's cvs to build a new httpd and have crossed
> my fingers that it won't happen again.
> 
> Strange thing is they were all working fine for the last 8 days then
> went bang within a few short hours of each other.
> 
> 
> --
> Rob Hartill                              Internet Movie Database (Ltd)
> http://www.moviedatabase.com/   .. a site for sore eyes.
> 
>