Re: More on FreeBSD timeout problem

[Randy Terbush <ra...@zyzzyva.com>]

> [still on vacation... checking in ... lots of mail still to go]
> 
> I have an idea for an adaptive caching algorithm that would involve
> changing this sleep to a select anyhow.  Essentially I've been trying to
> figure out a way to get an adaptive cache of mmap()d files.  The best I
> can think of so far involves the parent doing cache maintenance and some
> of the children sending samples up to the parent so it can decide how to
> adapt the cache.  I need to experiment a bunch first though.

Sounds cool.

> Oh yeah and I've got another idea that'll help servers with lots of
> children -- right now it scans all children once per hit in child_main. 
> It really should only scan once per second in standalone_main and use
> SIGUSR1 to kill off one child if it determines itself to be over the
> MaxSpare setting. 

Another good idea. Vacations are good! :-)  Have fun.

> Dean

Re: More on FreeBSD timeout problem

[Dean Gaudet <dg...@arctic.org>]

And one more! 

Observation:  apache uses a lot of file descriptors in many users'
configurations.  select() is linear in the value of the highest
descriptor.

Using poll() instead of select() on those architectures that support it is
a potential big sys-time win.  Apache's fd_sets are all sparse -- even
when there's lots of Listens there's also typically two log files per
vhost giving a density of 1/3rd.

Of course it all depends on the order we open our files... and we also
need to consider the select() of a single socket inside bread().  I also
think some resolver libraries use select() but haven't checked.  There's
gotta be third party libraries that use select() behind our backs too. 

So the goal would be to use a trick like my solaris > 256 fd patch to keep
log files above 256... let sockets go in below 256, and keep enough room <
32 for libraries and the per-request file descriptors.  (All those numbers
can be calculated after the config has been read of course.)  Then even if
third part libraries do select() there's a good chance they'll be doing it
on descriptors < 32 (which of course fit inside a single 32-bit int, and
means only 1 memory access for the entire fd_set ...) 

Top it off by using poll() on archs which support it and I think we've got
some sys-time savings on the configs that have lots o' vhosts.

experimentation necessary :) 

Ok laters, I'll check in next week some time.

Dean

On Fri, 23 May 1997, Randy Terbush wrote:

> > [still on vacation... checking in ... lots of mail still to go]
> > 
> > I have an idea for an adaptive caching algorithm that would involve
> > changing this sleep to a select anyhow.  Essentially I've been trying to
> > figure out a way to get an adaptive cache of mmap()d files.  The best I
> > can think of so far involves the parent doing cache maintenance and some
> > of the children sending samples up to the parent so it can decide how to
> > adapt the cache.  I need to experiment a bunch first though.
> 
> Sounds cool.
> 
> > Oh yeah and I've got another idea that'll help servers with lots of
> > children -- right now it scans all children once per hit in child_main. 
> > It really should only scan once per second in standalone_main and use
> > SIGUSR1 to kill off one child if it determines itself to be over the
> > MaxSpare setting. 
> 
> Another good idea. Vacations are good! :-)  Have fun.
> 
> > Dean
> 
> 
> 
> 
>