PR#467

[Dean Gaudet <dg...@arctic.org>]

I've done a bunch more thought exercises on a solution to this one... I'm
convinced that some variant on serialised accepts is the way to go. 

The only alternatives I've thought of so far are serialisation,
non-blocking listening sockets, and partial serialisation (let N tasks
in, where N is much less than the potential number of idle children,
requires non-blocking listening sockets).

Regardless of which of these we use, there is still a starvation situation
when multiple sockets are in use:  if one of the lower numbered sockets
is really busy it will easily starve the higher numbered sockets.
To solve that I intend to keep a global in each child that tracks which
socket it last served.  Then next time through the loop it will check
the socket after that one first.  The parent also needs to seed that
global differently for each spawned child.

There was a recent bug report where the user complained of periodic
very short-lived spikes in his load under SunOS 4.1.4, which doesn't
use serialised accepts.  Since BSD select() code wakes up all waiting
processes when an event happens, he's seeing every single idle child
wake up, and then most of them go right back to sleep.  At least that's
what I'm guessing, he hasn't responded if serialisation helps yet.
I'm guessing this is how most unixes work, I can't imagine how the
select() semantics could be implemented differently.

Similarly, most unixes implement accept() in a manner that cause all
blocked tasks to wake up, at least long enough to spin and discover
they have to go back to sleep 'cause another task got the socket.
Marc has hinted that FreeBSD has worked or will work on changing that.
The Linux folk are considering a similar change.

Consider the non-blocking socket scenario.  In this scenario the code
includes a select() followed by a test accept() on each socket marked
ready for read.  The only part of this which blocks is the select().
Every idle child will awake from select() and try every socket ... this
can get quite expensive if you've got a lot of idle children.  (A server
serving 100hit/second loads should have a buffer of probably 50 or 60
idle children for load spikes... pulling numbers out of my butt.)

On multiprocessor servers I can imagine serialisation loses a bit of the
concurrency possible.  In single-socket cases, if we just blocked everything
in accept() then we're at the operating system's mercy to parallelize
things well, and I bet Solaris 2.6 does quite well at this (others may
too).  In multiple socket cases, if you consider the anti-starving tweak
above, it's still possible for parallelization because multiple processes
will be checking different sockets.  So by serialising we're losing both
of these opportunities.

It would seem that partial serialisation can get at this parallelisation...

Anyhow, this is just thought experiment.  Anyone else have any ideas before
I try to put data behind any of these thoughts?

To support older Unixes we'll need to continue with something essentially
like the current serialisation code.  I'll probably just leave it as-is.
The newer code will have separate single-socket and multiple-socket cases,
since single socket can be far better optimized.  I'll use sysv semaphores
to do the partial serialisation.

Dean