Reliable piped logs and select() Linux bug

[Paul Marquis <pm...@iname.com>]

If this is not the right list, please let me know.

I have uncovered what I believe to be a bug with the select() call on
Linux when checking for writability of a pipe.  This bug, when combined
with Reliable Piped Logs, can cause Apache to unnecessarily respawn
piped log child processes.

The select() call on Linux will report "false negatives" determining if
a pipe is writable when any unread data is in the pipe and only when the
pipe is completely empty will select() do the right thing.  So, for
example, after succesfully calling pipe(), if you select() on the write
end of the pipe, it will succeed.  If you write a single byte and then
don't read it from the other end and call select() again, it will fail,
but a call to write() will succeed.  If you flush the pipe by reading
all the data from the other end, select() will succeed again.  I don't
know if this problem extends beyond pipes.  The attached code, which
works as expected on FreeBSD and Solaris, demonstrates the bug on Linux.

In Apache, the Reliable Piped Logs feature determines that a child
process has died by checking if the pipe is writable using select().  If
it's not writable, then Apache kills the process and restarts it (I'm
basing this on my quick inspection of the Apache documentation and
source code).  Since, on Linux, select() has the above bug for pipes,
Apache will unnecessarily respawn child processes.

Until either I'm corrected that this is not a bug, select() on Linux is
fixed or there's an Apache workaround (which off the cuff, I don't think
is possible), I'm disabling the Reliable Piped Log feature for our web
servers.  If this is truly a bug, then, to be honest, I don't know how
to filter this through the appropriate Linux development channels,
though I'm sure some of my co-workers will.

Comments?

--
Paul Marquis
pmarquis@iname.com

Re: Reliable piped logs and select() Linux bug

[dean gaudet <dg...@arctic.org>]

yeah i forwarded some stuff about this as well.  my suggestion was to
eliinate the select() test entirely.  it's not even just linux it's broken
on, it was a mistake for me to do it in the first place.  (it's just that
linux kernel developers, for better or worse, prefer to make such race
conditions explicit even if they're only theoretically possible.  there's
another example of such a thing during TCP closes which they deliberately
made explicitly fail.)

the parent should only respawn the piped child if the piped child exits.

-dean

On Tue, 31 Oct 2000, Paul Marquis wrote:

> If this is not the right list, please let me know.
>
> I have uncovered what I believe to be a bug with the select() call on
> Linux when checking for writability of a pipe.  This bug, when combined
> with Reliable Piped Logs, can cause Apache to unnecessarily respawn
> piped log child processes.
>
> The select() call on Linux will report "false negatives" determining if
> a pipe is writable when any unread data is in the pipe and only when the
> pipe is completely empty will select() do the right thing.  So, for
> example, after succesfully calling pipe(), if you select() on the write
> end of the pipe, it will succeed.  If you write a single byte and then
> don't read it from the other end and call select() again, it will fail,
> but a call to write() will succeed.  If you flush the pipe by reading
> all the data from the other end, select() will succeed again.  I don't
> know if this problem extends beyond pipes.  The attached code, which
> works as expected on FreeBSD and Solaris, demonstrates the bug on Linux.
>
> In Apache, the Reliable Piped Logs feature determines that a child
> process has died by checking if the pipe is writable using select().  If
> it's not writable, then Apache kills the process and restarts it (I'm
> basing this on my quick inspection of the Apache documentation and
> source code).  Since, on Linux, select() has the above bug for pipes,
> Apache will unnecessarily respawn child processes.
>
> Until either I'm corrected that this is not a bug, select() on Linux is
> fixed or there's an Apache workaround (which off the cuff, I don't think
> is possible), I'm disabling the Reliable Piped Log feature for our web
> servers.  If this is truly a bug, then, to be honest, I don't know how
> to filter this through the appropriate Linux development channels,
> though I'm sure some of my co-workers will.
>
> Comments?
>
> --
> Paul Marquis
> pmarquis@iname.com