[1.3 PATCH] don't hold accept mutex while running 3rd-party code at termination

[Jeff Trawick <tr...@attglobal.net>]

two scenarios make it a bad idea to be able to hold the accept mutex 
when exiting a child process for idle child cleanup:

1) a child exit hook or a pchild cleanup takes a while

This results in a brief server outage since other children can't accept 
new connections.

(If a child exit hook or a pchild cleanup takes forever/deadlocks, we're 
toast anyway, since the scoreboard slot is occupied and we won't be able 
to keep creating new child processes.)

2) a child exit hook or a pchild cleanup segfaults and accept mutex 
mechanism isn't automatically cleaned up by the OS

(mechanism = pthread, 3rd party module = XXX)

deadlock :(

If anybody wants a toy 1.3 module to sleep or segfault in child exit 
hook, just let me know.  It is much easier to install than the 
commercial version.

Re: [1.3 PATCH] don't hold accept mutex while running 3rd-party code at termination

[Jeff Trawick <tr...@attglobal.net>]

Bill Stoddard wrote:
> Jeff Trawick wrote:
> 
>> two scenarios make it a bad idea to be able to hold the accept mutex 
>> when exiting a child process for idle child cleanup:
>>
>> 1) a child exit hook or a pchild cleanup takes a while
>>
>> This results in a brief server outage since other children can't 
>> accept new connections.
>>
>> (If a child exit hook or a pchild cleanup takes forever/deadlocks, 
>> we're toast anyway, since the scoreboard slot is occupied and we won't 
>> be able to keep creating new child processes.)
>>
>> 2) a child exit hook or a pchild cleanup segfaults and accept mutex 
>> mechanism isn't automatically cleaned up by the OS
>>
>> (mechanism = pthread, 3rd party module = XXX)
>>
>> deadlock :(
>>
>> If anybody wants a toy 1.3 module to sleep or segfault in child exit 
>> hook, just let me know.  It is much easier to install than the 
>> commercial version.
> 
> 
> 
> In 1.3 (on Unix), I would expect that the thread that does the accept 
> would be the same thread that runs the child exit hook and pchild 
> cleanups. When would that not be the case?  So how can case 2) ever occur?

Yes, we're always talking about single-threaded processes here (we're 
agnostic about other threads).

The base level of understanding required is this:

When a child process exits due to graceful restart or due to the parent 
cleaning up an idle child, the child process does not explicitly release 
the accept mutex prior to calling a child exit hook or a cleanup 
registered against pchild.  So maybe the accept mutex is held, maybe it 
is not.  (For pthread mutex, it is guaranteed to be held for this type 
of exit, since SIGUSR1 will not result in a child process going away if 
the child process is waiting on the accept mutex.)

Given that this is how the server works, condition 1 or condition 2 can 
occur if a child exit hook or pchild cleanup can take a while or segfault.

The purpose of the patch is to ensure that the accept mutex is never 
held when calling a child exit hook or pchild cleanup, thus making the 
server more reliable in the presence of 3rd-party code which may not be 
as robust as desired.

Re: [1.3 PATCH] don't hold accept mutex while running 3rd-party code at termination

[Bill Stoddard <bi...@wstoddard.com>]

Jeff Trawick wrote:

> two scenarios make it a bad idea to be able to hold the accept mutex 
> when exiting a child process for idle child cleanup:
>
> 1) a child exit hook or a pchild cleanup takes a while
>
> This results in a brief server outage since other children can't 
> accept new connections.
>
> (If a child exit hook or a pchild cleanup takes forever/deadlocks, 
> we're toast anyway, since the scoreboard slot is occupied and we won't 
> be able to keep creating new child processes.)
>
> 2) a child exit hook or a pchild cleanup segfaults and accept mutex 
> mechanism isn't automatically cleaned up by the OS
>
> (mechanism = pthread, 3rd party module = XXX)
>
> deadlock :(
>
> If anybody wants a toy 1.3 module to sleep or segfault in child exit 
> hook, just let me know.  It is much easier to install than the 
> commercial version.


In 1.3 (on Unix), I would expect that the thread that does the accept 
would be the same thread that runs the child exit hook and pchild 
cleanups. When would that not be the case?  So how can case 2) ever occur?

Bill

Re: [1.3 PATCH] don't hold accept mutex while running 3rd-party code at termination

[Jeff Trawick <tr...@attglobal.net>]

Jeff Trawick wrote:
> two scenarios make it a bad idea to be able to hold the accept mutex 
> when exiting a child process for idle child cleanup:

Any comments on releasing the accept mutex in the child before running 
3rd-party child exit hooks or cleanups?  I see that 1.3.28 headed for 
the runway.