You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Rainer Jung <ra...@kippdata.de> on 2011/06/13 17:07:04 UTC
Crash in APR when pausing endpoint
I had some spare time for analyzing the crashes in
TestMapperWelcomeFiles. It seems there are several varieties, but at
least I do now understand enough to report here:
In Http11AprProcessor.process() when parsing the request line fails,
e.g. it is not yet there, we add the socket back to the poller. Then we
check the endpoint for being paused. If so, we set "error" to true.
At the end of process(), if "error" is true, we return
SocketState.CLOSED, which in turn lets the SocketProcessor closes the
socket (and destroys the native memory allocated with it).
Now what I experience sporadically, is that the original thread A is
thrown off the CPU after adding the socket back to the poller, but
before proceeding.
Another thread B now gets the socket from the poller, uses it for
request processing and at the end destroys it. Now A wakes up and
proceeds as described above, and since now the endpoint is paused, it
also tries to destroy the socket, leading to a crash.
This seems to be a flaw. I suggest we add only to the poller if we know
no error occured and the endpoint is not paused. The patch for AjpApr is
simpler but similar. See for both classes:
http://people.apache.org/~rjung/patches/tc7-apr-add_to_poller.patch
I checked this patch against one full run of the test suite with HTTP
APR and also against 150 runs of TestMapperWelcomeFiles. No crashes or
errors any more.
WDYT?
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Crash in APR when pausing endpoint
Posted by Rainer Jung <ra...@kippdata.de>.
On 13.06.2011 17:49, Mark Thomas wrote:
> On 13/06/2011 16:07, Rainer Jung wrote:
>> I had some spare time for analyzing the crashes in
>> TestMapperWelcomeFiles. It seems there are several varieties, but at
>> least I do now understand enough to report here:
>>
...
>> This seems to be a flaw. I suggest we add only to the poller if we know
>> no error occured and the endpoint is not paused. The patch for AjpApr is
>> simpler but similar. See for both classes:
>>
>> http://people.apache.org/~rjung/patches/tc7-apr-add_to_poller.patch
>>
>> I checked this patch against one full run of the test suite with HTTP
>> APR and also against 150 runs of TestMapperWelcomeFiles. No crashes or
>> errors any more.
>>
>> WDYT?
>
> Analysis and patch make sense to me.
Thanks for reviewing. After about 900 test iterations I committed in
r1135208.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Crash in APR when pausing endpoint
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Rainer,
On 6/13/2011 11:49 AM, Mark Thomas wrote:
> Analysis and patch make sense to me.
+1
Nice analysis. Nice to see I'm not the only one who can code-up a nice
race condition.
-chris
Re: Crash in APR when pausing endpoint
Posted by Mark Thomas <ma...@apache.org>.
On 13/06/2011 16:07, Rainer Jung wrote:
> I had some spare time for analyzing the crashes in
> TestMapperWelcomeFiles. It seems there are several varieties, but at
> least I do now understand enough to report here:
>
> In Http11AprProcessor.process() when parsing the request line fails,
> e.g. it is not yet there, we add the socket back to the poller. Then we
> check the endpoint for being paused. If so, we set "error" to true.
>
> At the end of process(), if "error" is true, we return
> SocketState.CLOSED, which in turn lets the SocketProcessor closes the
> socket (and destroys the native memory allocated with it).
>
> Now what I experience sporadically, is that the original thread A is
> thrown off the CPU after adding the socket back to the poller, but
> before proceeding.
>
> Another thread B now gets the socket from the poller, uses it for
> request processing and at the end destroys it. Now A wakes up and
> proceeds as described above, and since now the endpoint is paused, it
> also tries to destroy the socket, leading to a crash.
>
> This seems to be a flaw. I suggest we add only to the poller if we know
> no error occured and the endpoint is not paused. The patch for AjpApr is
> simpler but similar. See for both classes:
>
> http://people.apache.org/~rjung/patches/tc7-apr-add_to_poller.patch
>
> I checked this patch against one full run of the test suite with HTTP
> APR and also against 150 runs of TestMapperWelcomeFiles. No crashes or
> errors any more.
>
> WDYT?
Analysis and patch make sense to me.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org