You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Robert Elliot <ro...@lidalia.org.uk> on 2011/07/27 18:23:55 UTC

Wrong Response returned for a Request

Hi

We are using Tomcat 6.0.29, and occasionally experiencing concurrency issues which look very much like the wrong response is being returned for a request.  While we cannot reproduce it reliably, it seems to happen when Keep-Alive is turned on.

I noticed this issue with Tomcat 7:
https://issues.apache.org/bugzilla/show_bug.cgi?id=50957

Is there any chance that issue also exists in 6.0.29?

Thanks,
Rob

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Wrong Response returned for a Request

Posted by Robert Elliot <ro...@lidalia.org.uk>.
On 28 Jul 2011, at 10:16, Mark Thomas wrote:

> On 28/07/2011 07:34, Robert Elliot wrote:
>> 
>> 
>> 
>> 
>> On 28 Jul 2011, at 00:24, David Rees <dr...@gmail.com> wrote:
>>> 
>>> But perhaps this issue applies:
>>> https://issues.apache.org/bugzilla/show_bug.cgi?id=50189
>>> 
>>> -Dave
>>> 
>> Thanks very much for the pointer - however, we're using mod_proxy rather than mod_jk so presumably this wouldn't apply?
> 
> mod_proxy_ajp or mod_proxy_http?
> 
> If mod_proxy_ajp, then it is possible that this issue might occur (I
> haven't checked the source code or tested it to be sure).
> 
> Mark

Mod_proxy_http.  We enabled the access valve with a log pattern that included the number of bytes in the response in order to be able to match requests to responses at the Tomcat level and saw that the correct responses were being returned there but incorrect ones then returned by HTTPD/mod_proxy.  As we were using an old version of HTTPD we've upgraded it (from 2.2.3 to 2.2.19), and thus far we've been unable to reproduce the problem again.  Thanks for the help.



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Wrong Response returned for a Request

Posted by Mark Thomas <ma...@apache.org>.
On 28/07/2011 07:34, Robert Elliot wrote:
> 
> 
> 
> 
> On 28 Jul 2011, at 00:24, David Rees <dr...@gmail.com> wrote:
> 
>> On Wed, Jul 27, 2011 at 10:39 AM, Mark Thomas <ma...@apache.org> wrote:
>>> On 27/07/2011 17:23, Robert Elliot wrote:
>>>> We are using Tomcat 6.0.29, and occasionally experiencing concurrency issues
>>>> which look very much like the wrong response is being returned for a request.
>>>> While we cannot reproduce it reliably, it seems to happen when Keep-Alive is turned on.
>>>>
>>>> I noticed this issue with Tomcat 7:
>>>> https://issues.apache.org/bugzilla/show_bug.cgi?id=50957
>>>>
>>>> Is there any chance that issue also exists in 6.0.29?
>>>
>>> None.
> 
> Thanks for the confirmation - it seemed unlikely.
> 
>>
>> But perhaps this issue applies:
>> https://issues.apache.org/bugzilla/show_bug.cgi?id=50189
>>
>> -Dave
>>
> Thanks very much for the pointer - however, we're using mod_proxy rather than mod_jk so presumably this wouldn't apply?

mod_proxy_ajp or mod_proxy_http?

If mod_proxy_ajp, then it is possible that this issue might occur (I
haven't checked the source code or tested it to be sure).

Mark



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Wrong Response returned for a Request

Posted by Pid <pi...@pidster.com>.
On 28/07/2011 09:26, Robert Elliot wrote:
> ----- Original Message -----
>> From: "Pid" <pi...@pidster.com>
>> To: "Tomcat Users List" <us...@tomcat.apache.org>
>> Sent: Thursday, 28 July, 2011 9:07:13 AM
>> Subject: Re: Wrong Response returned for a Request
>>
>>>>> On 27/07/2011 17:23, Robert Elliot wrote:
>>>>>> We are using Tomcat 6.0.29, and occasionally experiencing
>>>>>> concurrency issues
>>>>>> which look very much like the wrong response is being returned
>>>>>> for a request.
>>>>>> While we cannot reproduce it reliably, it seems to happen when
>>>>>> Keep-Alive is turned on.
>>
>> In most cases this symptom turns out to be a result of the session,
>> request or response object being retained (perhaps as an instance
>> field
>> in a Servlet) where they should not be.
>>
>> Can you double check your code to see if this is possible?
>>
> 
> Unfortunately it's a third party application we're running.  

Ah.

> However as a gut feeling I'd expect a straightforward bug like that to be easier to reproduce; we've spent a day chucking between 100 and 1200 concurrent clients at it and we only see the problem very occasionally.  

Try reducing the number of connector threads, try 10 and work down from
there.

> Oddly once it has happened it keeps happening for a period - and then
stops.

Maybe not.  Speculation: once the JVM is hot, JIT optimisation might
reduce the window size further.

> I think we need to think harder about how to diagnose it - at the moment we can't pin it to a specific layer, it could be the application (most likely), Tomcat, mod_proxy or Apache HTTPD (we're running 2.2.3). I only asked here as the bug sounded so like what we were seeing.

Fair enough.


p


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Wrong Response returned for a Request

Posted by Robert Elliot <ro...@lidalia.org.uk>.
----- Original Message -----
> From: "Pid" <pi...@pidster.com>
> To: "Tomcat Users List" <us...@tomcat.apache.org>
> Sent: Thursday, 28 July, 2011 9:07:13 AM
> Subject: Re: Wrong Response returned for a Request
>
> >>> On 27/07/2011 17:23, Robert Elliot wrote:
> >>>> We are using Tomcat 6.0.29, and occasionally experiencing
> >>>> concurrency issues
> >>>> which look very much like the wrong response is being returned
> >>>> for a request.
> >>>> While we cannot reproduce it reliably, it seems to happen when
> >>>> Keep-Alive is turned on.
>
> In most cases this symptom turns out to be a result of the session,
> request or response object being retained (perhaps as an instance
> field
> in a Servlet) where they should not be.
> 
> Can you double check your code to see if this is possible?
> 

Unfortunately it's a third party application we're running.  However as a gut feeling I'd expect a straightforward bug like that to be easier to reproduce; we've spent a day chucking between 100 and 1200 concurrent clients at it and we only see the problem very occasionally.  Oddly once it has happened it keeps happening for a period - and then stops.

I think we need to think harder about how to diagnose it - at the moment we can't pin it to a specific layer, it could be the application (most likely), Tomcat, mod_proxy or Apache HTTPD (we're running 2.2.3). I only asked here as the bug sounded so like what we were seeing.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Wrong Response returned for a Request

Posted by Pid <pi...@pidster.com>.
On 28/07/2011 07:34, Robert Elliot wrote:
> 
> 
> 
> 
> On 28 Jul 2011, at 00:24, David Rees <dr...@gmail.com> wrote:
> 
>> On Wed, Jul 27, 2011 at 10:39 AM, Mark Thomas <ma...@apache.org> wrote:
>>> On 27/07/2011 17:23, Robert Elliot wrote:
>>>> We are using Tomcat 6.0.29, and occasionally experiencing concurrency issues
>>>> which look very much like the wrong response is being returned for a request.
>>>> While we cannot reproduce it reliably, it seems to happen when Keep-Alive is turned on.
>>>>
>>>> I noticed this issue with Tomcat 7:
>>>> https://issues.apache.org/bugzilla/show_bug.cgi?id=50957
>>>>
>>>> Is there any chance that issue also exists in 6.0.29?
>>>
>>> None.
> 
> Thanks for the confirmation - it seemed unlikely.
> 
>>
>> But perhaps this issue applies:
>> https://issues.apache.org/bugzilla/show_bug.cgi?id=50189
>>
>> -Dave
>>
> Thanks very much for the pointer - however, we're using mod_proxy rather than mod_jk so presumably this wouldn't apply?

In most cases this symptom turns out to be a result of the session,
request or response object being retained (perhaps as an instance field
in a Servlet) where they should not be.

Can you double check your code to see if this is possible?


p

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Wrong Response returned for a Request

Posted by Robert Elliot <ro...@lidalia.org.uk>.



On 28 Jul 2011, at 00:24, David Rees <dr...@gmail.com> wrote:

> On Wed, Jul 27, 2011 at 10:39 AM, Mark Thomas <ma...@apache.org> wrote:
>> On 27/07/2011 17:23, Robert Elliot wrote:
>>> We are using Tomcat 6.0.29, and occasionally experiencing concurrency issues
>>> which look very much like the wrong response is being returned for a request.
>>> While we cannot reproduce it reliably, it seems to happen when Keep-Alive is turned on.
>>> 
>>> I noticed this issue with Tomcat 7:
>>> https://issues.apache.org/bugzilla/show_bug.cgi?id=50957
>>> 
>>> Is there any chance that issue also exists in 6.0.29?
>> 
>> None.

Thanks for the confirmation - it seemed unlikely.

> 
> But perhaps this issue applies:
> https://issues.apache.org/bugzilla/show_bug.cgi?id=50189
> 
> -Dave
> 
Thanks very much for the pointer - however, we're using mod_proxy rather than mod_jk so presumably this wouldn't apply?

Rob

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Wrong Response returned for a Request

Posted by David Rees <dr...@gmail.com>.
On Wed, Jul 27, 2011 at 10:39 AM, Mark Thomas <ma...@apache.org> wrote:
> On 27/07/2011 17:23, Robert Elliot wrote:
>> We are using Tomcat 6.0.29, and occasionally experiencing concurrency issues
>> which look very much like the wrong response is being returned for a request.
>> While we cannot reproduce it reliably, it seems to happen when Keep-Alive is turned on.
>>
>> I noticed this issue with Tomcat 7:
>> https://issues.apache.org/bugzilla/show_bug.cgi?id=50957
>>
>> Is there any chance that issue also exists in 6.0.29?
>
> None.

But perhaps this issue applies:
https://issues.apache.org/bugzilla/show_bug.cgi?id=50189

-Dave

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Wrong Response returned for a Request

Posted by Mark Thomas <ma...@apache.org>.
On 27/07/2011 17:23, Robert Elliot wrote:
> Hi
> 
> We are using Tomcat 6.0.29, and occasionally experiencing concurrency issues which look very much like the wrong response is being returned for a request.  While we cannot reproduce it reliably, it seems to happen when Keep-Alive is turned on.
> 
> I noticed this issue with Tomcat 7:
> https://issues.apache.org/bugzilla/show_bug.cgi?id=50957
> 
> Is there any chance that issue also exists in 6.0.29?

None.

Mark



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org