You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Erik Nilsson <er...@sitevision.se> on 2021/07/01 06:16:04 UTC
Re: Possible bug in http2 window size handling in tomcat 9.0.45
Hmm I can still get the same exceptions even if I set useAsyncIO="false",
but with maxConcurrentStreamExecution="1" it's stable.
/Erik
Den ons 30 juni 2021 kl 18:41 skrev Mark Thomas <ma...@apache.org>:
> On 30/06/2021 16:49, Erik Nilsson wrote:
> > Perfect that u can reproduce this with another webapp. Thankful for your
> > quick response. As I pointed out in the beginning of the conversation we
> > also got this problem behind our f5 loadbalancer. But not with if we
> > connect to Tomcat directly without a loadbalancer.
>
> Yes, this is a Tomcat bug. You should be able to work-around it with:
>
> useAsyncIO="false" on the Connector.
>
> I know where the problem is. I "just" need to figure out the fix.
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
--
Re: Possible bug in http2 window size handling in tomcat 9.0.45
Posted by Erik Nilsson <er...@sitevision.se>.
Perfect, glad to see a release after the vacation :)
Thanks again and happy vacationing.
/Erik
Den fre 2 juli 2021 kl 14:06 skrev Mark Thomas <ma...@apache.org>:
> On 02/07/2021 12:46, Erik Nilsson wrote:
> > Great!! With tomcat-9.0-20210701.191821-3270 in our environment it seems
> to
> > be stable without any RST_STREAMs.
>
> Excellent. Thanks for confirming the fix.
>
> > When will you release this version?
>
> Releases happen ~monthly. We aim for early in the month and typically
> have the release available some time in the second week.
>
> There is a set of releases in progress at the moment but they do not
> include these fixes. The next set of releases is scheduled for August.
> It isn't unusual for us to skip the August release as folks are often on
> holiday but given the severity of this issue, my current intention is to
> try and do an August release.
>
> Mark
>
>
> > /Erik
> >
> >
> > Den fre 2 juli 2021 kl 10:28 skrev Mark Thomas <ma...@apache.org>:
> >
> >> On 01/07/2021 08:57, Mark Thomas wrote:
> >>> On 01/07/2021 08:41, Erik Nilsson wrote:
> >>>> <Connector port="443"
> >>>> protocol="org.apache.coyote.http11.Http11NioProtocol"
> >>>> connectionTimeout="20000" compression="on"
> >> useAsyncIO="false"
> >>>>
> >>>>
> >>
> compressibleMimeType="text/html,text/xml,text/css,text/javascript,text/plain,application/xml,application/javascript,application/json,image/svg+xml"
> >>
> >>>>
> >>>> SSLEnabled="true" scheme="https" secure="true"
> >>>> URIEncoding="UTF-8" >
> >>>> <UpgradeProtocol
> >>>> className="org.apache.coyote.http2.Http2Protocol" />
> >>>> </Connector>
> >>>
> >>> Tx.
> >>>
> >>> It looks like there are multiple issues going on here then - in
> >>> additional to the sendfile issue I already fixed. I'll fix the issue
> I'm
> >>> currently seeing and then try again to recreate the issue you are
> >>> seeing. I'll keep the thread updated with how I get on.
> >>
> >> I've found and fixed a couple of issues with the management of the
> >> connection flow control window. The fixes are consistent with the
> >> workarounds you described (reduce the size of the stream flow control
> >> window, reduce concurrent streams).
> >>
> >> The fixes are included in the latest snapshot from:
> >>
> >>
> >>
> https://repository.apache.org/content/groups/snapshots/org/apache/tomcat/tomcat/9.0-SNAPSHOT/
> >>
> >> (tomcat-9.0-20210701.191821-3270 or later)
> >>
> >> If you are able to test the snapshot in your dev environment, it would
> >> be very helpful to know if the issue has been resolved.
> >>
> >> Mark
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> >> For additional commands, e-mail: users-help@tomcat.apache.org
> >>
> >>
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
--
Re: Possible bug in http2 window size handling in tomcat 9.0.45
Posted by Mark Thomas <ma...@apache.org>.
On 02/07/2021 12:46, Erik Nilsson wrote:
> Great!! With tomcat-9.0-20210701.191821-3270 in our environment it seems to
> be stable without any RST_STREAMs.
Excellent. Thanks for confirming the fix.
> When will you release this version?
Releases happen ~monthly. We aim for early in the month and typically
have the release available some time in the second week.
There is a set of releases in progress at the moment but they do not
include these fixes. The next set of releases is scheduled for August.
It isn't unusual for us to skip the August release as folks are often on
holiday but given the severity of this issue, my current intention is to
try and do an August release.
Mark
> /Erik
>
>
> Den fre 2 juli 2021 kl 10:28 skrev Mark Thomas <ma...@apache.org>:
>
>> On 01/07/2021 08:57, Mark Thomas wrote:
>>> On 01/07/2021 08:41, Erik Nilsson wrote:
>>>> <Connector port="443"
>>>> protocol="org.apache.coyote.http11.Http11NioProtocol"
>>>> connectionTimeout="20000" compression="on"
>> useAsyncIO="false"
>>>>
>>>>
>> compressibleMimeType="text/html,text/xml,text/css,text/javascript,text/plain,application/xml,application/javascript,application/json,image/svg+xml"
>>
>>>>
>>>> SSLEnabled="true" scheme="https" secure="true"
>>>> URIEncoding="UTF-8" >
>>>> <UpgradeProtocol
>>>> className="org.apache.coyote.http2.Http2Protocol" />
>>>> </Connector>
>>>
>>> Tx.
>>>
>>> It looks like there are multiple issues going on here then - in
>>> additional to the sendfile issue I already fixed. I'll fix the issue I'm
>>> currently seeing and then try again to recreate the issue you are
>>> seeing. I'll keep the thread updated with how I get on.
>>
>> I've found and fixed a couple of issues with the management of the
>> connection flow control window. The fixes are consistent with the
>> workarounds you described (reduce the size of the stream flow control
>> window, reduce concurrent streams).
>>
>> The fixes are included in the latest snapshot from:
>>
>>
>> https://repository.apache.org/content/groups/snapshots/org/apache/tomcat/tomcat/9.0-SNAPSHOT/
>>
>> (tomcat-9.0-20210701.191821-3270 or later)
>>
>> If you are able to test the snapshot in your dev environment, it would
>> be very helpful to know if the issue has been resolved.
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Possible bug in http2 window size handling in tomcat 9.0.45
Posted by Erik Nilsson <er...@sitevision.se>.
Great!! With tomcat-9.0-20210701.191821-3270 in our environment it seems to
be stable without any RST_STREAMs.
When will you release this version?
/Erik
Den fre 2 juli 2021 kl 10:28 skrev Mark Thomas <ma...@apache.org>:
> On 01/07/2021 08:57, Mark Thomas wrote:
> > On 01/07/2021 08:41, Erik Nilsson wrote:
> >> <Connector port="443"
> >> protocol="org.apache.coyote.http11.Http11NioProtocol"
> >> connectionTimeout="20000" compression="on"
> useAsyncIO="false"
> >>
> >>
> compressibleMimeType="text/html,text/xml,text/css,text/javascript,text/plain,application/xml,application/javascript,application/json,image/svg+xml"
>
> >>
> >> SSLEnabled="true" scheme="https" secure="true"
> >> URIEncoding="UTF-8" >
> >> <UpgradeProtocol
> >> className="org.apache.coyote.http2.Http2Protocol" />
> >> </Connector>
> >
> > Tx.
> >
> > It looks like there are multiple issues going on here then - in
> > additional to the sendfile issue I already fixed. I'll fix the issue I'm
> > currently seeing and then try again to recreate the issue you are
> > seeing. I'll keep the thread updated with how I get on.
>
> I've found and fixed a couple of issues with the management of the
> connection flow control window. The fixes are consistent with the
> workarounds you described (reduce the size of the stream flow control
> window, reduce concurrent streams).
>
> The fixes are included in the latest snapshot from:
>
>
> https://repository.apache.org/content/groups/snapshots/org/apache/tomcat/tomcat/9.0-SNAPSHOT/
>
> (tomcat-9.0-20210701.191821-3270 or later)
>
> If you are able to test the snapshot in your dev environment, it would
> be very helpful to know if the issue has been resolved.
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
--
Re: Possible bug in http2 window size handling in tomcat 9.0.45
Posted by Mark Thomas <ma...@apache.org>.
On 01/07/2021 08:57, Mark Thomas wrote:
> On 01/07/2021 08:41, Erik Nilsson wrote:
>> <Connector port="443"
>> protocol="org.apache.coyote.http11.Http11NioProtocol"
>> connectionTimeout="20000" compression="on" useAsyncIO="false"
>>
>> compressibleMimeType="text/html,text/xml,text/css,text/javascript,text/plain,application/xml,application/javascript,application/json,image/svg+xml"
>>
>> SSLEnabled="true" scheme="https" secure="true"
>> URIEncoding="UTF-8" >
>> <UpgradeProtocol
>> className="org.apache.coyote.http2.Http2Protocol" />
>> </Connector>
>
> Tx.
>
> It looks like there are multiple issues going on here then - in
> additional to the sendfile issue I already fixed. I'll fix the issue I'm
> currently seeing and then try again to recreate the issue you are
> seeing. I'll keep the thread updated with how I get on.
I've found and fixed a couple of issues with the management of the
connection flow control window. The fixes are consistent with the
workarounds you described (reduce the size of the stream flow control
window, reduce concurrent streams).
The fixes are included in the latest snapshot from:
https://repository.apache.org/content/groups/snapshots/org/apache/tomcat/tomcat/9.0-SNAPSHOT/
(tomcat-9.0-20210701.191821-3270 or later)
If you are able to test the snapshot in your dev environment, it would
be very helpful to know if the issue has been resolved.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Possible bug in http2 window size handling in tomcat 9.0.45
Posted by Mark Thomas <ma...@apache.org>.
On 01/07/2021 08:41, Erik Nilsson wrote:
> <Connector port="443" protocol="org.apache.coyote.http11.Http11NioProtocol"
> connectionTimeout="20000" compression="on" useAsyncIO="false"
> compressibleMimeType="text/html,text/xml,text/css,text/javascript,text/plain,application/xml,application/javascript,application/json,image/svg+xml"
> SSLEnabled="true" scheme="https" secure="true" URIEncoding="UTF-8" >
> <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
> </Connector>
Tx.
It looks like there are multiple issues going on here then - in
additional to the sendfile issue I already fixed. I'll fix the issue I'm
currently seeing and then try again to recreate the issue you are
seeing. I'll keep the thread updated with how I get on.
Mark
>
>
> Den tors 1 juli 2021 kl 09:32 skrev Mark Thomas <ma...@apache.org>:
>
>> On 01/07/2021 07:16, Erik Nilsson wrote:
>>> Hmm I can still get the same exceptions even if I set useAsyncIO="false",
>>> but with maxConcurrentStreamExecution="1" it's stable.
>>
>> Can you provide your entire Connector configuration that includes
>> useAsyncIO="false" please?
>>
>> Thanks,
>>
>> Mark
>>
>>
>>> /Erik
>>>
>>> Den ons 30 juni 2021 kl 18:41 skrev Mark Thomas <ma...@apache.org>:
>>>
>>>> On 30/06/2021 16:49, Erik Nilsson wrote:
>>>>> Perfect that u can reproduce this with another webapp. Thankful for
>> your
>>>>> quick response. As I pointed out in the beginning of the conversation
>> we
>>>>> also got this problem behind our f5 loadbalancer. But not with if we
>>>>> connect to Tomcat directly without a loadbalancer.
>>>>
>>>> Yes, this is a Tomcat bug. You should be able to work-around it with:
>>>>
>>>> useAsyncIO="false" on the Connector.
>>>>
>>>> I know where the problem is. I "just" need to figure out the fix.
>>>>
>>>> Mark
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>
>>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Possible bug in http2 window size handling in tomcat 9.0.45
Posted by Erik Nilsson <er...@sitevision.se>.
<Connector port="443" protocol="org.apache.coyote.http11.Http11NioProtocol"
connectionTimeout="20000" compression="on" useAsyncIO="false"
compressibleMimeType="text/html,text/xml,text/css,text/javascript,text/plain,application/xml,application/javascript,application/json,image/svg+xml"
SSLEnabled="true" scheme="https" secure="true" URIEncoding="UTF-8" >
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
</Connector>
Den tors 1 juli 2021 kl 09:32 skrev Mark Thomas <ma...@apache.org>:
> On 01/07/2021 07:16, Erik Nilsson wrote:
> > Hmm I can still get the same exceptions even if I set useAsyncIO="false",
> > but with maxConcurrentStreamExecution="1" it's stable.
>
> Can you provide your entire Connector configuration that includes
> useAsyncIO="false" please?
>
> Thanks,
>
> Mark
>
>
> > /Erik
> >
> > Den ons 30 juni 2021 kl 18:41 skrev Mark Thomas <ma...@apache.org>:
> >
> >> On 30/06/2021 16:49, Erik Nilsson wrote:
> >>> Perfect that u can reproduce this with another webapp. Thankful for
> your
> >>> quick response. As I pointed out in the beginning of the conversation
> we
> >>> also got this problem behind our f5 loadbalancer. But not with if we
> >>> connect to Tomcat directly without a loadbalancer.
> >>
> >> Yes, this is a Tomcat bug. You should be able to work-around it with:
> >>
> >> useAsyncIO="false" on the Connector.
> >>
> >> I know where the problem is. I "just" need to figure out the fix.
> >>
> >> Mark
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> >> For additional commands, e-mail: users-help@tomcat.apache.org
> >>
> >>
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
--
Re: Possible bug in http2 window size handling in tomcat 9.0.45
Posted by Mark Thomas <ma...@apache.org>.
On 01/07/2021 07:16, Erik Nilsson wrote:
> Hmm I can still get the same exceptions even if I set useAsyncIO="false",
> but with maxConcurrentStreamExecution="1" it's stable.
Can you provide your entire Connector configuration that includes
useAsyncIO="false" please?
Thanks,
Mark
> /Erik
>
> Den ons 30 juni 2021 kl 18:41 skrev Mark Thomas <ma...@apache.org>:
>
>> On 30/06/2021 16:49, Erik Nilsson wrote:
>>> Perfect that u can reproduce this with another webapp. Thankful for your
>>> quick response. As I pointed out in the beginning of the conversation we
>>> also got this problem behind our f5 loadbalancer. But not with if we
>>> connect to Tomcat directly without a loadbalancer.
>>
>> Yes, this is a Tomcat bug. You should be able to work-around it with:
>>
>> useAsyncIO="false" on the Connector.
>>
>> I know where the problem is. I "just" need to figure out the fix.
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org