Re: Firefox triggers HTTP2 overhead protection - known issue?

[Mark Thomas <ma...@apache.org>]

On 06/05/2021 14:36, Mark Thomas wrote:

<snip/>

>> It's probably worth us taking some time to adapt markt's SO answer 
>> there into a whole section on "Protocol Abuse and Protection Features" 
>> in the HTTP/2 configuration guide.
> 
> There is an open issue for Chrome:
> https://bugs.chromium.org/p/chromium/issues/detail?id=1000809
> 
> but there hasn't been much movement.
> 
> I'll add something to my TODO list to see what can be done to avoid 
> false positives like this as it looks like short sequences of small 
> packets can occur depending on how the client buffers data.

I've been doing some testing with both Chrome and FireFox and it appears 
that the behaviour of both has changed and that Tomcat's current 
overhead protection is likely to get triggered in normal usage.

I'm currently investigating different overhead settings. I am trying to 
achieve the following:

- a steady stream of normal requests should result in a steady fall in 
the overhead count (it currently results in no change over time)

- the occasional "silly" frame (such as 2 30 byte data frames in the 
middle of a 700K upload) should not trigger the excessive overhead 
protection

- there should be a greater range of possible values for 
overheadCountFactor other than 1 (normal), 2 (aggresive) and (>2 nothing 
will work)

- additional debug logging so it is easier to track the current overhead 
count and what is driving it

I hope to have something in time for the July release round.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Firefox triggers HTTP2 overhead protection - known issue?

[Mark Thomas <ma...@apache.org>]

On 11/06/2021 12:40, Mark Thomas wrote:
> On 06/05/2021 14:36, Mark Thomas wrote:
> 
> <snip/>
> 
>>> It's probably worth us taking some time to adapt markt's SO answer 
>>> there into a whole section on "Protocol Abuse and Protection 
>>> Features" in the HTTP/2 configuration guide.
>>
>> There is an open issue for Chrome:
>> https://bugs.chromium.org/p/chromium/issues/detail?id=1000809
>>
>> but there hasn't been much movement.
>>
>> I'll add something to my TODO list to see what can be done to avoid 
>> false positives like this as it looks like short sequences of small 
>> packets can occur depending on how the client buffers data.
> 
> I've been doing some testing with both Chrome and FireFox and it appears 
> that the behaviour of both has changed and that Tomcat's current 
> overhead protection is likely to get triggered in normal usage.
> 
> I'm currently investigating different overhead settings. I am trying to 
> achieve the following:
> 
> - a steady stream of normal requests should result in a steady fall in 
> the overhead count (it currently results in no change over time)
> 
> - the occasional "silly" frame (such as 2 30 byte data frames in the 
> middle of a 700K upload) should not trigger the excessive overhead 
> protection
> 
> - there should be a greater range of possible values for 
> overheadCountFactor other than 1 (normal), 2 (aggresive) and (>2 nothing 
> will work)
> 
> - additional debug logging so it is easier to track the current overhead 
> count and what is driving it
> 
> I hope to have something in time for the July release round.

I think there is merit in the above approach but it doesn't fully 
address the issue with FireFox.

After some further investigation I have opted for a different approach 
to address this issue.

Clients tend to send small amounts of data when they reach the end of 
the buffer. If the client buffer size and the HTTP/2 initial window size 
are similar, these writes can be very small.

These small writes are what Tomcat thinks are abusive (because small 
writes in the middle of a request body are inefficient and some servers 
could be DoS'd with lots of them).

Tomcat currently responds with a WINDOW_UPDATE after every read. Small 
client writes mean small window updates.

Clients also appear to write until the current flow control window is 
empty and then look to see if there is a window update. If there is, 
they apply that Window update and carry on writing. If the window update 
is small, so is the subsequent write.

The fix that seems to be working well for Chrome and Firefox is to delay 
any small window updates for the stream, combine them with subsequent 
window updates and only send the window update when there is a 
reasonable (> overheadWindowUpdateThreshold) sized update to send.

I also have some additional debug logging for sending Window updates and 
for tracking the current overhead count that I'll add.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org