HTTP/2 DoS issues announced today - Impact for Apache Tomcat

[Mark Thomas <ma...@apache.org>]

Today Netflix has published a report highlighting various potential DoS
attacks against HTTP/2 implementations [1].

No immediate action is required for Tomcat users since none of the
described attacks result in a DoS with Apache Tomcat.

The Tomcat Security Team has reviewed the impact on Tomcat of each of
these attacks. The load generated by the attacks is comparable to the
load generated by a similar amount of valid client traffic. Therefore,
these requests are not viewed as a DoS by the Tomcat Security Team. We
did look a little harder at the CVE-2019-9513 "Resource Loop" attack as
came closest to exceeding the load generated by valid traffic.

While we do not consider the described attacks to represent a DoS for
Apache Tomcat, they do all represent abusive client behaviour. In
response to these reports we will be expanding the overhead protection
already in place to detect these abusive behaviours and to close the
connection when they are detected.

The expanded overhead detection will be configurable, including the
option to disable it. The configuration will be provided with what we
consider to be reasonable defaults although there is the possibility
that these defaults will be adjusted based on user feedback in future
versions.

This additional protection will be in the next releases of 9.0.x and
8.5.x, currently expected to be 9.0.23 and 8.5.44. The release process
for these versions is expected to start later today.

Mark
on behalf of the Tomcat Security Team




[1]
https://github.com/Netflix/security-bulletins/blob/master/advisories/third-party/2019-002.md

Re: HTTP/2 DoS issues announced today - Impact for Apache Tomcat

["André Warnier (tomcat)" <aw...@ice-sa.com>]

On 13.08.2019 19:48, Mark Thomas wrote:
> Today Netflix has published a report highlighting various potential DoS
> attacks against HTTP/2 implementations [1].
>
> No immediate action is required for Tomcat users since none of the
> described attacks result in a DoS with Apache Tomcat.
>
> The Tomcat Security Team has reviewed the impact on Tomcat of each of
> these attacks. The load generated by the attacks is comparable to the
> load generated by a similar amount of valid client traffic. Therefore,
> these requests are not viewed as a DoS by the Tomcat Security Team. We
> did look a little harder at the CVE-2019-9513 "Resource Loop" attack as
> came closest to exceeding the load generated by valid traffic.
>
> While we do not consider the described attacks to represent a DoS for
> Apache Tomcat, they do all represent abusive client behaviour. In
> response to these reports we will be expanding the overhead protection
> already in place to detect these abusive behaviours and to close the
> connection when they are detected.
>
> The expanded overhead detection will be configurable, including the
> option to disable it. The configuration will be provided with what we
> consider to be reasonable defaults although there is the possibility
> that these defaults will be adjusted based on user feedback in future
> versions.
>
> This additional protection will be in the next releases of 9.0.x and
> 8.5.x, currently expected to be 9.0.23 and 8.5.44. The release process
> for these versions is expected to start later today.
>
> Mark
> on behalf of the Tomcat Security Team
>

Hi.

I believe that any sysadmin responsible for several webservers open to the Internet would 
agree with me, that such attacks are quite common, and extremely annoying to say the very 
least; and that any of these same sysadmins has already had many undisclosable wishes 
regarding the ultimate fate of the perpetrators. And I believe that many of them are quite 
annoyed by the fact that so far, the only defense has seemed to be to install additional 
patches, and to present the other cheek.

This was just some background for the suggestion which follows.
It is in fact not an alternative solution, but only an amendment (or an augmentation), to 
the scheme proposed above. Here it goes..

Instead of just closing the connection when malicious behaviour is detected, would it not 
be more effective, in the long run, to provide a (optional) way to keep this connection 
open for some variable and adjustable time (without doing anything resource-intensive, but 
ignoring what the attacker may be sending on it), in order to use up the resources of the 
attacker, for a change ?
Whoever wanted to indeed close the connection right away, could set up his server to do 
just that. But whoever wanted to participate to the collective self-defense scheme which 
is suggested here, would also be able to do that.

I mention collective self-defense, because I am convinced that if such a scheme was 
somewhat widely adopted, it would quickly become uneconomical for perpetrators in general 
to even try their schemes. After all, any DoS (or other similar) attack on a webserver, 
requires an agent, which itself uses resources and thus has a cost. If such an agent can 
quickly determine that its attack doesn't work (as when its connection is immediately 
closed), it can just as quickly move on to the next target and start annoying someone 
else. But if the target reacts in such a way that the agent cannot determine if this is 
just a normal server-busy/network delay or "the scheme", then it cannot move on, and has 
to continue using up its own resources on the current target (ultimately to no avail, but 
that is just the point); which means that to attack a similar number of targets, an 
attacker ultimately has to put more and more resources in play. That's the economic 
argument (*).

Some random delay, with inferior and superior adjustable limits, might just do that.
To be most effective, this should be the default tomcat configuration, with possibility 
for the sysadmin to disable it for edge cases (or just because they are unbelievers).
As people update their tomcats over time, it should be so that at some point, the 
worldwide percentage population of tomcat servers applying this form of defense would 
render it uneconomical for potential attackers to target them, leading to the population 
of perpetrators itself to decline.  Which would be a universal benefit not only to tomcat 
admins, but to webserver admins in general. So it would act a bit like a vaccine..
That might be worth an IgNobel prize, wouldn't it ?

I am leaving the implementation details to my colleagues here who can really program Java 
and Tomcat..

Having proposed a similar scheme to the Apache httpd list a few years ago (unsuccesfully, 
I must admit), I am quite familiar with some of the counter-arguments. I'm curious to see 
if the same ones occur here and now.

(*) The basic observation is : if the average server closes the connection within 10 ms 
e.g., then one malicious agent can try 100 servers/second.  But if the average server 
closes the connection only after 2 seconds e.g. (randomly between 0 and 4 seconds(**)), 
then one agent can try only 0.5 servers per second. Vive-versa, if some perpetrator wants 
to keep trying 100 servers/s, they now need 200 agents instead of one.
So it boils down to a cost comparison like :
- how much does it cost (in resources) for a tomcat server, to keep some malicious client 
connection open for an extra 2 seconds after detecting the malice, while doing nothing
versus
- how much does it cost (in resources) for a malicious attacker, to allocate 199 
additional agents
and I have a feeling that on the tomcat side, allocating a light-weight thread to roll a 
dice, wait, and close a connection is not very costly; while on the client side, 
allocating an extra process or thread to make a connection to a server and try its noxious 
thing, is a lot more expensive.

For anyone interested in calculating the economics more in detail, there is an article 
here with some data :
https://securityaffairs.co/wordpress/57429/cyber-crime/cost-ddos-attack-service.html

(**) it should be somewhat random, so that an atacker could not determine a pattern. But 
it does not have to be encryption-grade random.


>
>
>
> [1]
> https://github.com/Netflix/security-bulletins/blob/master/advisories/third-party/2019-002.md
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org