You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Sergey Tsalkov <fl...@gmail.com> on 2006/05/28 19:50:29 UTC
[users@httpd] need help fighting DoS attack on Apache
Hey guys.. My Apache was hit with a DoS attack, where the attacker was
opening connections to the server and not sending any data. It quickly
reached the MaxClients limit and prevented any further connections to
the server.
The Server Status was filled with lines like this:
7-2 4039 0/8/8 R 0.01 3 25 0.0 0.01 0.01 ? ? ..reading..
..and the apache log with lines like this:
87.10.176.44 - - [28/May/2006:17:26:24 +0000] "-" 408 - "-" "-"
For some reason, Apache isn't listing the IP of the connection in
Server Status until that connection actually makes a request. Anyone
know why?
Anyways, I tried mod_choke's functionality for limiting multiple
connections from the same IP. That didn't help.. I suspect mod_choke
doesn't activate until a request is received through the connection,
so this script can dodge it by opening connections, not requesting
anything, and keeping them open until they time out. mod_evasive was
similarly unhelpful.
I managed to stop the attack by setting IP bans at the firewall, but
that doesn't actually solve the core problem.
Anyone have any suggestions?
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] need help fighting DoS attack on Apache
Posted by Alexey Polyakov <al...@gmail.com>.
On 5/28/06, Sergey Tsalkov <fl...@gmail.com> wrote:
> Anyone have any suggestions?
What you've described doesn't really look like a dangerous DOS-attack.
If clients just open connections and stay idle, there's a lot of good
workarounds:
1) lower timeout, and raise number of listening servers
2) hack server code a bit and use TCP_DEFER_ACCEPT option if you're on
Linux, or http ready data filter if it's FreeBSD
3) put an Apache 2.0 with threaded MPM & mod_proxy with minimal set of
modules in front of your Apache 1.3 server - it will be able to serve
thousands of idle connections without wasting too much resources.
--
Alexey Polyakov
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] need help fighting DoS attack on Apache
Posted by Joshua Slive <jo...@slive.ca>.
On 5/28/06, Sergey Tsalkov <fl...@gmail.com> wrote:
> This is very wrong. I can't figure out why Apache doesn't have any
> defense against such an obvious attack -- even the connection limiting
> modules can't help because they have no way of knowing that all the
> requests are coming from the same IP.
I believe that some third-party modules can deal with this, although
I've never tried them myself. I would guess that mod_ip_count could
do it, for example.
But the correct way to deal with this, as Grahm already suggested, is
using your OS firewall. It has the capabilities to handle this type
of problem with much less complication and resources than anything
that could be built into apache. Most firewalls can limit the total
connections from any particular IP address, completely eliminating
this simple attack.
See also:
http://httpd.apache.org/docs/trunk/misc/security_tips.html#dos
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] need help fighting DoS attack on Apache
Posted by Nick Kew <ni...@webthing.com>.
On Sunday 28 May 2006 21:23, Sergey Tsalkov wrote:
> I'm using Apache 1.3.36.
*sigh*. Of course a server that's been obnsolete for more than
four years isn't up to date. Any 1.x server leaves you with every
connection tying up an entire process.
> mod_choke
Never heard of it.
> Nick, you mentioned that Apache 2.2 has built-in countermeasures to
> stop this sort of attack..
Compared to what you're using, any 2.x with the worker MPM is going to
be a huge improvement. 2.2's event MPM may improve on that. See also
http://httpd.apache.org/docs/2.2/mod/core.html#acceptfilter
for specific defence that works with any MPM on some operating systems.
--
Nick Kew
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] need help fighting DoS attack on Apache
Posted by Sergey Tsalkov <fl...@gmail.com>.
I'm using Apache 1.3.36. mod_choke is supposed to be able to limit the
number of connections per IP, but fails to do so for the reason
discussed earlier in this thread. mod_evasive, and anti-DoS tool, also
failed to stop the attack.
Nick, you mentioned that Apache 2.2 has built-in countermeasures to
stop this sort of attack.. I haven't been able to find anything on
this myself, so would you mind pointing me to the relevant information
in apache's docs? Maybe I'll have to upgrade..
On 5/28/06, Nick Kew <ni...@webthing.com> wrote:
> On Sunday 28 May 2006 19:23, Sergey Tsalkov wrote:
>
> > This is very wrong. I can't figure out why Apache doesn't have any
> > defense against such an obvious attack -- even the connection limiting
> > modules can't help because they have no way of knowing that all the
> > requests are coming from the same IP.
>
> Which ones have you tried? And are you by any chance using
> the prefork MPM to maximise the cost to you of each connection?
>
> Apache 2.2 on some platforms uses connection filters to defend
> against precisely this kind of attack.
>
> --
> Nick Kew
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] need help fighting DoS attack on Apache
Posted by Nick Kew <ni...@webthing.com>.
On Sunday 28 May 2006 19:23, Sergey Tsalkov wrote:
> This is very wrong. I can't figure out why Apache doesn't have any
> defense against such an obvious attack -- even the connection limiting
> modules can't help because they have no way of knowing that all the
> requests are coming from the same IP.
Which ones have you tried? And are you by any chance using
the prefork MPM to maximise the cost to you of each connection?
Apache 2.2 on some platforms uses connection filters to defend
against precisely this kind of attack.
--
Nick Kew
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] need help fighting DoS attack on Apache
Posted by Sergey Tsalkov <fl...@gmail.com>.
This has nothing to do with the server's ability to serve the content.
Heck, I can even reproduce the effect myself. If I simply run "telnet
localhost 80" from the server, a line like:
2-2 14313 0/3/52 R 0.02 4 3 0.0 0.01 0.09 ? ? ..reading..
is added to Apache Status, and stays there until it either times out
(error 408!) or I close it. Clearly, apache doesn't even try to
determine the IP that the connection is originating from until an
actual request is made. It stands to reason that the origin IP isn't
being passed to modules like mod_choke, so they can't block the
connections.
This means that I could easily DoS myself by running the "telnet
localhost 80" command MaxClients times, leaving each one open until it
times out several minutes later, and re-running.
This is very wrong. I can't figure out why Apache doesn't have any
defense against such an obvious attack -- even the connection limiting
modules can't help because they have no way of knowing that all the
requests are coming from the same IP.
On 5/28/06, Graham Frank <gf...@neoservers.com> wrote:
> Error 408 means request timeout. Make sure your server isn't having an
> issue serving the content.
>
> If you can verify that it is an attack, then read the following; otherwise,
> skip it.
>
> While I will leave the Apache modding suggestions to the people here who are
> sure to do so ... let me give you the iptables command to get rid of this
> issue (assuming you use linux).
>
> iptables -A INPUT -s 87.10.176.44 -j DROP
> iptables -A OUTPUT -s 87.10.176.44 -j DROP
>
> Look into getting a firewall so that you can easily defend against other
> types of DOS attacks.
>
> As for why it doesn't list the IP is exactly because of what server-status
> says: It's still reading the request, and that includes the IP.
>
> --Graham Frank
>
> -----Original Message-----
> From: Sergey Tsalkov [mailto:flightsimguy@gmail.com]
> Sent: Sunday, May 28, 2006 12:50 PM
> To: users@httpd.apache.org
> Subject: [users@httpd] need help fighting DoS attack on Apache
>
> Hey guys.. My Apache was hit with a DoS attack, where the attacker was
> opening connections to the server and not sending any data. It quickly
> reached the MaxClients limit and prevented any further connections to
> the server.
>
> The Server Status was filled with lines like this:
> 7-2 4039 0/8/8 R 0.01 3 25 0.0 0.01 0.01
> ? ? ..reading..
>
> ..and the apache log with lines like this:
> 87.10.176.44 - - [28/May/2006:17:26:24 +0000] "-" 408 - "-" "-"
>
> For some reason, Apache isn't listing the IP of the connection in
> Server Status until that connection actually makes a request. Anyone
> know why?
>
> Anyways, I tried mod_choke's functionality for limiting multiple
> connections from the same IP. That didn't help.. I suspect mod_choke
> doesn't activate until a request is received through the connection,
> so this script can dodge it by opening connections, not requesting
> anything, and keeping them open until they time out. mod_evasive was
> similarly unhelpful.
>
> I managed to stop the attack by setting IP bans at the firewall, but
> that doesn't actually solve the core problem.
>
> Anyone have any suggestions?
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] need help fighting DoS attack on Apache
Posted by Graham Frank <gf...@neoservers.com>.
Error 408 means request timeout. Make sure your server isn't having an
issue serving the content.
If you can verify that it is an attack, then read the following; otherwise,
skip it.
While I will leave the Apache modding suggestions to the people here who are
sure to do so ... let me give you the iptables command to get rid of this
issue (assuming you use linux).
iptables -A INPUT -s 87.10.176.44 -j DROP
iptables -A OUTPUT -s 87.10.176.44 -j DROP
Look into getting a firewall so that you can easily defend against other
types of DOS attacks.
As for why it doesn't list the IP is exactly because of what server-status
says: It's still reading the request, and that includes the IP.
--Graham Frank
-----Original Message-----
From: Sergey Tsalkov [mailto:flightsimguy@gmail.com]
Sent: Sunday, May 28, 2006 12:50 PM
To: users@httpd.apache.org
Subject: [users@httpd] need help fighting DoS attack on Apache
Hey guys.. My Apache was hit with a DoS attack, where the attacker was
opening connections to the server and not sending any data. It quickly
reached the MaxClients limit and prevented any further connections to
the server.
The Server Status was filled with lines like this:
7-2 4039 0/8/8 R 0.01 3 25 0.0 0.01 0.01
? ? ..reading..
..and the apache log with lines like this:
87.10.176.44 - - [28/May/2006:17:26:24 +0000] "-" 408 - "-" "-"
For some reason, Apache isn't listing the IP of the connection in
Server Status until that connection actually makes a request. Anyone
know why?
Anyways, I tried mod_choke's functionality for limiting multiple
connections from the same IP. That didn't help.. I suspect mod_choke
doesn't activate until a request is received through the connection,
so this script can dodge it by opening connections, not requesting
anything, and keeping them open until they time out. mod_evasive was
similarly unhelpful.
I managed to stop the attack by setting IP bans at the firewall, but
that doesn't actually solve the core problem.
Anyone have any suggestions?
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org