You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Ben Macintosh <bm...@gmail.com> on 2007/11/18 16:28:36 UTC
[users@httpd] How to prevent from simple DoS?
Hi
I'm currently facing a problem which I can't find any help for.
Every once in a while, my webserver doesn't respond to requests
anymore, i.e. the browser simply keeps on loading but doesn't get any
data.
Using the status mod I found that in such a situation every possible
"slot" is being used by requests staying in "..reading.." status.
After restarting apache all the pending requests get processed but
after a few seconds all the slots are being blocked by the
"..reading.." status again.
After some tests I could reproduce the situation with simply
initiating multiple telnet session to the webserver without sending
any data. Every such request blocks a slot for the default timeout of
300 seconds.
Is this common behaviour? If so, how to prevent it?
As I understand the issue it's a very simple DoS as it neither does
require a lot of cpu nor bandwidth on the client side.
The only solutions I can see is lowering the timeout and increasing
MaxClients. Although a higher MaxClients probably won't help much,
because it's easy to start hundreds of sessions.
Regarding timeout - can I lower this drastically to about 5 secs
without creating new problems?
I'm using Apache 2.0.54 from Debian Sarge.
Here's a screenshot of the status page:
http://img520.imageshack.us/my.php?image=statushi3.png
Any help appreciated
Thanks, Ben
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Ben Macintosh <bm...@gmail.com>.
2007/11/19, Greg Boyington <gr...@regex.ca>:
> On Nov 19, 2007 3:21 AM, Christian Folini <ch...@post.ch> wrote:
> > Hey Greg,
> >
> > could you elaborate on this? How would you prevent this
> > attack with mod_access?
>
> In one case where an attack was under way but I didn't have access to
> the firewall, I added something like:
>
> Order Deny,Allow
> # offending requests coming from this class C
> Deny from 192.168.123
> Allow from all
Using mod_access can stop an attacker, but it can't prevent the attack
at first, right? In my case the IPs were always different, so this
won't help me much in the future.
Ben
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Greg Boyington <gr...@regex.ca>.
On Nov 19, 2007 10:47 AM, Nick Kew <ni...@webthing.com> wrote:
> On Mon, 19 Nov 2007 09:59:20 -0500
> "Greg Boyington" <gr...@regex.ca> wrote:
>
> > On Nov 19, 2007 3:21 AM, Christian Folini <ch...@post.ch>
> > wrote:
> > > Hey Greg,
> > >
> > > could you elaborate on this? How would you prevent this
> > > attack with mod_access?
> >
> > In one case where an attack was under way but I didn't have access to
> > the firewall, I added something like:
> >
> > Order Deny,Allow
> > # offending requests coming from this class C
> > Deny from 192.168.123
>
> "This class C" is in 192.168, which means it's your organisation's
> internal network!
Er, this was for example purposes only. I figured "Something like"
might have made that clear; sorry for the confusion.
> Then you just played quite a big part in DOSing yourself.
> Enabling .htaccess is a BIG performance hit, affecting
> all HTTP traffic - including what you subsequently allow.
I would suggest that you are overstating the impact of .htaccess.
That aside, enabling .htaccess and encurring whatever performance
penalty it places upon the server is better than suffering the effects
of a DoS, at least for the duration of the attack.
-G
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Nick Kew <ni...@webthing.com>.
On Mon, 19 Nov 2007 09:59:20 -0500
"Greg Boyington" <gr...@regex.ca> wrote:
> On Nov 19, 2007 3:21 AM, Christian Folini <ch...@post.ch>
> wrote:
> > Hey Greg,
> >
> > could you elaborate on this? How would you prevent this
> > attack with mod_access?
>
> In one case where an attack was under way but I didn't have access to
> the firewall, I added something like:
>
> Order Deny,Allow
> # offending requests coming from this class C
> Deny from 192.168.123
"This class C" is in 192.168, which means it's your organisation's
internal network!
> Allow from all
>
> to the docroot's .htaccess file.
Then you just played quite a big part in DOSing yourself.
Enabling .htaccess is a BIG performance hit, affecting
all HTTP traffic - including what you subsequently allow.
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Greg Boyington <gr...@regex.ca>.
> Thanks. Hoped you had found the silver bullet though. :)
Nope, sadly. Although I haven't attempted it myself, I've been
wondering of late if using tcp wrappers in combination with httpd
would be effective/advisable. On our FreeBSD systems, we use a little
python script in combination with hosts.allow that effectively
blackholes all those automated ssh brute force attempts you see from
time to time. Has anyone on the list wrapped httpd?
-G
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Christian Folini <ch...@post.ch>.
On Mon, Nov 19, 2007 at 09:59:20AM -0500, Greg Boyington wrote:
> to the docroot's .htaccess file. Not as effective as dropping the
> packets at the border, obviously, but in this case the attacker wasn't
> very bright/determined.
I see. Yes, this helps for a non-determined DoS attack.
Thanks. Hoped you had found the silver bullet though. :)
regs,
Christian
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Greg Boyington <gr...@regex.ca>.
On Nov 19, 2007 3:21 AM, Christian Folini <ch...@post.ch> wrote:
> Hey Greg,
>
> could you elaborate on this? How would you prevent this
> attack with mod_access?
In one case where an attack was under way but I didn't have access to
the firewall, I added something like:
Order Deny,Allow
# offending requests coming from this class C
Deny from 192.168.123
Allow from all
to the docroot's .htaccess file. Not as effective as dropping the
packets at the border, obviously, but in this case the attacker wasn't
very bright/determined.
-G
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Ben Macintosh <bm...@gmail.com>.
2007/11/19, Christian Folini <ch...@post.ch>:
> > > > As I understand the issue it's a very simple DoS as it neither does
> > > > require a lot of cpu nor bandwidth on the client side.
>
> Is there a proper name for this kind of attack. I am not sure
> the original question was referring to a real attack. But if
> it is one, what would be the correct name for this type?
> I have been referring to it as "Request Delaying". But there
> might be a better name, that is more widespread.
>
> See http://permalink.gmane.org/gmane.comp.apache.mod-security.user/1923
> form some thoughts.
Hi Christian
I don't know any particular name for this attack, but I guess "Request
Delaying" is suitable.
I'm actually not entirely sure whether I'm dealing with an attack or
not, but currently I tend more towards some corrupted client software
/ script or the like, as it only happens about once a week, never
lasts longer than a few minutes and originates always from very
different ip addresses. In addition I searched the logfiles for these
ips and always found legitimate traffic before the "attack".
Interestingly on a test on Apache 2.2.3 from Debian Etch the behaviour
seems to be different. When opening a telnet session it doesn't look
like a httpd child has been assigned to this connection and it isn't
even mentioned in the server-status page. Maybe this issue has already
been addressed in newer releases?
Note the different output from netstat on Apache 2.0.54/Debian Sarge
when telnetting to the server (State, User, INode, PID):
ESTABLISHED 33 18526669 15770/apache2
and Apache 2.2.3/Debian Etch:
SYN_RECV 0 0 -
The requests are really handled very differently.
Thanks for your help, Ben
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Christian Folini <ch...@post.ch>.
On Sun, Nov 18, 2007 at 11:02:21AM -0500, Greg Boyington wrote:
> I like the firewall approach myself, as it seems likely that anyone
> with malicious intent (as distinct from the uninformed download
> accelerator user, etc) should forfeit their rights to your bandwidth
> regardless of protocol. But for a purely apache solution, have a look
> at mod_access ( http://httpd.apache.org/docs/2.0/mod/mod_access.html
> ).
Hey Greg,
could you elaborate on this? How would you prevent this
attack with mod_access?
> > > As I understand the issue it's a very simple DoS as it neither does
> > > require a lot of cpu nor bandwidth on the client side.
Is there a proper name for this kind of attack. I am not sure
the original question was referring to a real attack. But if
it is one, what would be the correct name for this type?
I have been referring to it as "Request Delaying". But there
might be a better name, that is more widespread.
See http://permalink.gmane.org/gmane.comp.apache.mod-security.user/1923
form some thoughts.
regs,
Christian
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Greg Boyington <gr...@regex.ca>.
On Nov 18, 2007 10:32 AM, Joshua Slive <jo...@slive.ca> wrote:
> On Nov 18, 2007 10:28 AM, Ben Macintosh <bm...@gmail.com> wrote:
> > Hi
> > I'm currently facing a problem which I can't find any help for.
> > Every once in a while, my webserver doesn't respond to requests
> > anymore, i.e. the browser simply keeps on loading but doesn't get any
> > data.
> >
> > Using the status mod I found that in such a situation every possible
> > "slot" is being used by requests staying in "..reading.." status.
> > After restarting apache all the pending requests get processed but
> > after a few seconds all the slots are being blocked by the
> > "..reading.." status again.
> >
> > After some tests I could reproduce the situation with simply
> > initiating multiple telnet session to the webserver without sending
> > any data. Every such request blocks a slot for the default timeout of
> > 300 seconds.
> >
> > Is this common behaviour? If so, how to prevent it?
> > As I understand the issue it's a very simple DoS as it neither does
> > require a lot of cpu nor bandwidth on the client side.
>
> See:
> http://httpd.apache.org/docs/trunk/misc/security_tips.html#dos
>
> The standard solution is a simple firewall rule to control number of
> connections per ip at some reasonable level.
>
> Joshua.
I like the firewall approach myself, as it seems likely that anyone
with malicious intent (as distinct from the uninformed download
accelerator user, etc) should forfeit their rights to your bandwidth
regardless of protocol. But for a purely apache solution, have a look
at mod_access ( http://httpd.apache.org/docs/2.0/mod/mod_access.html
).
-G
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Nick Kew <ni...@webthing.com>.
On Mon, 19 Nov 2007 21:19:29 +0100
"Ben Macintosh" <bm...@gmail.com> wrote:
> > 5 minutes??? Where does that come from?
>
> That's the default timeout that a http child waits for, before closing
> the connection.
Really? I thought it was something more sensible these days.
5 seconds would make much more sense. And the Event MPM also helps,
by breaking the link between keepalives and workers.
> Thanks for pointing me to the right direction - never heard about
> AcceptFilter before.
:-)
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Ben Macintosh <bm...@gmail.com>.
2007/11/19, Joshua Slive <jo...@slive.ca>:
> On Nov 19, 2007 3:19 PM, Ben Macintosh <bm...@gmail.com> wrote:
>
> > Thanks for pointing me to the right direction - never heard about
> > AcceptFilter before.
>
> Interesting, because it is specifically suggested in the link that I
> sent you two days ago.
Hi Joshua
*reading link again*
Yeah, you are right. Sorry, don't know why I skipped it, maybe because
of the "This is active by default in Apache httpd". Should have seen
it.
Anyway, thanks to everyone for your help.
Ben
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Joshua Slive <jo...@slive.ca>.
On Nov 19, 2007 3:19 PM, Ben Macintosh <bm...@gmail.com> wrote:
> Thanks for pointing me to the right direction - never heard about
> AcceptFilter before.
Interesting, because it is specifically suggested in the link that I
sent you two days ago.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Ben Macintosh <bm...@gmail.com>.
2007/11/19, Nick Kew <ni...@webthing.com>:
> On Mon, 19 Nov 2007 20:19:20 +0100
> "Ben Macintosh" <bm...@gmail.com> wrote:
> > I already thought about using a firewall rule. Although it could be
> > quite difficult to get it right. As every malicious request blocks a
> > slot for 5 minutes there hasn't got to be a lot of traffic/requests.
>
> 5 minutes??? Where does that come from?
That's the default timeout that a http child waits for, before closing
the connection.
> Maybe you might want to use AcceptFilter to prevent malicious requests
> tying anything up for more than a couple of microseconds?
That's it!
And since AcceptFilter is only available on Apache > 2.1.5 that's also
the reason why it didn't work with Debian Sarge (Apache 2.0.54) but
was working as intended ootb in Debian Etch (Apache 2.2.3).
If "AcceptFilter http none" is set, both versions behave exactly in
the same (bad) way, but when "AcceptFilter http httpready" is set,
it's working again.
Thanks for pointing me to the right direction - never heard about
AcceptFilter before.
Ben
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Nick Kew <ni...@webthing.com>.
On Mon, 19 Nov 2007 20:19:20 +0100
"Ben Macintosh" <bm...@gmail.com> wrote:
> 2007/11/18, Joshua Slive <jo...@slive.ca>:
>
> > See:
> > http://httpd.apache.org/docs/trunk/misc/security_tips.html#dos
> >
> > The standard solution is a simple firewall rule to control number of
> > connections per ip at some reasonable level.
>
> I already thought about using a firewall rule. Although it could be
> quite difficult to get it right. As every malicious request blocks a
> slot for 5 minutes there hasn't got to be a lot of traffic/requests.
5 minutes??? Where does that come from?
Maybe you might want to use AcceptFilter to prevent malicious requests
tying anything up for more than a couple of microseconds?
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Ben Macintosh <bm...@gmail.com>.
2007/11/18, Joshua Slive <jo...@slive.ca>:
> See:
> http://httpd.apache.org/docs/trunk/misc/security_tips.html#dos
>
> The standard solution is a simple firewall rule to control number of
> connections per ip at some reasonable level.
I already thought about using a firewall rule. Although it could be
quite difficult to get it right. As every malicious request blocks a
slot for 5 minutes there hasn't got to be a lot of traffic/requests.
But I guess that's a possibility which would work, even though I'd
prefer an apache solution.
Ben
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] How to prevent from simple DoS?
Posted by Joshua Slive <jo...@slive.ca>.
On Nov 18, 2007 10:28 AM, Ben Macintosh <bm...@gmail.com> wrote:
> Hi
> I'm currently facing a problem which I can't find any help for.
> Every once in a while, my webserver doesn't respond to requests
> anymore, i.e. the browser simply keeps on loading but doesn't get any
> data.
>
> Using the status mod I found that in such a situation every possible
> "slot" is being used by requests staying in "..reading.." status.
> After restarting apache all the pending requests get processed but
> after a few seconds all the slots are being blocked by the
> "..reading.." status again.
>
> After some tests I could reproduce the situation with simply
> initiating multiple telnet session to the webserver without sending
> any data. Every such request blocks a slot for the default timeout of
> 300 seconds.
>
> Is this common behaviour? If so, how to prevent it?
> As I understand the issue it's a very simple DoS as it neither does
> require a lot of cpu nor bandwidth on the client side.
See:
http://httpd.apache.org/docs/trunk/misc/security_tips.html#dos
The standard solution is a simple firewall rule to control number of
connections per ip at some reasonable level.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org