You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by "Mark A. Craig" <ma...@gmail.com> on 2007/09/14 00:57:35 UTC
[users@httpd] and deny directives
My little humble blog has been getting way too much attention from
comment spammers, and I'd like to begin cutting them off. I've
identified the hostnames they use to snoop for details to construct fake
referer data, and now I want to cut them off. I thought I could do it
like this:
<Directory "C:/www/blog">
Options None
AllowOverride None
#Order allow,deny
#Allow from all
order deny,allow
deny from .svservers.com
deny from .static.theplanet.com
deny from .goo.ne.jp
deny from .cable.casema.nl
deny from -xbox.dedi.inhoster.com
deny from .knology.net
deny from .pppoe.mtu-net.ru
deny from .barak-online.net
deny from .barak.net.il
deny from .keymachine.de
deny from .hostingprod.com
deny from .client.bresnan.net
deny from 210.240.
deny from .dsl.ip.tiscali.nl
deny from .qwerty.ru
deny from .t-ipconnect.de
</Directory>
There's only one problem: it's not working! The log still shows visits
from these hostnames. What am I missing? Do I need to add "/*" to the
end of the <Directory> directive, or do subdirectories implicitly
inherit the same directives?
Mark
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] and deny directives
Posted by "Mark A. Craig" <ma...@gmail.com>.
Danie:
I presume that you meant to imply that my DENY list should then follow ALLOW
FROM ALL? Given that my directives aren't denying anything they shouldn't, but
only failing to deny a few things they should, I guess I'm not understanding why
I would need to change the ordering in the way you suggested. Could you explain
it further?
Mark
-------- Original Message --------
Subject: Re: [users@httpd] <directory> and deny directives
From: Danie Qian <da...@bestningning.com>
To: users@httpd.apache.org
Date: Thursday, September 13, 2007 08:31:24 PM
>
> ----- Original Message ----- From: "Mark A. Craig" <ma...@gmail.com>
> To: <us...@httpd.apache.org>
> Sent: Thursday, September 13, 2007 6:57 PM
> Subject: [users@httpd] <directory> and deny directives
>
>
>> My little humble blog has been getting way too much attention from
>> comment spammers, and I'd like to begin cutting them off. I've
>> identified the hostnames they use to snoop for details to construct
>> fake referer data, and now I want to cut them off. I thought I could
>> do it like this:
>>
>> <Directory "C:/www/blog">
>> Options None
>> AllowOverride None
>> #Order allow,deny
>> #Allow from all
>> order deny,allow
>> deny from .svservers.com
>> deny from .static.theplanet.com
>> deny from .goo.ne.jp
>> deny from .cable.casema.nl
>> deny from -xbox.dedi.inhoster.com
>> deny from .knology.net
>> deny from .pppoe.mtu-net.ru
>> deny from .barak-online.net
>> deny from .barak.net.il
>> deny from .keymachine.de
>> deny from .hostingprod.com
>> deny from .client.bresnan.net
>> deny from 210.240.
>> deny from .dsl.ip.tiscali.nl
>> deny from .qwerty.ru
>> deny from .t-ipconnect.de
>> </Directory>
>>
>> There's only one problem: it's not working! The log still shows
>> visits from these hostnames. What am I missing? Do I need to add
>> "/*" to the end of the <Directory> directive, or do subdirectories
>> implicitly inherit the same directives?
>>
>> Mark
>>
>>
>>
>
> you should use
>
> Order allow,deny
> Allow from all
>
> instead of
>
> order deny,allow
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] and deny directives
Posted by Danie Qian <da...@bestningning.com>.
----- Original Message -----
From: "Mark A. Craig" <ma...@gmail.com>
To: <us...@httpd.apache.org>
Sent: Thursday, September 13, 2007 6:57 PM
Subject: [users@httpd] <directory> and deny directives
> My little humble blog has been getting way too much attention from comment
> spammers, and I'd like to begin cutting them off. I've identified the
> hostnames they use to snoop for details to construct fake referer data,
> and now I want to cut them off. I thought I could do it like this:
>
> <Directory "C:/www/blog">
> Options None
> AllowOverride None
> #Order allow,deny
> #Allow from all
> order deny,allow
> deny from .svservers.com
> deny from .static.theplanet.com
> deny from .goo.ne.jp
> deny from .cable.casema.nl
> deny from -xbox.dedi.inhoster.com
> deny from .knology.net
> deny from .pppoe.mtu-net.ru
> deny from .barak-online.net
> deny from .barak.net.il
> deny from .keymachine.de
> deny from .hostingprod.com
> deny from .client.bresnan.net
> deny from 210.240.
> deny from .dsl.ip.tiscali.nl
> deny from .qwerty.ru
> deny from .t-ipconnect.de
> </Directory>
>
> There's only one problem: it's not working! The log still shows visits
> from these hostnames. What am I missing? Do I need to add "/*" to the
> end of the <Directory> directive, or do subdirectories implicitly inherit
> the same directives?
>
> Mark
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
you should use
Order allow,deny
Allow from all
instead of
order deny,allow
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] deny directives
Posted by "Mark A. Craig" <ma...@gmail.com>.
Everyone:
I found a workaround to this non-intuitive little quirk in how Apache evaluates
DENY directives with (partial) hostnames: I'm using SetEnvIf expressions to
control a variable, which I then reference in a single DENY directive:
# assume spammers doing recon will only perform GETs
SetEnvIfNoCase Request_Method "GET" spammer_recon
# assume spammers always use both empty Referer and U-A
SetEnvIfNoCase Referer ".+" !spammer_recon
SetEnvIfNoCase User-Agent ".+" !spammer_recon
# if the host/IP is any of these, they're definitely spammers regardless!
SetEnvIfNoCase Remote_Host "\.barak\-online\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.barak\.net\.il" spammer_recon
SetEnvIfNoCase Remote_Host "\.cable\.casema\.nl" spammer_recon
SetEnvIfNoCase Remote_Host "\.client\.bresnan\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.ctinets\.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.dip\.t\-dialin\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.dsl\.ip\.tiscali\.nl" spammer_recon
SetEnvIfNoCase Remote_Host "\.easyspeedy\.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.goo\.ne\.jp" spammer_recon
SetEnvIfNoCase Remote_Host "\.hostingprod\.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.internetserviceteam\.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.keymachine\.de" spammer_recon
SetEnvIfNoCase Remote_Host "\.knology\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.lorerweb\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.onlinehome\-server\.info" spammer_recon
SetEnvIfNoCase Remote_Host "\.pppoe\.mtu-net\.ru" spammer_recon
SetEnvIfNoCase Remote_Host "\.qwerty\.ru" spammer_recon
SetEnvIfNoCase Remote_Host "\.sputnikmedia\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.static\.theplanet\.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.starnet\.md" spammer_recon
SetEnvIfNoCase Remote_Host "\.starnet\.ru" spammer_recon
SetEnvIfNoCase Remote_Host "\.svservers\.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.dip\.t\-dialin\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.dip0\.t\-ipconnect\.de" spammer_recon
SetEnvIfNoCase Remote_Host "\-xbox\.dedi\.inhoster\.com" spammer_recon
SetEnvIfNoCase Remote_Addr "^\210\.240\." spammer_recon
<Directory "C:/Documents and Settings/macraig/Desktop/local Web/blog">
Options None
AllowOverride None
#Order allow,deny
#Allow from all
order deny,allow
deny from env=spammer_recon
</Directory>
The conditionals don't suffer from the same quirk as DENY directives, so my
partial hostnames *always* work as intended, regardless of the results of an
rDNS. So far this is working like a charm. Since I also found that most of the
time the spammers don't send U-A or Referer info when they're doing recon and of
course are using GETs, I was able to add a generic catch-all at the front, with
the confirmed evil-doers bringing up the rear. I don't know what effect this
has on performance, but I don't expect this list to grow enormously. If anyone
sees a way to improve the parsing efficiency of my regular expressions (I
suspect I suck at constructing them), I'd love the help.
Mark
-------- Original Message --------
Subject: Re: [users@httpd] <directory> and deny directives
From: Mark A. Craig <ma...@gmail.com>
To: users@httpd.apache.org
Date: Friday, September 14, 2007 08:57:49 AM
> Joshua:
>
> I see what you mean about the rDNS, though perversely it was the
> svservers.com case that drove me to use partial hostnames in the first
> place, because they lease from multiple IP blocks from multiple sources,
> and I've been getting spam recon from all of them, so I thought I could
> kill all the birds with just the one hostname stone. It seemed
> intuitive at the time.... As of last night, I have another instance
> just like it, a different partial hostname from my list that passes
> thru, apparently because the actual IP address doesn't fall in the
> returned IP range when an rDNS *on just the primary domain* is
> performed. I think you're right.
>
> One way to test it: substitute partial IP addresses, to represent each
> of the leased IP blocks. It's more work, but I'll see what happens. It
> would sure be nice if the code didn't pull a non-intuitive stunt like
> this, though! If the DNS lookup resolves to the specified *partial*
> hostname, it should act on it, not second-guess it with an rDNS like this.
>
> Mark
>
> -------- Original Message --------
> Subject: Re: [users@httpd] <directory> and deny directives
> From: Joshua Slive <jo...@slive.ca>
> To: users@httpd.apache.org
> Date: Friday, September 14, 2007 06:06:13 AM
>
>> On 9/14/07, Mark A. Craig <ma...@gmail.com> wrote:
>>> Joshua:
>>>
>>> Thanks for the quick and comprehensive reply. Lemme address
>>> everything in order:
>>>
>>> 1. Whatcha mean by "the config is inherited"? Did you mean to
>>> address my
>>> question about sub-directories? I suspect so, but if not please
>>> clarify.
>>>
>>> 2. The status codes are in fact mostly 403s, but not ALL... some that
>>> match my
>>> deny directives, notably ".svservers.com", are still being allowed
>>> with 200s.
>>> The 403s that are occurring could also be the result of the http:BL
>>> module in
>>> the blog software itself, which checks the IPs of attempted
>>> commenters against
>>> the Project Honeypot DNS blacklist and bounces them with a 403 if the
>>> IP is a
>>> match (there's a lot of 403s for hostnames not in my little DENY
>>> list). At
>>> least that's the only explanation I can imagine for the inconsistency.
>>>
>>> My goal here is to nail the spammy GETs; at first I'd considered a
>>> <LIMIT GET>
>>> directive, but I couldn't figure out where/how to apply it and so
>>> resorted to
>>> this current technique.
>>
>> Don't use <Limit GET>. See the docs on <Limit> for why that would be a
>> mistake.
>>
>> Your config looks basically correct. But of course, other things in
>> your config file could be overriding it. If you replace all those Deny
>> directives with a "Deny from all", do you block all access? If not,
>> then either you aren't editing the correct place in the config file,
>> or you are overriding this config someplace else (such as in a
>> <Location> section).
>>
>> Another likely issue is your use of hostnames. The hostnames that are
>> getting the 200 response above have messed-up reverse lookups. (The
>> domain you get when looking up the IP address does not map back to
>> that IP address.) Although I haven't checked the code, it is possible
>> that apache is ignoring those ones because it can't confirm whether or
>> not the client is really in that domain.
>>
>> In general, it is better to use IP addresses for blocking instead of
>> domains.
>>
>> Joshua.
>>
>> ---------------------------------------------------------------------
>> The official User-To-User support forum of the Apache HTTP Server
>> Project.
>> See <URL:http://httpd.apache.org/userslist.html> for more info.
>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>> " from the digest: users-digest-unsubscribe@httpd.apache.org
>> For additional commands, e-mail: users-help@httpd.apache.org
>>
>>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] and deny directives
Posted by "Mark A. Craig" <ma...@gmail.com>.
BTW, I have solved this little problem by relying on SetEnvIf(NoCase)
directives. Perhaps someone else will find this strategy useful:
# assume spammers doing recon will only perform GETs
SetEnvIfNoCase Request_Method "GET" spammer_recon
# assume spammers always use both empty Referer and U-A
SetEnvIfNoCase Referer ".+" !spammer_recon
SetEnvIfNoCase User-Agent ".+" !spammer_recon
# if the host/IP is any of these, they're spammers regardless
SetEnvIfNoCase Remote_Host "\.barak\-online\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.barak\.net\.il" spammer_recon
SetEnvIfNoCase Remote_Host "\.cable\.casema\.nl" spammer_recon
SetEnvIfNoCase Remote_Host "\.client\.bresnan\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.ctinets\.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.dip\.t\-dialin\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.dsl\.ip\.tiscali\.nl" spammer_recon
SetEnvIfNoCase Remote_Host "\.easyspeedy\.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.goo\.ne\.jp" spammer_recon
SetEnvIfNoCase Remote_Host "\.hostingprod\.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.internetserviceteam\.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.keymachine\.de" spammer_recon
SetEnvIfNoCase Remote_Host "\.knology\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.lorerweb\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.onlinehome\-server\.info" spammer_recon
SetEnvIfNoCase Remote_Host "\.pppoe\.mtu-net\.ru" spammer_recon
SetEnvIfNoCase Remote_Host "\.qwerty\.ru" spammer_recon
SetEnvIfNoCase Remote_Host "\.sputnikmedia\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.static\.theplanet\.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.starnet\.md" spammer_recon
SetEnvIfNoCase Remote_Host "\.starnet\.ru" spammer_recon
SetEnvIfNoCase Remote_Host "\.svservers\.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.dip\.t\-dialin\.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.dip0\.t\-ipconnect\.de" spammer_recon
SetEnvIfNoCase Remote_Host "\-xbox\.dedi\.inhoster\.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.keymachine\.de" spammer_recon
SetEnvIfNoCase Remote_Host "\.static\.reverse\.ltdomains.com" spammer_recon
SetEnvIfNoCase Remote_Host "\.pccwglobal\.net" spammer_recon
SetEnvIfNoCase Remote_Host "garner\.funtaff\.com" spammer_recon
SetEnvIfNoCase Remote_Host "server\.rnd\.pl" spammer_recon
SetEnvIfNoCase Remote_Host "\.ap\.yournet\.ne\.jp" spammer_recon
SetEnvIfNoCase Remote_Host "\-rev\.cernel\.net" spammer_recon
SetEnvIfNoCase Remote_Addr "^210\.240\." spammer_recon
SetEnvIfNoCase Remote_Addr "216\.72\.28\.8" spammer_recon
SetEnvIfNoCase Remote_Addr "222\.240\.212\.29" spammer_recon
SetEnvIfNoCase Remote_Addr "220\.84\.58\.27" spammer_recon
SetEnvIfNoCase Remote_Addr "124\.53\.202\.111" spammer_recon
SetEnvIfNoCase Remote_Addr "121\.35\.254\.97" spammer_recon
SetEnvIfNoCase User-Agent "PlantyNet\_WebRobot.*" spammer_recon
SetEnvIfNoCase User-Agent "WordPress.*" spammer_recon
SetEnvIfNoCase User-Agent "topicblogs.*" spammer_recon
#not really spammer recon but a repeat spam POSTer
SetEnvIfNoCase Remote_Host "softbank[0-9]{12}\.bbtec.net" spammer_recon
SetEnvIfNoCase Remote_Host "\.phx\.gbl" spammer_recon
<Directory document_root>
Options Indexes FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all
deny from env=spammer_recon
</Directory>
-------- Original Message --------
Subject: Re: [users@httpd] <directory> and deny directives
From: Joshua Slive <jo...@slive.ca>
To: users@httpd.apache.org
Date: Friday, September 14, 2007 09:08:30 AM
> On 9/14/07, Mark A. Craig <ma...@gmail.com> wrote:
>> It would sure be
>> nice if the code didn't pull a non-intuitive stunt like this, though! If the
>> DNS lookup resolves to the specified *partial* hostname, it should act on it,
>> not second-guess it with an rDNS like this.
>
> Yes, it is non-intuitive. But on the other hand, it is much more
> common to use hostnames for Allow directives than for Deny directives
> (since the hostname is often under the control of the attacker). You
> MUST check the forward and reverse for Allow directives, or else they
> would be worthless. And then it could potentially cause even more
> confusion if the Allow and Deny directives matched differently.
>
> Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] Re: Syntactic consistency? Pfft! {directory and deny directives}
Posted by "Mark A. Craig" <ma...@gmail.com>.
Joshua:
Consistency be damned! As one example of how unproductive that consistency is,
I'm now forced to replace just that ONE ".svservers.com" partial hostname denial
with ELEVEN - and probably counting - separate IP address denials! These are
all addresses used by one person or perhaps a small intimate group. Now
extrapolate that burden to all the other partial IP addresses that I similarly
want to deny.
Isn't that a wonderful extra burden to place on my server just for the sake of
algorithmic and syntactic consistency between two directives? If I had access
and motivation, I'd start an argument about it with the developer of that bit of
code.
Mark
-------- Original Message --------
Subject: Re: [users@httpd] <directory> and deny directives
From: Joshua Slive <jo...@slive.ca>
To: users@httpd.apache.org
Date: Friday, September 14, 2007 09:08:30 AM
> On 9/14/07, Mark A. Craig <ma...@gmail.com> wrote:
>> It would sure be
>> nice if the code didn't pull a non-intuitive stunt like this, though! If the
>> DNS lookup resolves to the specified *partial* hostname, it should act on it,
>> not second-guess it with an rDNS like this.
>
> Yes, it is non-intuitive. But on the other hand, it is much more
> common to use hostnames for Allow directives than for Deny directives
> (since the hostname is often under the control of the attacker). You
> MUST check the forward and reverse for Allow directives, or else they
> would be worthless. And then it could potentially cause even more
> confusion if the Allow and Deny directives matched differently.
>
> Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] and deny directives
Posted by Joshua Slive <jo...@slive.ca>.
On 9/14/07, Mark A. Craig <ma...@gmail.com> wrote:
> It would sure be
> nice if the code didn't pull a non-intuitive stunt like this, though! If the
> DNS lookup resolves to the specified *partial* hostname, it should act on it,
> not second-guess it with an rDNS like this.
Yes, it is non-intuitive. But on the other hand, it is much more
common to use hostnames for Allow directives than for Deny directives
(since the hostname is often under the control of the attacker). You
MUST check the forward and reverse for Allow directives, or else they
would be worthless. And then it could potentially cause even more
confusion if the Allow and Deny directives matched differently.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] and deny directives
Posted by Joshua Slive <jo...@slive.ca>.
On 9/14/07, Mark A. Craig <ma...@gmail.com> wrote:
> It would sure be
> nice if the code didn't pull a non-intuitive stunt like this, though! If the
> DNS lookup resolves to the specified *partial* hostname, it should act on it,
> not second-guess it with an rDNS like this.
Yes, it is non-intuitive. But on the other hand, it is much more
common to use hostnames for Allow directives than for Deny directives
(since the hostname is often under the control of the attacker). You
MUST check the forward and reverse for Allow directives, or else they
would be worthless. And then it could potentially cause even more
confusion if the Allow and Deny directives matched differently.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] and deny directives
Posted by "Mark A. Craig" <ma...@gmail.com>.
Joshua:
I see what you mean about the rDNS, though perversely it was the svservers.com
case that drove me to use partial hostnames in the first place, because they
lease from multiple IP blocks from multiple sources, and I've been getting spam
recon from all of them, so I thought I could kill all the birds with just the
one hostname stone. It seemed intuitive at the time.... As of last night, I
have another instance just like it, a different partial hostname from my list
that passes thru, apparently because the actual IP address doesn't fall in the
returned IP range when an rDNS *on just the primary domain* is performed. I
think you're right.
One way to test it: substitute partial IP addresses, to represent each of the
leased IP blocks. It's more work, but I'll see what happens. It would sure be
nice if the code didn't pull a non-intuitive stunt like this, though! If the
DNS lookup resolves to the specified *partial* hostname, it should act on it,
not second-guess it with an rDNS like this.
Mark
-------- Original Message --------
Subject: Re: [users@httpd] <directory> and deny directives
From: Joshua Slive <jo...@slive.ca>
To: users@httpd.apache.org
Date: Friday, September 14, 2007 06:06:13 AM
> On 9/14/07, Mark A. Craig <ma...@gmail.com> wrote:
>> Joshua:
>>
>> Thanks for the quick and comprehensive reply. Lemme address everything in order:
>>
>> 1. Whatcha mean by "the config is inherited"? Did you mean to address my
>> question about sub-directories? I suspect so, but if not please clarify.
>>
>> 2. The status codes are in fact mostly 403s, but not ALL... some that match my
>> deny directives, notably ".svservers.com", are still being allowed with 200s.
>> The 403s that are occurring could also be the result of the http:BL module in
>> the blog software itself, which checks the IPs of attempted commenters against
>> the Project Honeypot DNS blacklist and bounces them with a 403 if the IP is a
>> match (there's a lot of 403s for hostnames not in my little DENY list). At
>> least that's the only explanation I can imagine for the inconsistency.
>>
>> My goal here is to nail the spammy GETs; at first I'd considered a <LIMIT GET>
>> directive, but I couldn't figure out where/how to apply it and so resorted to
>> this current technique.
>
> Don't use <Limit GET>. See the docs on <Limit> for why that would be a mistake.
>
> Your config looks basically correct. But of course, other things in
> your config file could be overriding it. If you replace all those Deny
> directives with a "Deny from all", do you block all access? If not,
> then either you aren't editing the correct place in the config file,
> or you are overriding this config someplace else (such as in a
> <Location> section).
>
> Another likely issue is your use of hostnames. The hostnames that are
> getting the 200 response above have messed-up reverse lookups. (The
> domain you get when looking up the IP address does not map back to
> that IP address.) Although I haven't checked the code, it is possible
> that apache is ignoring those ones because it can't confirm whether or
> not the client is really in that domain.
>
> In general, it is better to use IP addresses for blocking instead of domains.
>
> Joshua.
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] and deny directives
Posted by Joshua Slive <jo...@slive.ca>.
On 9/14/07, Mark A. Craig <ma...@gmail.com> wrote:
> Joshua:
>
> Thanks for the quick and comprehensive reply. Lemme address everything in order:
>
> 1. Whatcha mean by "the config is inherited"? Did you mean to address my
> question about sub-directories? I suspect so, but if not please clarify.
>
> 2. The status codes are in fact mostly 403s, but not ALL... some that match my
> deny directives, notably ".svservers.com", are still being allowed with 200s.
> The 403s that are occurring could also be the result of the http:BL module in
> the blog software itself, which checks the IPs of attempted commenters against
> the Project Honeypot DNS blacklist and bounces them with a 403 if the IP is a
> match (there's a lot of 403s for hostnames not in my little DENY list). At
> least that's the only explanation I can imagine for the inconsistency.
>
> My goal here is to nail the spammy GETs; at first I'd considered a <LIMIT GET>
> directive, but I couldn't figure out where/how to apply it and so resorted to
> this current technique.
Don't use <Limit GET>. See the docs on <Limit> for why that would be a mistake.
Your config looks basically correct. But of course, other things in
your config file could be overriding it. If you replace all those Deny
directives with a "Deny from all", do you block all access? If not,
then either you aren't editing the correct place in the config file,
or you are overriding this config someplace else (such as in a
<Location> section).
Another likely issue is your use of hostnames. The hostnames that are
getting the 200 response above have messed-up reverse lookups. (The
domain you get when looking up the IP address does not map back to
that IP address.) Although I haven't checked the code, it is possible
that apache is ignoring those ones because it can't confirm whether or
not the client is really in that domain.
In general, it is better to use IP addresses for blocking instead of domains.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] and deny directives
Posted by "Mark A. Craig" <ma...@gmail.com>.
Joshua:
Thanks for the quick and comprehensive reply. Lemme address everything in order:
1. Whatcha mean by "the config is inherited"? Did you mean to address my
question about sub-directories? I suspect so, but if not please clarify.
2. The status codes are in fact mostly 403s, but not ALL... some that match my
deny directives, notably ".svservers.com", are still being allowed with 200s.
The 403s that are occurring could also be the result of the http:BL module in
the blog software itself, which checks the IPs of attempted commenters against
the Project Honeypot DNS blacklist and bounces them with a 403 if the IP is a
match (there's a lot of 403s for hostnames not in my little DENY list). At
least that's the only explanation I can imagine for the inconsistency.
My goal here is to nail the spammy GETs; at first I'd considered a <LIMIT GET>
directive, but I couldn't figure out where/how to apply it and so resorted to
this current technique.
Censored log sample:
66.199.244.34/66.199.244.34.svservers.com [13/Sep/2007:09:44:17 -0700] "GET
/blog/pivot/entry.php?id=29 HTTP/1.1" 200 24892 "-" "-"
70.85.237.82/52.ed.5546.static.theplanet.com [13/Sep/2007:10:01:22 -0700] "GET
/blog/pivot/entry.php?id=74 HTTP/1.1" 403 222 "-" "-"
70.85.237.82/52.ed.5546.static.theplanet.com [13/Sep/2007:10:02:02 -0700] "GET
/blog/pivot/entry.php?id=29 HTTP/1.1" 403 222 "-" "-"
70.85.237.82/52.ed.5546.static.theplanet.com [13/Sep/2007:11:08:23 -0700] "GET
/blog/pivot/entry.php?id=40 HTTP/1.1" 403 222 "-" "-"
70.85.237.82/52.ed.5546.static.theplanet.com [13/Sep/2007:11:38:21 -0700] "GET
/blog/pivot/entry.php?id=84 HTTP/1.1" 403 222 "-" "-"
70.85.237.82/52.ed.5546.static.theplanet.com [13/Sep/2007:12:05:24 -0700] "GET
/blog/pivot/entry.php?id=71 HTTP/1.1" 403 222 "-" "-"
70.85.237.82/52.ed.5546.static.theplanet.com [13/Sep/2007:12:51:01 -0700] "GET
/blog/pivot/entry.php?id=23 HTTP/1.1" 403 222 "-" "-"
70.85.237.82/52.ed.5546.static.theplanet.com [13/Sep/2007:13:02:21 -0700] "GET
/blog/pivot/entry.php?id=74 HTTP/1.1" 403 222 "-" "-"
70.85.237.82/52.ed.5546.static.theplanet.com [13/Sep/2007:13:06:17 -0700] "GET
/blog/pivot/entry.php?id=60 HTTP/1.1" 403 222 "-" "-"
66.199.244.34/66.199.244.34.svservers.com [13/Sep/2007:13:10:04 -0700] "GET
/blog/pivot/entry.php?id=63 HTTP/1.1" 200 28255 "-" "-"
70.85.237.82/52.ed.5546.static.theplanet.com [13/Sep/2007:13:13:16 -0700] "GET
/blog/pivot/entry.php?id=29 HTTP/1.1" 403 222 "-" "-"
66.199.244.34/66.199.244.34.svservers.com [13/Sep/2007:13:24:28 -0700] "GET
/blog/pivot/entry.php?id=40 HTTP/1.1" 200 28032 "-" "-"
66.199.244.34/66.199.244.34.svservers.com [13/Sep/2007:14:14:46 -0700] "GET
/blog/pivot/entry.php?id=40 HTTP/1.1" 200 27820 "-" "-"
70.85.237.82/52.ed.5546.static.theplanet.com [13/Sep/2007:14:32:16 -0700] "GET
/blog/pivot/entry.php?id=71 HTTP/1.1" 403 222 "-" "-"
81.195.31.71/ppp31-71.pppoe.mtu-net.ru [13/Sep/2007:14:34:48 -0700] "GET
/blog/pivot/entry.php?id=66 HTTP/1.1" 403 222 "/blog/pivot/"
"Mozilla/5.0+(Windows;+U;+Windows+NT+5.2;+en-US;+rv:1.8.0.3)+Gecko/20060426+Firefox/1.5.0.11"
70.85.237.82/52.ed.5546.static.theplanet.com [13/Sep/2007:15:03:56 -0700] "GET
/blog/pivot/entry.php?id=60 HTTP/1.1" 403 222 "-" "-"
70.85.237.82/52.ed.5546.static.theplanet.com [13/Sep/2007:15:05:58 -0700] "GET
/blog/pivot/entry.php?id=84 HTTP/1.1" 403 222 "-" "-"
70.85.237.82/52.ed.5546.static.theplanet.com [13/Sep/2007:15:10:04 -0700] "GET
/blog/pivot/entry.php?id=23 HTTP/1.1" 403 222 "-" "-"
Each of those hostnames match my DENY list, yet the svservers.com GETs are being
allowed.
3. "c:/www/blog/" is actually the parent for all the blog content.
4. I only have the one config file, and other changes to it have certainly had
effects (not all good).
5. Yep, I did restart Apache. I always make a habit of killing it before I even
edit the config.
6. No other <directory> directives for anything underneath ./blog/.
Could it have anyhing to do with the fact that .svservers.com is the FIRST deny
directive? Did I perhaps not structure the permissions correctly? Someone else
suggested I should have stuck with ORDER ALLOW,DENY and then ALLOW FROM ALL (and
presumably followed by the list of DENY); is that how I should have structured it?
Mark
-------- Original Message --------
Subject: Re: [users@httpd] <directory> and deny directives
From: Joshua Slive <jo...@slive.ca>
To: users@httpd.apache.org
Date: Thursday, September 13, 2007 07:09:43 PM
> On 9/13/07, Mark A. Craig <ma...@gmail.com> wrote:
>
>> There's only one problem: it's not working! The log still shows visits
>> from these hostnames. What am I missing? Do I need to add "/*" to the
>> end of the <Directory> directive, or do subdirectories implicitly
>> inherit the same directives?
>
> The config is inherited.
>
> What status code is being reported for the accesses? If it is 403,
> then they are indeed being denied.
>
> Otherwise, show us a few access_log entries that you think should be denied.
>
> Also, check to make sure that the content is really living under
> C:/www/blog, that you are editing the right config file, that you are
> restarting apache after making config changes, and that you don't have
> anything else in your config file applying to that directory or lower.
>
> Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] and deny directives
Posted by Joshua Slive <jo...@slive.ca>.
On 9/13/07, Mark A. Craig <ma...@gmail.com> wrote:
> There's only one problem: it's not working! The log still shows visits
> from these hostnames. What am I missing? Do I need to add "/*" to the
> end of the <Directory> directive, or do subdirectories implicitly
> inherit the same directives?
The config is inherited.
What status code is being reported for the accesses? If it is 403,
then they are indeed being denied.
Otherwise, show us a few access_log entries that you think should be denied.
Also, check to make sure that the content is really living under
C:/www/blog, that you are editing the right config file, that you are
restarting apache after making config changes, and that you don't have
anything else in your config file applying to that directory or lower.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org