You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by "Ruth, Brice D" <BR...@amfam.com> on 2006/09/22 16:03:48 UTC

[users@httpd] authenticating robots by user-agent & double reverse lookup

Good morning.
 
After reading the latest Google Webmaster blog entry
(http://googlewebmastercentral.blogspot.com/2006/09/how-to-verify-google
bot.html) I got to thinking about how one might be able to configure
Apache deny visits from bots that purport to be Googlebot (or anything
else) but do not pass the double reverse lookup for googlebot.com.
 
Reading the docs, it looks like mod_access will do a double reverse
lookup on something like:
 
Allow from googlebot.com
 
And mod_access can also respect env variables set via
 
SetEnvIf User-Agent Googlebot
 
But, how would a webmaster go about combining these to stop scammers?
 
Thanks for any tips!
 
-Brice