You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Gene <li...@Bomgardner.net> on 2005/12/05 16:36:46 UTC
[users@httpd] Out of control Referrer spam; I'm puzzled.
Hi All:
I don't know about everyone else, but Referrer Spam on my site has
reached epic proportions. I've been reading up on it, but one thing
still puzzles me.
If the referrer url is phony and the ip address from which it came is
phony, how does that benefit the spammer? It seems to me that the url
that got posted to a stats page would simply be a dead end.
Moving on - Since a lot of the spam urls have certain things in common
(ie., "porn", "foto", etc.) I can return an error code to a lot of
spammers via mod_security, but of course, the spammers don't care. They
wind up in the logs anyway. This raises some questions.
1. Is it possible to redirect the http request back to the referrer url,
thus letting spammers with valid urls spam themselves?
2. Is it possible, using the commonalities above, to prevent them from
getting into the log file? Can apache simply drop the request making it
appear there is no server running?
3. Can the referrer url be validated before accepting the http request?
4. Since my page shouldn't be linked to from anywhere else, except maybe
search engines, is it possible to create a sort of whitelist of referrer
urls and reject all others (letting the request drop into a black hole) ?
5. How about an rbl for referrer spam? (or did I see that somewhere
already. I don't remember.)
Anyway, anyone find a way to deal with it that at least reduces the
spam? My ISP graciously allows me to run a server as a hobby, but if
this spam keeps increasing, they're liable to consider me a commercial
enterprise.
Maybe we should get a thread going and see what we come up with (or was
there one that I missed?)
Any thoughts appreciated.
Regards,
Gene
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Out of control Referrer spam; I'm puzzled.
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
You may be running applications which are vulnerable to cross site
scripting vulnerabilities, using your domain at the 'common site'.
Suggest you look over vulnerabilities, particularly in any PHP apps
you are running.
Why would you want them lost from the log file? Feel free to filter
your log file any-which-way after it's been written. There are various
tricks to filter it, but it's questionable if you could write a pattern
match that mod_rewrite+mod_setenvif could use to deflect all of this
noise.
Bill
Gene wrote:
> Hi All:
>
> I don't know about everyone else, but Referrer Spam on my site has
> reached epic proportions. I've been reading up on it, but one thing
> still puzzles me.
>
> If the referrer url is phony and the ip address from which it came is
> phony, how does that benefit the spammer? It seems to me that the url
> that got posted to a stats page would simply be a dead end.
>
> Moving on - Since a lot of the spam urls have certain things in common
> (ie., "porn", "foto", etc.) I can return an error code to a lot of
> spammers via mod_security, but of course, the spammers don't care. They
> wind up in the logs anyway. This raises some questions.
>
> 1. Is it possible to redirect the http request back to the referrer url,
> thus letting spammers with valid urls spam themselves?
>
> 2. Is it possible, using the commonalities above, to prevent them from
> getting into the log file? Can apache simply drop the request making it
> appear there is no server running?
>
> 3. Can the referrer url be validated before accepting the http request?
>
> 4. Since my page shouldn't be linked to from anywhere else, except maybe
> search engines, is it possible to create a sort of whitelist of referrer
> urls and reject all others (letting the request drop into a black hole) ?
>
> 5. How about an rbl for referrer spam? (or did I see that somewhere
> already. I don't remember.)
>
> Anyway, anyone find a way to deal with it that at least reduces the
> spam? My ISP graciously allows me to run a server as a hobby, but if
> this spam keeps increasing, they're liable to consider me a commercial
> enterprise.
>
> Maybe we should get a thread going and see what we come up with (or was
> there one that I missed?)
>
> Any thoughts appreciated.
>
> Regards,
> Gene
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Out of control Referrer spam; I'm puzzled.
Posted by Joshua Slive <js...@gmail.com>.
On 12/5/05, Gene <li...@bomgardner.net> wrote:
> Hi All:
>
> I don't know about everyone else, but Referrer Spam on my site has
> reached epic proportions. I've been reading up on it, but one thing
> still puzzles me.
>
> If the referrer url is phony and the ip address from which it came is
> phony, how does that benefit the spammer? It seems to me that the url
> that got posted to a stats page would simply be a dead end.
>
I haven't checked myself, but I would guess that in general the
referer urls are not phony. You may just have hit some cases where
the site was shut down before you got to it.
> 1. Is it possible to redirect the http request back to the referrer url,
> thus letting spammers with valid urls spam themselves?
There's no point in that. Their client is highly unlikely to follow reidrects.
>
> 2. Is it possible, using the commonalities above, to prevent them from
> getting into the log file? Can apache simply drop the request making it
> appear there is no server running?
Yes, you can prevent them from getting into the log files. See the
conditional logging section of
http://httpd.apache.org/docs/2.2/logs.html#accesslog
>
> 3. Can the referrer url be validated before accepting the http request?
Theoretically yes. But this would slow down your site! And I know of
no way to actually impliment this with current modules.
>
> 4. Since my page shouldn't be linked to from anywhere else, except maybe
> search engines, is it possible to create a sort of whitelist of referrer
> urls and reject all others (letting the request drop into a black hole) ?
Yes, see the "prevent image theft" example here:
http://httpd.apache.org/docs/2.2/env.html#examples
> Anyway, anyone find a way to deal with it that at least reduces the
> spam? My ISP graciously allows me to run a server as a hobby, but if
> this spam keeps increasing, they're liable to consider me a commercial
> enterprise.
In general, you should just be ignoring this stuff, with two possible
exceptions:
- Make sure you don't post your logfiles to let them benefit from the spam.
- If they are hitting very large pages, you can save some bandwidth
and server load by blocking them using one of the techniques mentioned
above.
But there is no way to prevent the requests entirely other than
finding the computer that is doing it and unplugging it from the
Internet.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org