SpamCopURI and URIDNSBL vs. redirectors

[Burnie <bu...@dod.no>]

(I also mentioned this on sa-users)

Currently I get quite a few URIs which contains redirectors.
I.e. http://drs.yahoo.com/incomplete/*http://spammer/address
     http://g.msn.com/0US!s5.31472_315529/HP.1001?http://spammer/address

AFAIK URIDNSBL.pm (and SpamCopURI) will not trigger on the
spammers URI. - Even though the RBLs (mostly) will contain the
URI/IP of http://spammer/address.

Of course the above would be easy to parse to get the actual URIs, 
but what about other redirectors where the actual URI must be
looked up at the redirector itself? (Those who shortens the URI)

Should we run lookups to known redirectors to get the actual URI?
And how many times should we lookup, if the redirect is to
(another) redirector?

Any thoughts?
-- 
Bernt  'Burnie'  Pettersen  ///  DoD#2345  NAF
<E-...@dod.no>     ///  <URL:http://burnie.sh/>
 -  Yesterday was what you never did, today is what you aren't doing
            and tomorrow will be what you'll never do!  -  Burnie

Re: SpamCopURI and URIDNSBL vs. redirectors

[Jeff Chan <je...@surbl.org>]

On Monday, April 12, 2004, 4:58:11 PM, Burnie Burnie wrote:
> (I also mentioned this on sa-users)

> Currently I get quite a few URIs which contains redirectors.
> I.e. http://drs.yahoo.com/incomplete/*http://spammer/address
>      http://g.msn.com/0US!s5.31472_315529/HP.1001?http://spammer/address

> AFAIK URIDNSBL.pm (and SpamCopURI) will not trigger on the
> spammers URI. - Even though the RBLs (mostly) will contain the
> URI/IP of http://spammer/address.

> Of course the above would be easy to parse to get the actual URIs, 
> but what about other redirectors where the actual URI must be
> looked up at the redirector itself? (Those who shortens the URI)

> Should we run lookups to known redirectors to get the actual URI?
> And how many times should we lookup, if the redirect is to
> (another) redirector?

Definitely a good question.  Ideally mail handlers like SA that
want to parse message body URIs should somehow resolve the
redirections.  I just wrote a little about the topic on sa-users
and my site:

  http://www.surbl.org/

> Regarding the handling of redirection sites:

> Redirection sites like drs.yahoo.com, tinyurl.com, etc. take
> external URIs (unfortunately including spam ones) and redirect
> a browser to them. Therefore spammers can use the redirectors
> to trick a simple URI check that only looks at the initial URI,
> making the whole URI appear legitimate. For example the Yahoo
> redirection below might be (incorrectly) parsed as a legitimate
> yahoo domain:
> 
> http://drs.yahoo.com/covey/parr/*http://spammer.address/
> 
> SpamCop itself seems to disambiguate (most of) the redirection.
> If someone is using a redirector to send traffic to
> spamdomain.com, SpamCop seems to detect and resolve it
> correctly to spamdomain.com most of the time. So the data
> that's used as input to sc.surbl.org already has redirectors
> correctly handled to some extent. In other words, we're
> protected on the data input side by this processing which
> happens at SpamCop to take out the redirection in reported
> URIs.  
> 
> The SA code using sc.surbl.org such as SpamCopURI and urirhdbl
> may or may not be as capable of detecting and resolving the
> redirections. I can't really say for sure because I have not
> reviewed that code. My focus is on more the data side of
> things. Certainly it would be useful of the code handling
> messages coming in from the wild were able to resolve
> redirections fully, but I'm not sure that's currently the case.
> (If they don't, then they won't match the ahem, "unredirected"
> ones with get into sc.surbl.org via SpamCop.)   
> 
> The big picture solution is for the redirection sites to block
> spam domains on their own. In other words, they should not let
> spammers rediect through their sites. Until they do so, their
> services can be abused by spammers. 

Jeff C.
-- 
Jeff Chan
mailto:jeffc@surbl.org-nospam
http://www.surbl.org/