Re: SpamCopURI and URIDNSBL vs. redirectors

[Justin Mason <jm...@jmason.org>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Jeff Chan writes:
> On Monday, April 12, 2004, 4:58:11 PM, Burnie Burnie wrote:
> > (I also mentioned this on sa-users)
> 
> > Currently I get quite a few URIs which contains redirectors.
> > I.e. http://drs.yahoo.com/incomplete/*http://spammer/address
> >      http://g.msn.com/0US!s5.31472_315529/HP.1001?http://spammer/address
> 
> > AFAIK URIDNSBL.pm (and SpamCopURI) will not trigger on the
> > spammers URI. - Even though the RBLs (mostly) will contain the
> > URI/IP of http://spammer/address.
> 
> > Of course the above would be easy to parse to get the actual URIs, 
> > but what about other redirectors where the actual URI must be
> > looked up at the redirector itself? (Those who shortens the URI)
> 
> > Should we run lookups to known redirectors to get the actual URI?
> > And how many times should we lookup, if the redirect is to
> > (another) redirector?
> 
> Definitely a good question.  Ideally mail handlers like SA that
> want to parse message body URIs should somehow resolve the
> redirections.  I just wrote a little about the topic on sa-users
> and my site:
> 
>   http://www.surbl.org/

I think I agree.  We already translate %-escapes and return both the
encoded and unencoded URIs -- we should probably also strip off known
redirectors, as their use will become more popular.

Could you open a bug?

- --j.


> > Regarding the handling of redirection sites:
> 
> > Redirection sites like drs.yahoo.com, tinyurl.com, etc. take
> > external URIs (unfortunately including spam ones) and redirect
> > a browser to them. Therefore spammers can use the redirectors
> > to trick a simple URI check that only looks at the initial URI,
> > making the whole URI appear legitimate. For example the Yahoo
> > redirection below might be (incorrectly) parsed as a legitimate
> > yahoo domain:
> > 
> > http://drs.yahoo.com/covey/parr/*http://spammer.address/
> > 
> > SpamCop itself seems to disambiguate (most of) the redirection.
> > If someone is using a redirector to send traffic to
> > spamdomain.com, SpamCop seems to detect and resolve it
> > correctly to spamdomain.com most of the time. So the data
> > that's used as input to sc.surbl.org already has redirectors
> > correctly handled to some extent. In other words, we're
> > protected on the data input side by this processing which
> > happens at SpamCop to take out the redirection in reported
> > URIs.  
> > 
> > The SA code using sc.surbl.org such as SpamCopURI and urirhdbl
> > may or may not be as capable of detecting and resolving the
> > redirections. I can't really say for sure because I have not
> > reviewed that code. My focus is on more the data side of
> > things. Certainly it would be useful of the code handling
> > messages coming in from the wild were able to resolve
> > redirections fully, but I'm not sure that's currently the case.
> > (If they don't, then they won't match the ahem, "unredirected"
> > ones with get into sc.surbl.org via SpamCop.)   
> > 
> > The big picture solution is for the redirection sites to block
> > spam domains on their own. In other words, they should not let
> > spammers rediect through their sites. Until they do so, their
> > services can be abused by spammers. 
> 
> Jeff C.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFAezu4QTcbUG5Y7woRAuXmAJ9IGAiYgcRvLjf7OJsU6NNoqRcmTwCgnvKr
dg+34KBDJ5AP/Q45cbNrIVM=
=4oMu
-----END PGP SIGNATURE-----

Re: SpamCopURI and URIDNSBL vs. redirectors

[Jeff Chan <je...@surbl.org>]

On Monday, April 12, 2004, 6:00:40 PM, Justin Mason wrote:
> Jeff Chan writes:
>> On Monday, April 12, 2004, 4:58:11 PM, Burnie Burnie wrote:

>> > Currently I get quite a few URIs which contains redirectors.
>> > I.e. http://drs.yahoo.com/incomplete/*http://spammer/address
>> >      http://g.msn.com/0US!s5.31472_315529/HP.1001?http://spammer/address
>> 
>> > AFAIK URIDNSBL.pm (and SpamCopURI) will not trigger on the
>> > spammers URI. - Even though the RBLs (mostly) will contain the
>> > URI/IP of http://spammer/address.
>> 
>> > Of course the above would be easy to parse to get the actual URIs, 
>> > but what about other redirectors where the actual URI must be
>> > looked up at the redirector itself? (Those who shortens the URI)
>> 
>> > Should we run lookups to known redirectors to get the actual URI?
>> > And how many times should we lookup, if the redirect is to
>> > (another) redirector?
>> 
>> Definitely a good question.  Ideally mail handlers like SA that
>> want to parse message body URIs should somehow resolve the
>> redirections.  I just wrote a little about the topic on sa-users
>> and my site:
>> 
>>   http://www.surbl.org/

> I think I agree.  We already translate %-escapes and return both the
> encoded and unencoded URIs -- we should probably also strip off known
> redirectors, as their use will become more popular.

> Could you open a bug?

> - --j.

Done:

  http://bugzilla.spamassassin.org/show_bug.cgi?id=3261

This is my first bugzilla.   Feedback appreciated.    :-)

Jeff C.
-- 
Jeff Chan
mailto:jeffc@surbl.org-nospam
http://www.surbl.org/