Fwd: Re: [satalk] [Razor-users] Poor detection ratio

[Jeff Chan <je...@surbl.org>]

Forwarded from spamassassin-users, pardon the funky quoting;
messages in chronological order, top to bottom

Date: Thursday, April 22, 2004, 4:57:20 PM
Subject: [satalk] [Razor-users] Poor detection ratio

>>Gilbert, Joseph wrote:
>>> There are a number of tactics of URL obfuscation that could easily
>>> permanently kill a filter that was totally reliant on scanning urls.
>>>
>>> Third off, there are a significant number of ways to obfuscate/encode
>>> URLs. Commonly, most spam still uses a straight hostname based URL
>>> which keeps sburl pretty effective.  However, it is also feasible
>>> that more and more spammers will use a legitimate looking text
>>> display for a link and use an encoded URL, not visible to the average
>>> user, within the A HREF tag.

>On Thu, 22 Apr 2004, Mark wrote:
>> True. But that obfuscation comes at a price: the obfuscation. :) Seriously,
>> obfuscating a URL is branding yourself a spammer, in much the same manner
>> that writing "v1agra" is a sure giveaway of your malicious intent.
>>
>> In fact, obfuscating a URL, where no such obfuscation is required, will
>> actually make it easier for anti-spam detection tools to weed them out.
>> These obfuscated URLs carry, as it were, a spam-signature which transcends
>> the actual URL. Which means you can detect a spam URI, regardless of its
>> dereferenced location even: the obfuscation itself is evidence of spam.
>>
>> Granted, SURBL does not yet, to my knowledge, deploy such tests. I highly
>> recommend they do, though. If, and when, they do, SURBL will become
>> unbeatable: either because of matching spammy domain names directly, or
>> through detecting unnecessary obfuscation. Either way, the spammer loses.

On Thursday, April 22, 2004, 11:28:19 AM, Charles Gregory wrote:
> SURBL uses a very simple 'DNS' style lookup, which is very efficient. I
> don't believe they do any front-end processing. So I would suggest that
> resolving obfuscation (hex codes, etc) should be done by SA before it
> feeds the domain to SURBL.....

To put a finer point on this, if spammers start using a lot of
URI obfuscation, as Mark suggests, this should be detectable as
some specific signatures in SA.  Detecting those other signatures
would be something other code in SA would do, but any URI that
actually popped out of the obfuscated spam could still be checked
against an SURBL.

The deobfuscation part could require some additional SA coding in
the parts that check against SURBLs, i.e. urirhsbl in URIDNSBL
or SpamCopURI.  And obfuscation signature detection code would
presumably be some separate code.  All could work together or
separately. 

Jeff C.

Re: Fwd: Re: [satalk] [Razor-users] Poor detection ratio

[Jeff Chan <je...@surbl.org>]

>> On Thu, 22 Apr 2004, Mark wrote:
>> > Granted, SURBL does not yet, to my knowledge, deploy such tests. I highly
>> > recommend they do, though. If, and when, they do, SURBL will become
>> > unbeatable: either because of matching spammy domain names directly, or
>> > through detecting unnecessary obfuscation. Either way, the spammer loses.

Dooh!  I missed Chris Santerre's response:

> With weeds.cf and chickenpox.cf why even bother? Those files grab most of
> the obfuscation tricks. I see no reason to even try using an RBL when
> obfuscation is involved. The more they try to obfuscate, the easier it is to
> tag. 
> 
> I almost forgot about Fred's Obfu1.cf file!! That there is slick as well!
> (SA 2.50 or higher required.)

Jeff C.