You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Cédric Jeanneret <cj...@tengu.ch> on 2010/11/05 08:02:09 UTC

pattern problem

 Hello,

 I have a lot of mail like this one: http://tengu.pastebin.com/PCyGxApn

 The problem is, SA doesn't seem to see it as a spam (I've included all 
 headers).

 I'm willing to create a filter for those nasty mails, but I have to 
 confess I'm not really sure on how to do it.

 It seems that a patern like /[a-z]+~[a-z]+~[a-z]+/i is used in each 
 mail - I have to dig in my spambox to confirm that.
 Maybe I can use the return-path, from and X-Originating-Email in 
 addition ?

 Any idea will be welcomed.

 Cheers,

 C.

Re: pattern problem

Posted by Martin Gregorie <ma...@gregorie.org>.
On Fri, 2010-11-05 at 09:02 +0200, Cédric Jeanneret wrote:
>  It seems that a patern like /[a-z]+~[a-z]+~[a-z]+/i is used in each 
>  mail - I have to dig in my spambox to confirm that.
>
>  Maybe I can use the return-path, from and X-Originating-Email in 
>  addition ?
> 
If they all contain: 
- at least one Return-Path, X-Originating-Email or From header 
  that references hotmail.com
- a url matching /\.com\/[\d\w]{6}/
- a recognisable body pattern like the one you quote

it should be safe to combine all these elements as a meta and give that
a score of around 3.5. Of course it goes without saying that the body
and uri patterns are the simplest that match the single sample e-mail
and may need to be elaborated.


Martin



Re: pattern problem

Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
> On Tue, 9 Nov 2010 12:48:52 +0100
> Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:
> > can IP addresses in PBL get onto XBL? 

On 10.11.10 08:46, RW wrote:
> Yes, some servers reject on XBL, but not on PBL.

I was more interested iun spamhaus practice.
And not rejecting mail from PBL at SMTP level is imho strange...

-- 
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
99 percent of lawyers give the rest a bad name. 

Re: pattern problem

Posted by RW <rw...@googlemail.com>.
On Tue, 9 Nov 2010 12:48:52 +0100
Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:

> > > On 05.11.10 12:28, RW wrote:
> > > > It's not all that safe to deep-parse XBL because it's mostly
> > > > dynamically assigned IP addresses.
> 
> > On Fri, 5 Nov 2010 14:38:45 +0100
> > Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:
> > > the PBL from ZEN is mostly dynamically assigned.
> > > the XBL means exploits block list, and doing deep-parsing on it
> > > should be really safe.
> 
> On 05.11.10 14:47, RW wrote:
> > Most XBL entries are compromised PCs, most of which have  dynamic
> > addresses which can be reassigned. On last-external that doesn't
> > matter. The FP rate for deep XBL is, in general, much higher. 
> 
> can IP addresses in PBL get onto XBL? 

Yes, some servers reject on XBL, but not on PBL.

> 
> I've seen DCC_CHECK hit on some kind of funny story being massivelly
> sent over the net. They were bulk and they would apparently hit PBL
> if anyone sent them from such IP. Haven't you encountered these?

Not recently and it's only adding 1 point to the 1.1 from DCC.

Re: pattern problem

Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
> > On 05.11.10 12:28, RW wrote:
> > > It's not all that safe to deep-parse XBL because it's mostly
> > > dynamically assigned IP addresses.

> On Fri, 5 Nov 2010 14:38:45 +0100
> Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:
> > the PBL from ZEN is mostly dynamically assigned.
> > the XBL means exploits block list, and doing deep-parsing on it
> > should be really safe.

On 05.11.10 14:47, RW wrote:
> Most XBL entries are compromised PCs, most of which have  dynamic
> addresses which can be reassigned. On last-external that doesn't
> matter. The FP rate for deep XBL is, in general, much higher. 

can IP addresses in PBL get onto XBL? If not, it's even more important to IP
range admins to correctly assign ranges to PBL...

> I'm not saying it shouldn't be used, but it needs to be used with care.

of course

> > the PBL should NOT be used for deep parsing.
> 
> As I said, I only use it in meta-rules. Clearly it's not a direct-to-mx
> test beyond the last-external, but the information is still useful. For
> example I get a lot of FPs on DCC_CHECK, but zero FPs on "DCC_CHECK &&
> DEEP_PBL"

I've seen DCC_CHECK hit on some kind of funny story being massivelly sent
over the net. They were bulk and they would apparently hit PBL if anyone
sent them from such IP. Haven't you encountered these?

-- 
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"One World. One Web. One Program." - Microsoft promotional advertisement
"Ein Volk, ein Reich, ein Fuhrer!" - Adolf Hitler

Re: pattern problem

Posted by RW <rw...@googlemail.com>.
On Fri, 5 Nov 2010 14:38:45 +0100
Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:

> > On Fri, 05 Nov 2010 11:54:08 +0000
> > Ned Slider <ne...@unixmail.co.uk> wrote:
> > > are safe to do so. You should be safe to deep parse against
> > > Spamhaus SBL and XBL but not PBL. This instance hits against XBL
> > > and many others.
> 
> On 05.11.10 12:28, RW wrote:
> > It's not all that safe to deep-parse XBL because it's mostly
> > dynamically assigned IP addresses.
> 
> the PBL from ZEN is mostly dynamically assigned.
> the XBL means exploits block list, and doing deep-parsing on it
> should be really safe.

Most XBL entries are compromised PCs, most of which have  dynamic
addresses which can be reassigned. On last-external that doesn't
matter. The FP rate for deep XBL is, in general, much higher. 

I'm not saying it shouldn't be used, but it needs to be used with care.

> > It's safer if you don't get much
> > mail from such addresses, or it comes from ISPs that use very sticky
> > addresses, which is commoner in the UK. SBL is already deep because
> > it's based on IP addesses owned by spammers.
> > 
> > I do use deep PBL checks in meta rules. e.g. mail from banks is
> > unlikely to contain such IP addresses. 
> 
> the PBL should NOT be used for deep parsing.

As I said, I only use it in meta-rules. Clearly it's not a direct-to-mx
test beyond the last-external, but the information is still useful. For
example I get a lot of FPs on DCC_CHECK, but zero FPs on "DCC_CHECK &&
DEEP_PBL"

Re: pattern problem

Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
> On Fri, 05 Nov 2010 11:54:08 +0000
> Ned Slider <ne...@unixmail.co.uk> wrote:
> > are safe to do so. You should be safe to deep parse against Spamhaus
> > SBL and XBL but not PBL. This instance hits against XBL and many
> > others.

On 05.11.10 12:28, RW wrote:
> It's not all that safe to deep-parse XBL because it's mostly
> dynamically assigned IP addresses.

the PBL from ZEN is mostly dynamically assigned.
the XBL means exploits block list, and doing deep-parsing on it should be
really safe.

> It's safer if you don't get much
> mail from such addresses, or it comes from ISPs that use very sticky
> addresses, which is commoner in the UK. SBL is already deep because
> it's based on IP addesses owned by spammers.
> 
> I do use deep PBL checks in meta rules. e.g. mail from banks is
> unlikely to contain such IP addresses. 

the PBL should NOT be used for deep parsing.

-- 
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"The box said 'Requires Windows 95 or better', so I bought a Macintosh".

Re: pattern problem

Posted by RW <rw...@googlemail.com>.
On Fri, 05 Nov 2010 11:54:08 +0000
Ned Slider <ne...@unixmail.co.uk> wrote:


> are safe to do so. You should be safe to deep parse against Spamhaus
> SBL and XBL but not PBL. This instance hits against XBL and many
> others.
 
It's not all that safe to deep-parse XBL because it's mostly
dynamically assigned IP addresses. It's safer if you don't get much
mail from such addresses, or it comes from ISPs that use very sticky
addresses, which is commoner in the UK. SBL is already deep because
it's based on IP addesses owned by spammers.

I do use deep PBL checks in meta rules. e.g. mail from banks is
unlikely to contain such IP addresses. 



I notice that the quoted mail contains an invalid line in the header
part, the one that starts with "Bossythat", I'm surprised that didn't
trigger something: 

X-OriginalArrivalTime: 05 Nov 2010 06:40:00.0453 (UTC)
  FILETIME=[450B7F50:01CB7CB4] Sender: <st...@hotmail.com>
BossyThat SheIs She~Can~Go WildSometimes mpmffxwf.
 
 
http://itshrunk.com/88a5da


Re: pattern problem

Posted by Ned Slider <ne...@unixmail.co.uk>.
On 05/11/10 07:02, Cédric Jeanneret wrote:
> Hello,
>
> I have a lot of mail like this one: http://tengu.pastebin.com/PCyGxApn
>
> The problem is, SA doesn't seem to see it as a spam (I've included all
> headers).
>
> I'm willing to create a filter for those nasty mails, but I have to
> confess I'm not really sure on how to do it.
>
> It seems that a patern like /[a-z]+~[a-z]+~[a-z]+/i is used in each mail
> - I have to dig in my spambox to confirm that.
> Maybe I can use the return-path, from and X-Originating-Email in addition ?
>
> Any idea will be welcomed.
>
> Cheers,
>
> C.

Scanning that mail gives me hits against the X-Originating-IP. I deep 
parse headers against some DNSBLs but be careful as not all DNSBLs are 
safe to do so. You should be safe to deep parse against Spamhaus SBL and 
XBL but not PBL. This instance hits against XBL and many others.

++++++++++++++++++++++++++++++++++++++++++++++++++++
RUNNING CHECKS FOR 91.203.67.35
++++++++++++++++++++++++++++++++++++++++++++++++++++
==========================
CHECKING DNSBL BLACK LISTS
==========================
91.203.67.35    BLACKLISTED:    zen.spamhaus.org
91.203.67.35    UNLISTED:       sbl.spamhaus.org
91.203.67.35    BLACKLISTED:    pbl.spamhaus.org
91.203.67.35    BLACKLISTED:    xbl.spamhaus.org
91.203.67.35    BLACKLISTED:    cbl.abuseat.org
91.203.67.35    BLACKLISTED:    b.barracudacentral.org
91.203.67.35    UNLISTED:       dnsbl.njabl.org
91.203.67.35    BLACKLISTED:    dnsbl-1.uceprotect.net
91.203.67.35    BLACKLISTED:    dnsbl-2.uceprotect.net
91.203.67.35    BLACKLISTED:    dnsbl-3.uceprotect.net
91.203.67.35    UNLISTED:       dnsbl.sorbs.net
91.203.67.35    BLACKLISTED:    spam.dnsbl.sorbs.net
91.203.67.35    UNLISTED:       aspews.ext.sorbs.net
91.203.67.35    UNLISTED:       l1.apews.org
91.203.67.35    BLACKLISTED:    l2.apews.org
91.203.67.35    UNLISTED:       bl.spamcop.net
91.203.67.35    UNLISTED:       bl.spamcannibal.org
91.203.67.35    UNLISTED:       no-more-funn.moensted.dk
91.203.67.35    UNLISTED:       ips.backscatterer.org
91.203.67.35    BLACKLISTED:    hostkarma.junkemailfilter.com
91.203.67.35    BLACKLISTED:    bl.score.senderscore.com
.........................................................................

The url is also listed in URIBL_BLACK now but might not have been at the 
time it was received by you.

I've also got so fed up with this type of spam from hotmail.com 
accounts, I score ALL mail from hotmail.com with 3 points and then 
whitelist legitimate hotmail.com users with -3 points. I'm a small 
volume server with only a few accounts so it's easy for me to identify 
and whitelist regular hotmail senders. Then with only 2 points to play 
with mail from hotmail.com needs to be squeaky clean to get through. 
This works well for me and doesn't seem to cause any additional burden 
with false positives.

Also keep learning these through bayes as I've found that to be highly 
effective.